2024-09-10T22:26:49.743 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T22:26:49.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:26:49.878 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899739' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1+FllIQUCzBTAUDbc5haysX1AcKRuBl56Etmo42SXB3syXiSn4zeT4Pt/6S5u+b5k4NgteNyOh4RiRF1uPqgI= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGwf721c3BYoY50y1qemPNytcYRTZ7x4RG5GBV9WMxePsL/YThXBBvSTI70ShrDG3QiVOi4Kag5Tt2JEKtd158= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJkgVmQnirqvmTNGdDawVCYVwlJmX+sAIjoayrLP6R4AWVbIfLE9rBM4oEi+VLvjXioFdKB/o1dDC6bfPjLiy0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T22:26:49.878 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T22:26:49.879 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T22:26:49.879 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T22:26:49.880 INFO:teuthology.task.internal:Checking packages... 2024-09-10T22:26:49.902 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T22:26:49.902 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T22:26:49.902 INFO:teuthology.packaging:ref: None 2024-09-10T22:26:49.902 INFO:teuthology.packaging:tag: None 2024-09-10T22:26:49.903 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:26:49.903 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:26:49.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:26:50.069 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T22:26:50.071 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T22:26:50.077 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T22:26:50.077 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T22:26:50.099 INFO:teuthology.task.internal:Saving configuration 2024-09-10T22:26:50.121 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T22:26:50.127 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T22:26:50.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:21:53.667965', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1+FllIQUCzBTAUDbc5haysX1AcKRuBl56Etmo42SXB3syXiSn4zeT4Pt/6S5u+b5k4NgteNyOh4RiRF1uPqgI='} 2024-09-10T22:26:50.195 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:21:53.665961', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGwf721c3BYoY50y1qemPNytcYRTZ7x4RG5GBV9WMxePsL/YThXBBvSTI70ShrDG3QiVOi4Kag5Tt2JEKtd158='} 2024-09-10T22:26:50.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:21:53.669700', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJkgVmQnirqvmTNGdDawVCYVwlJmX+sAIjoayrLP6R4AWVbIfLE9rBM4oEi+VLvjXioFdKB/o1dDC6bfPjLiy0='} 2024-09-10T22:26:50.229 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T22:26:50.236 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T22:26:50.237 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T22:26:50.237 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T22:26:50.237 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T22:26:50.367 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f391c1e3be0>, signals=[15]) 2024-09-10T22:26:50.367 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T22:26:50.373 INFO:teuthology.task.internal:Opening connections... 2024-09-10T22:26:50.373 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:26:50.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:26:50.483 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T22:26:50.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:26:50.591 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T22:26:50.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:26:50.678 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T22:26:50.685 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-10T22:26:50.703 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-10T22:26:50.703 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-09-10T22:26:50.761 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:26:50.762 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:26:50.763 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-10T22:26:50.789 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-10T22:26:50.808 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-10T22:26:50.808 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-10T22:26:50.866 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-10T22:26:50.866 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-10T22:26:50.866 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-10T22:26:50.866 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:26:50.866 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:26:50.867 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:26:50.868 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-10T22:26:50.897 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-09-10T22:26:50.916 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-09-10T22:26:50.916 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-09-10T22:26:50.973 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:26:50.974 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:26:50.975 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-09-10T22:26:51.005 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T22:26:51.015 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T22:26:51.032 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T22:26:51.033 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:26:51.035 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:26:51.037 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:26:51.054 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T22:26:51.059 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T22:26:51.059 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:26:51.092 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:26:51.095 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:26:51.191 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T22:26:51.293 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T22:26:51.307 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T22:26:51.308 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T22:26:51.308 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T22:26:51.308 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:26:51.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:26:51.326 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-10T22:26:51.384 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:26:51.384 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-503.el9.x86_64 2024-09-10T22:26:51.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-09-10T22:26:56.200 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-10T22:27:10.856 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 20 MB 00:13 2024-09-10T22:27:16.098 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-10T22:27:17.723 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-10T22:27:18.853 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T22:27:24.250 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T22:27:26.314 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T22:27:26.314 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T22:27:26.397 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T22:27:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 79 M 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 126 M 2024-09-10T22:27:26.410 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-10T22:27:26.559 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 239 kB/s | 23 kB 00:00 2024-09-10T22:27:29.001 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-10T22:27:31.060 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.3 MB/s | 29 MB 00:04 2024-09-10T22:27:31.667 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 35 MB 00:05 2024-09-10T22:27:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:27:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-10T22:27:32.353 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-10T22:27:32.728 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-10T22:27:32.728 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-10T22:27:33.691 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-10T22:27:33.691 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-10T22:27:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-10T22:27:36.871 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:27:36.961 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:27:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:27:39.879 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:27:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:27:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:27:49.556 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:28:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:28:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:28:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:28:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:28:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:28:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:28:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:28:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:28:29.234 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-10T22:28:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:28:29.235 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-10T22:28:29.634 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:28:29.716 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.716 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:28:29.717 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T22:28:29.717 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T22:28:29.717 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T22:28:29.717 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T22:28:29.717 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T22:28:29.717 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:28:29.777 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:28:29.778 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:28:29.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.06 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 82.el9 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-09-10T22:28:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Group : Unspecified 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 8274670 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : builder@centos.org 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:28:29.813 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:28:29.814 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:28:29.815 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:28:29.815 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 5.14.0-505.el9.x86_64 2024-09-10T22:28:29.815 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:28:30.613 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-09-10T22:28:31.996 INFO:teuthology.orchestra.run.smithi096.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:28:32.022 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-09-10T22:28:32.029 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-10T22:28:32.045 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.FQZ0s6QPbL 2024-09-10T22:28:32.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FQZ0s6QPbL 2024-09-10T22:28:32.111 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.FQZ0s6QPbL 2024-09-10T22:28:32.256 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.FQZ0s6QPbL is 6KB 2024-09-10T22:28:32.304 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.FQZ0s6QPbL 2024-09-10T22:28:32.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:28:32.382 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:28:32.382 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:28:32.382 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:28:32.382 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T22:28:32.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T22:28:32.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-09-10T22:29:02.563 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T22:29:02.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-10T22:29:02.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:30:02.576 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T22:30:11.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-10T22:30:11.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:30:12.026 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-10T22:30:12.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-10T22:30:12.534 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T22:30:12.534 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-10T22:30:12.549 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-505.el9.x86_64 2024-09-10T22:30:12.549 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T22:30:12.549 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:30:12.549 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:30:13.550 DEBUG:teuthology.parallel:result is None 2024-09-10T22:30:13.551 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T22:30:13.559 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T22:30:13.559 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:30:13.562 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:30:13.565 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:30:13.584 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T22:30:13.599 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T22:30:13.676 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T22:30:13.677 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:30:13.679 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:30:13.681 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:30:13.807 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T22:30:13.815 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T22:30:13.815 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:30:13.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:30:13.832 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:30:13.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:30:13.850 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:30:13.852 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:30:13.875 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:30:13.926 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.930 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.948 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.959 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.962 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.981 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:30:13.983 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T22:30:13.991 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T22:30:13.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:30:13.993 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:30:13.995 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:30:14.023 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T22:30:14.033 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T22:30:14.033 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:30:14.058 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:30:14.064 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:30:14.080 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.203 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.293 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.326 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:30:14.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:30:14.391 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.491 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.582 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.618 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:30:14.618 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:30:14.684 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:30:14.783 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:30:14.911 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:30:14.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:30:14.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-10T22:30:14.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-10T22:30:14.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-09-10T22:30:15.020 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:30:15.022 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:30:15.061 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:30:15.553 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T22:30:15.561 INFO:teuthology.task.internal:Starting timer... 2024-09-10T22:30:15.561 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T22:30:15.610 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T22:30:15.642 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T22:30:15.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-09-10T22:30:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-09-10T22:30:15.712 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-10T22:30:15.767 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-10T22:30:15.768 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-09-10T22:30:15.816 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-09-10T22:30:15.818 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T22:30:15.818 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-09-10T22:30:15.846 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-09-10T22:30:15.847 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-10T22:30:15.871 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-10T22:30:15.871 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-09-10T22:30:15.893 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-09-10T22:30:15.893 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-09-10T22:30:15.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:30:15.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:30:15.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:30:15.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:30:15.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:30:15.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:30:15.974 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T22:30:15.974 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:30:16.073 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:30:16.102 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:30:16.144 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T22:30:16.154 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T22:30:16.166 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T22:30:16.167 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi117.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T22:34:22.902 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-09-10T22:34:22.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-10T22:34:22.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:34:22.996 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-10T22:34:23.092 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-10T22:34:23.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T22:34:23.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:34:23.178 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-10T22:34:23.253 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T22:34:23.254 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T22:34:23.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:34:23.344 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-09-10T22:34:23.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T22:34:23.427 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T22:34:23.442 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T22:34:23.443 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:34:23.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:34:23.445 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:34:23.446 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:34:23.448 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:34:23.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:34:23.475 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:34:23.477 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:34:23.490 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:34:23.492 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:34:23.500 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:34:23.515 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:34:23.516 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-09-10T22:34:23.518 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-10T22:34:23.527 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-09-10T22:34:23.530 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-10T22:34:23.541 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-09-10T22:34:23.541 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:34:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:34:23.553 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-09-10T22:34:23.556 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:34:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:34:23.568 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:34:23.582 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:34:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:34:23.619 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:34:23.620 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:34:23.620 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-10T22:34:23.620 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.621 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.621 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.621 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.623 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.636 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.640 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:34:23.641 INFO:teuthology.run_tasks:Running task install... 2024-09-10T22:34:23.652 DEBUG:teuthology.task.install:project ceph 2024-09-10T22:34:23.652 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T22:34:23.652 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T22:34:23.652 INFO:teuthology.task.install:Using flavor: default 2024-09-10T22:34:23.658 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T22:34:23.658 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T22:34:23.658 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:34:23.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:34:23.661 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:34:23.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:34:23.662 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:34:23.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:34:23.819 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:34:23.819 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:34:23.833 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:34:23.833 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:34:23.834 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:34:23.834 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:34:23.941 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:34:23.941 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:34:23.941 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:34:23.956 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:34:23.956 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:34:23.956 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:34:23.961 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:34:23.961 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:34:23.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:34:23.968 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:34:23.968 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:34:23.982 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:34:23.983 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:34:23.988 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:34:23.989 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:34:24.034 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:34:24.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:34:24.055 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:34:24.109 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:34:24.126 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:34:24.130 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:34:24.173 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-09-10T22:34:24.175 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-09-10T22:34:24.190 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-09-10T22:34:24.191 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-09-10T22:34:24.194 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-09-10T22:34:24.195 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-09-10T22:34:24.398 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-09-10T22:34:24.419 INFO:teuthology.orchestra.run.smithi121.stdout:45 files removed 2024-09-10T22:34:24.420 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-09-10T22:34:24.425 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:34:24.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:34:24.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:34:24.963 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-10T22:34:24.989 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-10T22:34:25.000 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-10T22:34:25.263 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T22:34:25.284 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-10T22:34:25.297 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 90 kB/s | 24 kB 00:00 2024-09-10T22:34:25.539 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-10T22:34:25.559 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T22:34:25.581 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-10T22:34:26.173 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-10T22:34:26.664 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.2 MB 00:01 2024-09-10T22:34:29.040 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-10T22:34:29.674 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 20 MB 00:02 2024-09-10T22:34:30.527 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.2 MB 00:04 2024-09-10T22:34:33.892 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-10T22:34:34.443 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 20 MB 00:02 2024-09-10T22:34:35.492 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-10T22:34:36.539 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T22:34:38.744 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-10T22:34:40.378 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-10T22:34:40.607 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 855 kB/s | 6.5 MB 00:07 2024-09-10T22:34:41.890 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-10T22:34:41.935 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T22:34:42.073 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-10T22:34:43.027 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T22:34:44.220 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:44.221 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:44.232 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:34:44.233 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:34:44.233 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:34:44.234 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:34:44.236 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:34:44.238 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:34:44.239 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:34:44.301 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:34:44.320 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:34:44.321 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:34:44.322 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:34:44.323 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:34:44.324 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:34:44.325 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:34:44.326 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:34:44.327 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:34:44.328 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:34:44.329 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:34:44.330 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:34:44.331 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:34:44.332 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout:Install 149 Packages 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 8 Packages 2024-09-10T22:34:44.333 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:34:44.334 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 240 M 2024-09-10T22:34:44.334 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T22:34:44.945 INFO:teuthology.orchestra.run.smithi121.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-10T22:34:45.204 INFO:teuthology.orchestra.run.smithi121.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.4 MB/s | 900 kB 00:00 2024-09-10T22:34:45.287 INFO:teuthology.orchestra.run.smithi121.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.7 MB/s | 145 kB 00:00 2024-09-10T22:34:45.379 INFO:teuthology.orchestra.run.smithi121.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 9.7 MB/s | 5.5 MB 00:00 2024-09-10T22:34:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 14 MB/s | 1.5 MB 00:00 2024-09-10T22:34:45.705 INFO:teuthology.orchestra.run.smithi121.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 5.7 MB/s | 2.4 MB 00:00 2024-09-10T22:34:45.772 INFO:teuthology.orchestra.run.smithi121.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 4.8 MB 00:00 2024-09-10T22:34:45.980 INFO:teuthology.orchestra.run.smithi121.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 19 MB/s | 22 MB 00:01 2024-09-10T22:34:46.022 INFO:teuthology.orchestra.run.smithi121.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 604 kB/s | 25 kB 00:00 2024-09-10T22:34:46.381 INFO:teuthology.orchestra.run.smithi121.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 18 MB/s | 11 MB 00:00 2024-09-10T22:34:46.431 INFO:teuthology.orchestra.run.smithi121.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T22:34:46.498 INFO:teuthology.orchestra.run.smithi121.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 408 kB/s | 27 kB 00:00 2024-09-10T22:34:46.557 INFO:teuthology.orchestra.run.smithi121.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 545 kB/s | 32 kB 00:00 2024-09-10T22:34:46.657 INFO:teuthology.orchestra.run.smithi121.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 7.4 MB/s | 755 kB 00:00 2024-09-10T22:34:46.707 INFO:teuthology.orchestra.run.smithi121.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 3.2 MB/s | 162 kB 00:00 2024-09-10T22:34:46.766 INFO:teuthology.orchestra.run.smithi121.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.1 MB/s | 127 kB 00:00 2024-09-10T22:34:46.841 INFO:teuthology.orchestra.run.smithi121.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 6.6 MB/s | 507 kB 00:00 2024-09-10T22:34:47.158 INFO:teuthology.orchestra.run.smithi121.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 5.5 MB 00:00 2024-09-10T22:34:47.200 INFO:teuthology.orchestra.run.smithi121.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 1.1 MB/s | 45 kB 00:00 2024-09-10T22:34:47.237 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T22:34:47.375 INFO:teuthology.orchestra.run.smithi121.stdout:(20/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 10 MB/s | 17 MB 00:01 2024-09-10T22:34:47.400 INFO:teuthology.orchestra.run.smithi121.stdout:(21/157): python3-ceph-common-19.3.0-4854.g8f21 714 kB/s | 143 kB 00:00 2024-09-10T22:34:47.426 INFO:teuthology.orchestra.run.smithi121.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 3.2 MB/s | 161 kB 00:00 2024-09-10T22:34:47.451 INFO:teuthology.orchestra.run.smithi121.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 6.2 MB/s | 321 kB 00:00 2024-09-10T22:34:47.485 INFO:teuthology.orchestra.run.smithi121.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T22:34:47.510 INFO:teuthology.orchestra.run.smithi121.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-10T22:34:47.535 INFO:teuthology.orchestra.run.smithi121.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T22:34:47.594 INFO:teuthology.orchestra.run.smithi121.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T22:34:47.661 INFO:teuthology.orchestra.run.smithi121.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 21 MB/s | 3.2 MB 00:00 2024-09-10T22:34:47.686 INFO:teuthology.orchestra.run.smithi121.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 372 kB/s | 34 kB 00:00 2024-09-10T22:34:47.712 INFO:teuthology.orchestra.run.smithi121.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T22:34:48.346 INFO:teuthology.orchestra.run.smithi121.stdout:(31/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 22 MB/s | 50 MB 00:02 2024-09-10T22:34:48.424 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T22:34:48.454 INFO:teuthology.orchestra.run.smithi121.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 9.9 MB/s | 7.4 MB 00:00 2024-09-10T22:34:48.488 INFO:teuthology.orchestra.run.smithi121.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.0 MB/s | 291 kB 00:00 2024-09-10T22:34:48.513 INFO:teuthology.orchestra.run.smithi121.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 847 kB/s | 49 kB 00:00 2024-09-10T22:34:48.547 INFO:teuthology.orchestra.run.smithi121.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 280 kB/s | 16 kB 00:00 2024-09-10T22:34:48.572 INFO:teuthology.orchestra.run.smithi121.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.8 MB/s | 285 kB 00:00 2024-09-10T22:34:48.622 INFO:teuthology.orchestra.run.smithi121.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 10 MB/s | 783 kB 00:00 2024-09-10T22:34:48.647 INFO:teuthology.orchestra.run.smithi121.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 242 kB/s | 18 kB 00:00 2024-09-10T22:34:48.831 INFO:teuthology.orchestra.run.smithi121.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 383 kB/s | 80 kB 00:00 2024-09-10T22:34:48.982 INFO:teuthology.orchestra.run.smithi121.stdout:(40/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 11 MB/s | 14 MB 00:01 2024-09-10T22:34:49.007 INFO:teuthology.orchestra.run.smithi121.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-10T22:34:49.065 INFO:teuthology.orchestra.run.smithi121.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 801 kB 00:00 2024-09-10T22:34:49.116 INFO:teuthology.orchestra.run.smithi121.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T22:34:49.158 INFO:teuthology.orchestra.run.smithi121.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-10T22:34:49.183 INFO:teuthology.orchestra.run.smithi121.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T22:34:49.217 INFO:teuthology.orchestra.run.smithi121.stdout:(46/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-10T22:34:49.242 INFO:teuthology.orchestra.run.smithi121.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-10T22:34:49.276 INFO:teuthology.orchestra.run.smithi121.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-10T22:34:49.418 INFO:teuthology.orchestra.run.smithi121.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-09-10T22:34:49.476 INFO:teuthology.orchestra.run.smithi121.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-09-10T22:34:49.501 INFO:teuthology.orchestra.run.smithi121.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T22:34:49.535 INFO:teuthology.orchestra.run.smithi121.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-10T22:34:49.602 INFO:teuthology.orchestra.run.smithi121.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-10T22:34:49.623 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:49.624 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:49.627 INFO:teuthology.orchestra.run.smithi121.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-10T22:34:49.635 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:34:49.636 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:34:49.637 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:34:49.638 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:34:49.640 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:34:49.642 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:34:49.643 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:34:49.652 INFO:teuthology.orchestra.run.smithi121.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-10T22:34:49.677 INFO:teuthology.orchestra.run.smithi121.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T22:34:49.703 INFO:teuthology.orchestra.run.smithi121.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-10T22:34:49.705 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-10T22:34:49.723 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-09-10T22:34:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-10T22:34:49.723 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-09-10T22:34:49.723 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:34:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:34:49.725 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:34:49.726 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:34:49.727 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:34:49.728 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:34:49.729 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:34:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:34:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:34:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:34:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:34:49.734 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:34:49.735 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-09-10T22:34:49.736 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout:Install 149 Packages 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 8 Packages 2024-09-10T22:34:49.737 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:34:49.738 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 240 M 2024-09-10T22:34:49.738 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-10T22:34:49.745 INFO:teuthology.orchestra.run.smithi121.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-10T22:34:49.770 INFO:teuthology.orchestra.run.smithi121.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-09-10T22:34:49.865 INFO:teuthology.orchestra.run.smithi121.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.4 MB/s | 662 kB 00:00 2024-09-10T22:34:49.907 INFO:teuthology.orchestra.run.smithi121.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-10T22:34:49.940 INFO:teuthology.orchestra.run.smithi121.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 3.3 MB/s | 581 kB 00:00 2024-09-10T22:34:49.957 INFO:teuthology.orchestra.run.smithi121.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T22:34:50.016 INFO:teuthology.orchestra.run.smithi121.stdout:(64/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-09-10T22:34:50.041 INFO:teuthology.orchestra.run.smithi121.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T22:34:50.075 INFO:teuthology.orchestra.run.smithi121.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T22:34:50.092 INFO:teuthology.orchestra.run.smithi121.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-10T22:34:50.300 INFO:teuthology.orchestra.run.smithi121.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T22:34:50.442 INFO:teuthology.orchestra.run.smithi121.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-10T22:34:50.501 INFO:teuthology.orchestra.run.smithi121.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T22:34:50.718 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:50.719 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:34:50.730 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:34:50.731 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:34:50.732 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:34:50.733 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:34:50.735 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:34:50.737 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:34:50.738 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:34:50.800 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-10T22:34:50.805 INFO:teuthology.orchestra.run.smithi096.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:34:50.810 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:34:50.811 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:34:50.812 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:34:50.813 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:34:50.814 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:34:50.815 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:34:50.816 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:34:50.817 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:34:50.818 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:34:50.819 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:34:50.820 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:34:50.821 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:34:50.822 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Install 149 Packages 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 8 Packages 2024-09-10T22:34:50.823 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:34:50.824 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 240 M 2024-09-10T22:34:50.824 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-10T22:34:50.885 INFO:teuthology.orchestra.run.smithi121.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.9 MB/s | 4.9 MB 00:00 2024-09-10T22:34:51.110 INFO:teuthology.orchestra.run.smithi121.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-09-10T22:34:51.160 INFO:teuthology.orchestra.run.smithi121.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T22:34:51.306 INFO:teuthology.orchestra.run.smithi096.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.8 MB/s | 900 kB 00:00 2024-09-10T22:34:51.310 INFO:teuthology.orchestra.run.smithi121.stdout:(74/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-10T22:34:51.361 INFO:teuthology.orchestra.run.smithi121.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T22:34:51.406 INFO:teuthology.orchestra.run.smithi096.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.4 MB/s | 145 kB 00:00 2024-09-10T22:34:51.436 INFO:teuthology.orchestra.run.smithi121.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-10T22:34:51.520 INFO:teuthology.orchestra.run.smithi121.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T22:34:51.586 INFO:teuthology.orchestra.run.smithi121.stdout:(78/157): postgresql-server-13.16-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-09-10T22:34:51.597 INFO:teuthology.orchestra.run.smithi096.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 5.4 MB/s | 5.5 MB 00:01 2024-09-10T22:34:51.612 INFO:teuthology.orchestra.run.smithi121.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-09-10T22:34:51.637 INFO:teuthology.orchestra.run.smithi121.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T22:34:51.687 INFO:teuthology.orchestra.run.smithi121.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-10T22:34:51.701 INFO:teuthology.orchestra.run.smithi117.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-09-10T22:34:51.713 INFO:teuthology.orchestra.run.smithi121.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T22:34:51.789 INFO:teuthology.orchestra.run.smithi096.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 8.0 MB/s | 1.5 MB 00:00 2024-09-10T22:34:51.796 INFO:teuthology.orchestra.run.smithi121.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T22:34:51.880 INFO:teuthology.orchestra.run.smithi121.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T22:34:51.922 INFO:teuthology.orchestra.run.smithi121.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-10T22:34:51.972 INFO:teuthology.orchestra.run.smithi121.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-10T22:34:52.056 INFO:teuthology.orchestra.run.smithi121.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-10T22:34:52.114 INFO:teuthology.orchestra.run.smithi121.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-10T22:34:52.198 INFO:teuthology.orchestra.run.smithi121.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-10T22:34:52.249 INFO:teuthology.orchestra.run.smithi121.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T22:34:52.256 INFO:teuthology.orchestra.run.smithi096.stdout:(6/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 4.8 MB 00:00 2024-09-10T22:34:52.307 INFO:teuthology.orchestra.run.smithi096.stdout:(7/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-09-10T22:34:52.315 INFO:teuthology.orchestra.run.smithi121.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 809 kB/s | 54 kB 00:00 2024-09-10T22:34:52.359 INFO:teuthology.orchestra.run.smithi117.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.3 MB/s | 900 kB 00:00 2024-09-10T22:34:52.492 INFO:teuthology.orchestra.run.smithi117.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.1 MB/s | 145 kB 00:00 2024-09-10T22:34:52.932 INFO:teuthology.orchestra.run.smithi121.stdout:(92/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-10T22:34:52.999 INFO:teuthology.orchestra.run.smithi121.stdout:(93/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-10T22:34:53.024 INFO:teuthology.orchestra.run.smithi121.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-10T22:34:53.050 INFO:teuthology.orchestra.run.smithi121.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-10T22:34:53.100 INFO:teuthology.orchestra.run.smithi121.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-09-10T22:34:53.125 INFO:teuthology.orchestra.run.smithi121.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-10T22:34:53.151 INFO:teuthology.orchestra.run.smithi121.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-10T22:34:53.201 INFO:teuthology.orchestra.run.smithi121.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T22:34:53.226 INFO:teuthology.orchestra.run.smithi121.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-10T22:34:53.251 INFO:teuthology.orchestra.run.smithi121.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T22:34:53.277 INFO:teuthology.orchestra.run.smithi121.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T22:34:53.302 INFO:teuthology.orchestra.run.smithi121.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-10T22:34:53.351 INFO:teuthology.orchestra.run.smithi117.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-09-10T22:34:53.432 INFO:teuthology.orchestra.run.smithi096.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 7.6 MB/s | 22 MB 00:02 2024-09-10T22:34:53.515 INFO:teuthology.orchestra.run.smithi096.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 302 kB/s | 25 kB 00:00 2024-09-10T22:34:53.560 INFO:teuthology.orchestra.run.smithi121.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-10T22:34:53.627 INFO:teuthology.orchestra.run.smithi121.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-10T22:34:53.669 INFO:teuthology.orchestra.run.smithi121.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-10T22:34:53.686 INFO:teuthology.orchestra.run.smithi121.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-10T22:34:53.776 INFO:teuthology.orchestra.run.smithi117.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-09-10T22:34:53.907 INFO:teuthology.orchestra.run.smithi096.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 10 MB/s | 17 MB 00:01 2024-09-10T22:34:53.995 INFO:teuthology.orchestra.run.smithi121.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T22:34:54.007 INFO:teuthology.orchestra.run.smithi096.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.2 MB/s | 225 kB 00:00 2024-09-10T22:34:54.010 INFO:teuthology.orchestra.run.smithi117.stdout:(6/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 2.1 MB/s | 5.5 MB 00:02 2024-09-10T22:34:54.012 INFO:teuthology.orchestra.run.smithi121.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T22:34:54.037 INFO:teuthology.orchestra.run.smithi121.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T22:34:54.062 INFO:teuthology.orchestra.run.smithi121.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T22:34:54.088 INFO:teuthology.orchestra.run.smithi121.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T22:34:54.101 INFO:teuthology.orchestra.run.smithi096.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 295 kB/s | 27 kB 00:00 2024-09-10T22:34:54.171 INFO:teuthology.orchestra.run.smithi121.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-10T22:34:54.191 INFO:teuthology.orchestra.run.smithi096.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 346 kB/s | 32 kB 00:00 2024-09-10T22:34:54.205 INFO:teuthology.orchestra.run.smithi121.stdout:(114/157): protobuf-compiler-3.14.0-13.el9.x86_ 956 kB/s | 863 kB 00:00 2024-09-10T22:34:54.241 INFO:teuthology.orchestra.run.smithi121.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-10T22:34:54.263 INFO:teuthology.orchestra.run.smithi121.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-10T22:34:54.289 INFO:teuthology.orchestra.run.smithi121.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 516 kB/s | 43 kB 00:00 2024-09-10T22:34:54.314 INFO:teuthology.orchestra.run.smithi121.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 279 kB/s | 14 kB 00:00 2024-09-10T22:34:54.340 INFO:teuthology.orchestra.run.smithi121.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T22:34:54.365 INFO:teuthology.orchestra.run.smithi121.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T22:34:54.390 INFO:teuthology.orchestra.run.smithi121.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T22:34:54.416 INFO:teuthology.orchestra.run.smithi121.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T22:34:54.441 INFO:teuthology.orchestra.run.smithi121.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T22:34:54.466 INFO:teuthology.orchestra.run.smithi121.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-10T22:34:54.491 INFO:teuthology.orchestra.run.smithi121.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-10T22:34:54.517 INFO:teuthology.orchestra.run.smithi121.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-10T22:34:54.542 INFO:teuthology.orchestra.run.smithi121.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-10T22:34:54.567 INFO:teuthology.orchestra.run.smithi121.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-10T22:34:54.668 INFO:teuthology.orchestra.run.smithi117.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 5.4 MB/s | 4.8 MB 00:00 2024-09-10T22:34:54.726 INFO:teuthology.orchestra.run.smithi121.stdout:(129/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.7 MB/s | 16 MB 00:02 2024-09-10T22:34:54.777 INFO:teuthology.orchestra.run.smithi121.stdout:(130/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-10T22:34:54.793 INFO:teuthology.orchestra.run.smithi121.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 5.3 MB/s | 79 kB 00:00 2024-09-10T22:34:54.819 INFO:teuthology.orchestra.run.smithi121.stdout:(132/157): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-09-10T22:34:54.844 INFO:teuthology.orchestra.run.smithi121.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T22:34:54.869 INFO:teuthology.orchestra.run.smithi121.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-10T22:34:54.894 INFO:teuthology.orchestra.run.smithi121.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T22:34:54.920 INFO:teuthology.orchestra.run.smithi121.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-09-10T22:34:54.945 INFO:teuthology.orchestra.run.smithi121.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-10T22:34:54.970 INFO:teuthology.orchestra.run.smithi121.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-10T22:34:54.996 INFO:teuthology.orchestra.run.smithi121.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-10T22:34:55.021 INFO:teuthology.orchestra.run.smithi121.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T22:34:55.046 INFO:teuthology.orchestra.run.smithi121.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T22:34:55.066 INFO:teuthology.orchestra.run.smithi096.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 863 kB/s | 755 kB 00:00 2024-09-10T22:34:55.071 INFO:teuthology.orchestra.run.smithi121.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T22:34:55.097 INFO:teuthology.orchestra.run.smithi121.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-10T22:34:55.124 INFO:teuthology.orchestra.run.smithi121.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T22:34:55.139 INFO:teuthology.orchestra.run.smithi121.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T22:34:55.167 INFO:teuthology.orchestra.run.smithi096.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.6 MB/s | 162 kB 00:00 2024-09-10T22:34:55.181 INFO:teuthology.orchestra.run.smithi121.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-10T22:34:55.206 INFO:teuthology.orchestra.run.smithi121.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T22:34:55.232 INFO:teuthology.orchestra.run.smithi121.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T22:34:55.250 INFO:teuthology.orchestra.run.smithi096.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.5 MB/s | 127 kB 00:00 2024-09-10T22:34:55.332 INFO:teuthology.orchestra.run.smithi121.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T22:34:55.351 INFO:teuthology.orchestra.run.smithi096.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.0 MB/s | 507 kB 00:00 2024-09-10T22:34:55.682 INFO:teuthology.orchestra.run.smithi121.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 6.8 MB/s | 2.3 MB 00:00 2024-09-10T22:34:55.841 INFO:teuthology.orchestra.run.smithi121.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-09-10T22:34:55.891 INFO:teuthology.orchestra.run.smithi121.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 819 kB/s | 41 kB 00:00 2024-09-10T22:34:55.950 INFO:teuthology.orchestra.run.smithi121.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T22:34:55.993 INFO:teuthology.orchestra.run.smithi096.stdout:(18/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 3.0 MB/s | 11 MB 00:03 2024-09-10T22:34:56.058 INFO:teuthology.orchestra.run.smithi121.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-09-10T22:34:56.076 INFO:teuthology.orchestra.run.smithi096.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 544 kB/s | 45 kB 00:00 2024-09-10T22:34:56.150 INFO:teuthology.orchestra.run.smithi121.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-09-10T22:34:56.159 INFO:teuthology.orchestra.run.smithi096.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.7 MB/s | 143 kB 00:00 2024-09-10T22:34:56.200 INFO:teuthology.orchestra.run.smithi121.stdout:(156/157): librados2-19.3.0-4854.g8f219c47.el9. 3.5 MB/s | 3.5 MB 00:00 2024-09-10T22:34:56.252 INFO:teuthology.orchestra.run.smithi096.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 6.1 MB/s | 5.5 MB 00:00 2024-09-10T22:34:56.277 INFO:teuthology.orchestra.run.smithi096.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.3 MB/s | 161 kB 00:00 2024-09-10T22:34:56.327 INFO:teuthology.orchestra.run.smithi096.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 4.2 MB/s | 321 kB 00:00 2024-09-10T22:34:56.402 INFO:teuthology.orchestra.run.smithi096.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.3 MB/s | 99 kB 00:00 2024-09-10T22:34:56.428 INFO:teuthology.orchestra.run.smithi096.stdout:(25/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.0 MB/s | 306 kB 00:00 2024-09-10T22:34:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.4 MB/s | 85 kB 00:00 2024-09-10T22:34:56.537 INFO:teuthology.orchestra.run.smithi096.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.2 MB/s | 171 kB 00:00 2024-09-10T22:34:56.560 INFO:teuthology.orchestra.run.smithi117.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 5.8 MB/s | 11 MB 00:01 2024-09-10T22:34:56.595 INFO:teuthology.orchestra.run.smithi096.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 586 kB/s | 34 kB 00:00 2024-09-10T22:34:56.627 INFO:teuthology.orchestra.run.smithi117.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 378 kB/s | 25 kB 00:00 2024-09-10T22:34:56.671 INFO:teuthology.orchestra.run.smithi096.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.0 MB/s | 154 kB 00:00 2024-09-10T22:34:57.194 INFO:teuthology.orchestra.run.smithi117.stdout:(10/157): ceph-common-19.3.0-4854.g8f219c47.el9 3.7 MB/s | 22 MB 00:05 2024-09-10T22:34:57.296 INFO:teuthology.orchestra.run.smithi096.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 3.7 MB/s | 3.2 MB 00:00 2024-09-10T22:34:57.335 INFO:teuthology.orchestra.run.smithi117.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 1.6 MB/s | 225 kB 00:00 2024-09-10T22:34:57.419 INFO:teuthology.orchestra.run.smithi117.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 324 kB/s | 27 kB 00:00 2024-09-10T22:34:57.502 INFO:teuthology.orchestra.run.smithi117.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 382 kB/s | 32 kB 00:00 2024-09-10T22:34:57.720 INFO:teuthology.orchestra.run.smithi117.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 3.4 MB/s | 755 kB 00:00 2024-09-10T22:34:57.887 INFO:teuthology.orchestra.run.smithi117.stdout:(15/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 4.4 MB/s | 17 MB 00:03 2024-09-10T22:34:57.920 INFO:teuthology.orchestra.run.smithi117.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 808 kB/s | 162 kB 00:00 2024-09-10T22:34:57.970 INFO:teuthology.orchestra.run.smithi117.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.5 MB/s | 127 kB 00:00 2024-09-10T22:34:58.071 INFO:teuthology.orchestra.run.smithi117.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 3.3 MB/s | 507 kB 00:00 2024-09-10T22:34:58.188 INFO:teuthology.orchestra.run.smithi117.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 387 kB/s | 45 kB 00:00 2024-09-10T22:34:58.255 INFO:teuthology.orchestra.run.smithi117.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.1 MB/s | 143 kB 00:00 2024-09-10T22:34:58.321 INFO:teuthology.orchestra.run.smithi117.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.4 MB/s | 161 kB 00:00 2024-09-10T22:34:58.442 INFO:teuthology.orchestra.run.smithi121.stdout:(157/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 1.0 MB/s | 3.2 MB 00:03 2024-09-10T22:34:58.446 INFO:teuthology.orchestra.run.smithi117.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 2.5 MB/s | 321 kB 00:00 2024-09-10T22:34:58.447 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:34:58.447 INFO:teuthology.orchestra.run.smithi121.stdout:Total 17 MB/s | 240 MB 00:14 2024-09-10T22:34:58.564 INFO:teuthology.orchestra.run.smithi117.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.6 MB/s | 306 kB 00:00 2024-09-10T22:34:58.596 INFO:teuthology.orchestra.run.smithi096.stdout:(31/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 9.9 MB/s | 50 MB 00:05 2024-09-10T22:34:58.631 INFO:teuthology.orchestra.run.smithi117.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.5 MB/s | 99 kB 00:00 2024-09-10T22:34:58.698 INFO:teuthology.orchestra.run.smithi117.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.2 MB/s | 85 kB 00:00 2024-09-10T22:34:58.755 INFO:teuthology.orchestra.run.smithi096.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 6.8 MB/s | 14 MB 00:02 2024-09-10T22:34:58.788 INFO:teuthology.orchestra.run.smithi096.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.5 MB/s | 291 kB 00:00 2024-09-10T22:34:58.822 INFO:teuthology.orchestra.run.smithi096.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 740 kB/s | 49 kB 00:00 2024-09-10T22:34:58.855 INFO:teuthology.orchestra.run.smithi096.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 246 kB/s | 16 kB 00:00 2024-09-10T22:34:58.897 INFO:teuthology.orchestra.run.smithi096.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.7 MB/s | 285 kB 00:00 2024-09-10T22:34:58.956 INFO:teuthology.orchestra.run.smithi117.stdout:(26/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 5.6 MB/s | 5.5 MB 00:00 2024-09-10T22:34:58.973 INFO:teuthology.orchestra.run.smithi096.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 243 kB/s | 18 kB 00:00 2024-09-10T22:34:58.998 INFO:teuthology.orchestra.run.smithi096.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 5.4 MB/s | 783 kB 00:00 2024-09-10T22:34:59.015 INFO:teuthology.orchestra.run.smithi117.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T22:34:59.082 INFO:teuthology.orchestra.run.smithi117.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 514 kB/s | 34 kB 00:00 2024-09-10T22:34:59.149 INFO:teuthology.orchestra.run.smithi117.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.3 MB/s | 154 kB 00:00 2024-09-10T22:34:59.299 INFO:teuthology.orchestra.run.smithi117.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 5.3 MB/s | 3.2 MB 00:00 2024-09-10T22:34:59.390 INFO:teuthology.orchestra.run.smithi096.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 3.5 MB/s | 7.4 MB 00:02 2024-09-10T22:34:59.432 INFO:teuthology.orchestra.run.smithi096.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-10T22:34:59.641 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T22:34:59.724 INFO:teuthology.orchestra.run.smithi096.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 868 kB/s | 253 kB 00:00 2024-09-10T22:34:59.794 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T22:34:59.794 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T22:34:59.857 INFO:teuthology.orchestra.run.smithi096.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 933 kB/s | 801 kB 00:00 2024-09-10T22:34:59.966 INFO:teuthology.orchestra.run.smithi096.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-10T22:35:00.025 INFO:teuthology.orchestra.run.smithi096.stdout:(44/157): libquadmath-11.5.0-2.el9.x86_64.rpm 301 kB/s | 191 kB 00:00 2024-09-10T22:35:00.083 INFO:teuthology.orchestra.run.smithi096.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-10T22:35:00.125 INFO:teuthology.orchestra.run.smithi096.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-10T22:35:00.150 INFO:teuthology.orchestra.run.smithi096.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 993 kB/s | 125 kB 00:00 2024-09-10T22:35:00.201 INFO:teuthology.orchestra.run.smithi096.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-10T22:35:00.293 INFO:teuthology.orchestra.run.smithi096.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-10T22:35:00.335 INFO:teuthology.orchestra.run.smithi096.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-10T22:35:00.360 INFO:teuthology.orchestra.run.smithi096.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 673 kB/s | 107 kB 00:00 2024-09-10T22:35:00.385 INFO:teuthology.orchestra.run.smithi096.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 235 kB/s | 22 kB 00:00 2024-09-10T22:35:00.419 INFO:teuthology.orchestra.run.smithi096.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-10T22:35:00.444 INFO:teuthology.orchestra.run.smithi096.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-09-10T22:35:00.449 INFO:teuthology.orchestra.run.smithi117.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 6.4 MB/s | 7.4 MB 00:01 2024-09-10T22:35:00.503 INFO:teuthology.orchestra.run.smithi096.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 217 kB/s | 18 kB 00:00 2024-09-10T22:35:00.516 INFO:teuthology.orchestra.run.smithi117.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.2 MB/s | 291 kB 00:00 2024-09-10T22:35:00.553 INFO:teuthology.orchestra.run.smithi096.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-10T22:35:00.575 INFO:teuthology.orchestra.run.smithi117.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 843 kB/s | 49 kB 00:00 2024-09-10T22:35:00.578 INFO:teuthology.orchestra.run.smithi096.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-09-10T22:35:00.604 INFO:teuthology.orchestra.run.smithi096.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-10T22:35:00.634 INFO:teuthology.orchestra.run.smithi117.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 282 kB/s | 16 kB 00:00 2024-09-10T22:35:00.637 INFO:teuthology.orchestra.run.smithi096.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-10T22:35:00.663 INFO:teuthology.orchestra.run.smithi096.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-09-10T22:35:00.701 INFO:teuthology.orchestra.run.smithi117.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.2 MB/s | 285 kB 00:00 2024-09-10T22:35:00.713 INFO:teuthology.orchestra.run.smithi096.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-10T22:35:00.738 INFO:teuthology.orchestra.run.smithi096.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-10T22:35:00.764 INFO:teuthology.orchestra.run.smithi096.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-10T22:35:00.797 INFO:teuthology.orchestra.run.smithi096.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-10T22:35:00.831 INFO:teuthology.orchestra.run.smithi096.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-10T22:35:00.851 INFO:teuthology.orchestra.run.smithi117.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 5.1 MB/s | 783 kB 00:00 2024-09-10T22:35:00.856 INFO:teuthology.orchestra.run.smithi096.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-10T22:35:00.881 INFO:teuthology.orchestra.run.smithi096.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 459 kB/s | 38 kB 00:00 2024-09-10T22:35:00.901 INFO:teuthology.orchestra.run.smithi117.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 364 kB/s | 18 kB 00:00 2024-09-10T22:35:00.982 INFO:teuthology.orchestra.run.smithi096.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-10T22:35:01.023 INFO:teuthology.orchestra.run.smithi096.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T22:35:01.034 INFO:teuthology.orchestra.run.smithi117.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-10T22:35:01.090 INFO:teuthology.orchestra.run.smithi096.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-10T22:35:01.116 INFO:teuthology.orchestra.run.smithi096.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T22:35:01.149 INFO:teuthology.orchestra.run.smithi096.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-10T22:35:01.185 INFO:teuthology.orchestra.run.smithi117.stdout:(39/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 7.0 MB/s | 14 MB 00:02 2024-09-10T22:35:01.225 INFO:teuthology.orchestra.run.smithi096.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-10T22:35:01.227 INFO:teuthology.orchestra.run.smithi117.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-10T22:35:01.250 INFO:teuthology.orchestra.run.smithi096.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 871 kB/s | 87 kB 00:00 2024-09-10T22:35:01.360 INFO:teuthology.orchestra.run.smithi117.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-10T22:35:01.375 INFO:teuthology.orchestra.run.smithi096.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-10T22:35:01.401 INFO:teuthology.orchestra.run.smithi096.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 407 kB/s | 61 kB 00:00 2024-09-10T22:35:01.411 INFO:teuthology.orchestra.run.smithi117.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-10T22:35:01.494 INFO:teuthology.orchestra.run.smithi117.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-09-10T22:35:01.509 INFO:teuthology.orchestra.run.smithi096.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-09-10T22:35:01.534 INFO:teuthology.orchestra.run.smithi096.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-10T22:35:01.536 INFO:teuthology.orchestra.run.smithi117.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-09-10T22:35:01.560 INFO:teuthology.orchestra.run.smithi096.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-10T22:35:01.585 INFO:teuthology.orchestra.run.smithi096.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-09-10T22:35:01.652 INFO:teuthology.orchestra.run.smithi096.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-10T22:35:01.677 INFO:teuthology.orchestra.run.smithi096.stdout:(82/157): python3-jsonpointer-2.0-4.el9.noarch. 163 kB/s | 19 kB 00:00 2024-09-10T22:35:01.689 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T22:35:01.690 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T22:35:01.702 INFO:teuthology.orchestra.run.smithi096.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-09-10T22:35:01.819 INFO:teuthology.orchestra.run.smithi096.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-10T22:35:01.844 INFO:teuthology.orchestra.run.smithi096.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-10T22:35:01.888 INFO:teuthology.orchestra.run.smithi096.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-10T22:35:01.912 INFO:teuthology.orchestra.run.smithi096.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-09-10T22:35:01.962 INFO:teuthology.orchestra.run.smithi117.stdout:(45/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 9.4 MB/s | 50 MB 00:05 2024-09-10T22:35:01.971 INFO:teuthology.orchestra.run.smithi096.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-10T22:35:01.987 INFO:teuthology.orchestra.run.smithi117.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 277 kB/s | 125 kB 00:00 2024-09-10T22:35:02.012 INFO:teuthology.orchestra.run.smithi117.stdout:(47/157): python3-pycparser-2.20-6.el9.noarch.r 261 kB/s | 135 kB 00:00 2024-09-10T22:35:02.029 INFO:teuthology.orchestra.run.smithi096.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-10T22:35:02.054 INFO:teuthology.orchestra.run.smithi096.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-10T22:35:02.088 INFO:teuthology.orchestra.run.smithi096.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-10T22:35:02.112 INFO:teuthology.orchestra.run.smithi117.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-09-10T22:35:02.113 INFO:teuthology.orchestra.run.smithi096.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-10T22:35:02.139 INFO:teuthology.orchestra.run.smithi096.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-09-10T22:35:02.171 INFO:teuthology.orchestra.run.smithi117.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-10T22:35:02.180 INFO:teuthology.orchestra.run.smithi096.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-09-10T22:35:02.204 INFO:teuthology.orchestra.run.smithi117.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 654 kB/s | 22 kB 00:00 2024-09-10T22:35:02.206 INFO:teuthology.orchestra.run.smithi096.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-09-10T22:35:02.230 INFO:teuthology.orchestra.run.smithi117.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 912 kB/s | 107 kB 00:00 2024-09-10T22:35:02.263 INFO:teuthology.orchestra.run.smithi117.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-10T22:35:02.273 INFO:teuthology.orchestra.run.smithi096.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-10T22:35:02.288 INFO:teuthology.orchestra.run.smithi117.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-09-10T22:35:02.298 INFO:teuthology.orchestra.run.smithi096.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-09-10T22:35:02.322 INFO:teuthology.orchestra.run.smithi117.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T22:35:02.347 INFO:teuthology.orchestra.run.smithi117.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-10T22:35:02.348 INFO:teuthology.orchestra.run.smithi096.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-10T22:35:02.373 INFO:teuthology.orchestra.run.smithi117.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T22:35:02.399 INFO:teuthology.orchestra.run.smithi096.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-10T22:35:02.414 INFO:teuthology.orchestra.run.smithi117.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-10T22:35:02.432 INFO:teuthology.orchestra.run.smithi096.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-09-10T22:35:02.456 INFO:teuthology.orchestra.run.smithi117.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-10T22:35:02.482 INFO:teuthology.orchestra.run.smithi117.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-09-10T22:35:02.532 INFO:teuthology.orchestra.run.smithi117.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-10T22:35:02.565 INFO:teuthology.orchestra.run.smithi117.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-10T22:35:02.583 INFO:teuthology.orchestra.run.smithi096.stdout:(101/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-10T22:35:02.592 INFO:teuthology.orchestra.run.smithi117.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 409 kB/s | 24 kB 00:00 2024-09-10T22:35:02.608 INFO:teuthology.orchestra.run.smithi096.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 364 kB/s | 64 kB 00:00 2024-09-10T22:35:02.617 INFO:teuthology.orchestra.run.smithi117.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 916 kB/s | 47 kB 00:00 2024-09-10T22:35:02.625 INFO:teuthology.orchestra.run.smithi096.stdout:(103/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 425 kB/s | 96 kB 00:00 2024-09-10T22:35:02.643 INFO:teuthology.orchestra.run.smithi117.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T22:35:02.667 INFO:teuthology.orchestra.run.smithi096.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-09-10T22:35:02.676 INFO:teuthology.orchestra.run.smithi117.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-10T22:35:02.704 INFO:teuthology.orchestra.run.smithi117.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-10T22:35:02.751 INFO:teuthology.orchestra.run.smithi096.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-10T22:35:02.800 INFO:teuthology.orchestra.run.smithi096.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-09-10T22:35:02.826 INFO:teuthology.orchestra.run.smithi096.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-10T22:35:02.835 INFO:teuthology.orchestra.run.smithi117.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-10T22:35:02.902 INFO:teuthology.orchestra.run.smithi117.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-10T22:35:02.953 INFO:teuthology.orchestra.run.smithi117.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 18 MB/s | 910 kB 00:00 2024-09-10T22:35:02.994 INFO:teuthology.orchestra.run.smithi117.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-10T22:35:03.134 INFO:teuthology.orchestra.run.smithi096.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T22:35:03.145 INFO:teuthology.orchestra.run.smithi117.stdout:(71/157): postgresql-server-13.16-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-09-10T22:35:03.159 INFO:teuthology.orchestra.run.smithi096.stdout:(109/157): lua-devel-5.4.4-4.el9.x86_64.rpm 40 kB/s | 22 kB 00:00 2024-09-10T22:35:03.185 INFO:teuthology.orchestra.run.smithi096.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-10T22:35:03.195 INFO:teuthology.orchestra.run.smithi117.stdout:(72/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-10T22:35:03.218 INFO:teuthology.orchestra.run.smithi096.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-10T22:35:03.237 INFO:teuthology.orchestra.run.smithi117.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-10T22:35:03.252 INFO:teuthology.orchestra.run.smithi096.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-10T22:35:03.277 INFO:teuthology.orchestra.run.smithi096.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 414 kB/s | 49 kB 00:00 2024-09-10T22:35:03.279 INFO:teuthology.orchestra.run.smithi117.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-10T22:35:03.321 INFO:teuthology.orchestra.run.smithi117.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T22:35:03.344 INFO:teuthology.orchestra.run.smithi096.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-09-10T22:35:03.370 INFO:teuthology.orchestra.run.smithi096.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-10T22:35:03.395 INFO:teuthology.orchestra.run.smithi096.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 868 kB/s | 43 kB 00:00 2024-09-10T22:35:03.412 INFO:teuthology.orchestra.run.smithi096.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 762 kB/s | 32 kB 00:00 2024-09-10T22:35:03.421 INFO:teuthology.orchestra.run.smithi117.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-09-10T22:35:03.429 INFO:teuthology.orchestra.run.smithi096.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 417 kB/s | 14 kB 00:00 2024-09-10T22:35:03.454 INFO:teuthology.orchestra.run.smithi096.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-10T22:35:03.463 INFO:teuthology.orchestra.run.smithi117.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-10T22:35:03.488 INFO:teuthology.orchestra.run.smithi117.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-10T22:35:03.496 INFO:teuthology.orchestra.run.smithi096.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T22:35:03.505 INFO:teuthology.orchestra.run.smithi117.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-09-10T22:35:03.522 INFO:teuthology.orchestra.run.smithi096.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T22:35:03.522 INFO:teuthology.orchestra.run.smithi117.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-09-10T22:35:03.555 INFO:teuthology.orchestra.run.smithi096.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T22:35:03.556 INFO:teuthology.orchestra.run.smithi117.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-09-10T22:35:03.581 INFO:teuthology.orchestra.run.smithi117.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T22:35:03.589 INFO:teuthology.orchestra.run.smithi096.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T22:35:03.623 INFO:teuthology.orchestra.run.smithi096.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 321 kB/s | 11 kB 00:00 2024-09-10T22:35:03.648 INFO:teuthology.orchestra.run.smithi096.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-10T22:35:03.673 INFO:teuthology.orchestra.run.smithi096.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-10T22:35:03.698 INFO:teuthology.orchestra.run.smithi117.stdout:(83/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-10T22:35:03.699 INFO:teuthology.orchestra.run.smithi096.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 787 kB/s | 19 kB 00:00 2024-09-10T22:35:03.725 INFO:teuthology.orchestra.run.smithi096.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-10T22:35:03.725 INFO:teuthology.orchestra.run.smithi117.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-10T22:35:03.757 INFO:teuthology.orchestra.run.smithi117.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T22:35:03.782 INFO:teuthology.orchestra.run.smithi096.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-10T22:35:03.790 INFO:teuthology.orchestra.run.smithi117.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-10T22:35:03.808 INFO:teuthology.orchestra.run.smithi096.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:35:03.815 INFO:teuthology.orchestra.run.smithi117.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-09-10T22:35:03.841 INFO:teuthology.orchestra.run.smithi096.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T22:35:03.857 INFO:teuthology.orchestra.run.smithi117.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-10T22:35:03.875 INFO:teuthology.orchestra.run.smithi096.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-10T22:35:03.899 INFO:teuthology.orchestra.run.smithi117.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-10T22:35:03.909 INFO:teuthology.orchestra.run.smithi096.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-10T22:35:03.933 INFO:teuthology.orchestra.run.smithi117.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-10T22:35:03.934 INFO:teuthology.orchestra.run.smithi096.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-10T22:35:03.958 INFO:teuthology.orchestra.run.smithi117.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-10T22:35:03.959 INFO:teuthology.orchestra.run.smithi096.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-10T22:35:03.984 INFO:teuthology.orchestra.run.smithi117.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T22:35:03.984 INFO:teuthology.orchestra.run.smithi096.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T22:35:04.010 INFO:teuthology.orchestra.run.smithi096.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T22:35:04.026 INFO:teuthology.orchestra.run.smithi117.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-09-10T22:35:04.035 INFO:teuthology.orchestra.run.smithi096.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T22:35:04.060 INFO:teuthology.orchestra.run.smithi096.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-10T22:35:04.068 INFO:teuthology.orchestra.run.smithi117.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T22:35:04.086 INFO:teuthology.orchestra.run.smithi096.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T22:35:04.111 INFO:teuthology.orchestra.run.smithi096.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-10T22:35:04.136 INFO:teuthology.orchestra.run.smithi096.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T22:35:04.161 INFO:teuthology.orchestra.run.smithi096.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T22:35:04.168 INFO:teuthology.orchestra.run.smithi117.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-09-10T22:35:04.187 INFO:teuthology.orchestra.run.smithi096.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T22:35:04.221 INFO:teuthology.orchestra.run.smithi096.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 596 kB/s | 20 kB 00:00 2024-09-10T22:35:04.243 INFO:teuthology.orchestra.run.smithi117.stdout:(96/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.2 MB/s | 4.9 MB 00:01 2024-09-10T22:35:04.246 INFO:teuthology.orchestra.run.smithi096.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T22:35:04.269 INFO:teuthology.orchestra.run.smithi117.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-09-10T22:35:04.271 INFO:teuthology.orchestra.run.smithi096.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T22:35:04.294 INFO:teuthology.orchestra.run.smithi117.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-10T22:35:04.322 INFO:teuthology.orchestra.run.smithi096.stdout:(148/157): protobuf-compiler-3.14.0-13.el9.x86_ 509 kB/s | 863 kB 00:01 2024-09-10T22:35:04.380 INFO:teuthology.orchestra.run.smithi096.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T22:35:04.461 INFO:teuthology.orchestra.run.smithi117.stdout:(99/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-10T22:35:04.486 INFO:teuthology.orchestra.run.smithi117.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-09-10T22:35:04.512 INFO:teuthology.orchestra.run.smithi117.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T22:35:04.537 INFO:teuthology.orchestra.run.smithi117.stdout:(102/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-10T22:35:04.562 INFO:teuthology.orchestra.run.smithi117.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 844 kB/s | 64 kB 00:00 2024-09-10T22:35:04.587 INFO:teuthology.orchestra.run.smithi117.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-09-10T22:35:04.646 INFO:teuthology.orchestra.run.smithi117.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-10T22:35:04.664 INFO:teuthology.orchestra.run.smithi096.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 9.5 MB/s | 3.2 MB 00:00 2024-09-10T22:35:04.696 INFO:teuthology.orchestra.run.smithi117.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-09-10T22:35:04.721 INFO:teuthology.orchestra.run.smithi117.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 852 kB/s | 19 kB 00:00 2024-09-10T22:35:04.755 INFO:teuthology.orchestra.run.smithi117.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-10T22:35:04.781 INFO:teuthology.orchestra.run.smithi096.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 6.8 MB/s | 3.5 MB 00:00 2024-09-10T22:35:04.788 INFO:teuthology.orchestra.run.smithi117.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-10T22:35:04.814 INFO:teuthology.orchestra.run.smithi117.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-10T22:35:04.856 INFO:teuthology.orchestra.run.smithi117.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-09-10T22:35:04.889 INFO:teuthology.orchestra.run.smithi117.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-09-10T22:35:04.915 INFO:teuthology.orchestra.run.smithi117.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-10T22:35:04.981 INFO:teuthology.orchestra.run.smithi096.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 204 kB/s | 41 kB 00:00 2024-09-10T22:35:04.992 INFO:teuthology.orchestra.run.smithi117.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-09-10T22:35:05.016 INFO:teuthology.orchestra.run.smithi117.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T22:35:05.115 INFO:teuthology.orchestra.run.smithi117.stdout:(116/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-10T22:35:05.191 INFO:teuthology.orchestra.run.smithi117.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 184 kB/s | 32 kB 00:00 2024-09-10T22:35:05.216 INFO:teuthology.orchestra.run.smithi117.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 141 kB/s | 14 kB 00:00 2024-09-10T22:35:05.241 INFO:teuthology.orchestra.run.smithi117.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-10T22:35:05.275 INFO:teuthology.orchestra.run.smithi117.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T22:35:05.300 INFO:teuthology.orchestra.run.smithi117.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T22:35:05.367 INFO:teuthology.orchestra.run.smithi117.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-09-10T22:35:05.381 INFO:teuthology.orchestra.run.smithi096.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 622 kB/s | 249 kB 00:00 2024-09-10T22:35:05.393 INFO:teuthology.orchestra.run.smithi117.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-10T22:35:05.418 INFO:teuthology.orchestra.run.smithi117.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T22:35:05.423 INFO:teuthology.orchestra.run.smithi096.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-10T22:35:05.443 INFO:teuthology.orchestra.run.smithi117.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-10T22:35:05.469 INFO:teuthology.orchestra.run.smithi117.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-10T22:35:05.494 INFO:teuthology.orchestra.run.smithi117.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-10T22:35:05.519 INFO:teuthology.orchestra.run.smithi117.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-10T22:35:05.544 INFO:teuthology.orchestra.run.smithi117.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T22:35:05.570 INFO:teuthology.orchestra.run.smithi117.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:35:05.603 INFO:teuthology.orchestra.run.smithi117.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T22:35:05.628 INFO:teuthology.orchestra.run.smithi117.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T22:35:05.679 INFO:teuthology.orchestra.run.smithi117.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-09-10T22:35:05.712 INFO:teuthology.orchestra.run.smithi117.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-10T22:35:05.738 INFO:teuthology.orchestra.run.smithi117.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-10T22:35:05.763 INFO:teuthology.orchestra.run.smithi117.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T22:35:05.773 INFO:teuthology.orchestra.run.smithi096.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 1.4 MB 00:00 2024-09-10T22:35:05.788 INFO:teuthology.orchestra.run.smithi117.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-10T22:35:05.813 INFO:teuthology.orchestra.run.smithi117.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T22:35:05.824 INFO:teuthology.orchestra.run.smithi096.stdout:(156/157): NetworkManager-libnm-1.49.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-09-10T22:35:05.838 INFO:teuthology.orchestra.run.smithi117.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-10T22:35:05.864 INFO:teuthology.orchestra.run.smithi117.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T22:35:05.889 INFO:teuthology.orchestra.run.smithi117.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T22:35:05.914 INFO:teuthology.orchestra.run.smithi117.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T22:35:05.940 INFO:teuthology.orchestra.run.smithi117.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-10T22:35:05.965 INFO:teuthology.orchestra.run.smithi117.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T22:35:05.990 INFO:teuthology.orchestra.run.smithi117.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T22:35:06.007 INFO:teuthology.orchestra.run.smithi117.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-09-10T22:35:06.032 INFO:teuthology.orchestra.run.smithi117.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-10T22:35:06.058 INFO:teuthology.orchestra.run.smithi117.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-10T22:35:06.108 INFO:teuthology.orchestra.run.smithi117.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T22:35:06.258 INFO:teuthology.orchestra.run.smithi117.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-09-10T22:35:06.358 INFO:teuthology.orchestra.run.smithi117.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-09-10T22:35:06.409 INFO:teuthology.orchestra.run.smithi117.stdout:(152/157): librados2-19.3.0-4854.g8f219c47.el9. 9.2 MB/s | 3.5 MB 00:00 2024-09-10T22:35:06.451 INFO:teuthology.orchestra.run.smithi117.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 442 kB/s | 41 kB 00:00 2024-09-10T22:35:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-10T22:35:06.551 INFO:teuthology.orchestra.run.smithi117.stdout:(155/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 6.6 MB/s | 3.2 MB 00:00 2024-09-10T22:35:06.565 INFO:teuthology.orchestra.run.smithi096.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-10T22:35:06.571 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:35:06.571 INFO:teuthology.orchestra.run.smithi096.stdout:Total 14 MB/s | 240 MB 00:16 2024-09-10T22:35:06.593 INFO:teuthology.orchestra.run.smithi117.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-10T22:35:06.660 INFO:teuthology.orchestra.run.smithi117.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-10T22:35:06.666 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:35:06.667 INFO:teuthology.orchestra.run.smithi117.stdout:Total 15 MB/s | 240 MB 00:15 2024-09-10T22:35:07.827 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-10T22:35:07.901 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-10T22:35:07.981 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-10T22:35:07.981 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-10T22:35:08.054 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-10T22:35:08.054 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-10T22:35:08.054 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T22:35:08.186 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:35:08.224 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:35:08.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:35:08.312 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:35:08.337 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:35:08.380 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:35:08.790 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:35:08.897 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:09.020 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:09.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:09.140 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:09.169 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:35:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:35:09.351 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:35:09.400 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:35:09.437 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:09.541 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:09.571 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:09.634 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:09.656 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:09.675 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:09.865 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:09.952 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-10T22:35:09.966 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-10T22:35:09.990 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:10.112 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:35:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:10.297 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:10.402 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-10T22:35:10.403 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-10T22:35:10.531 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:10.639 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:35:10.668 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:35:10.703 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:35:10.776 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:35:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:35:10.861 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:35:10.904 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:35:10.937 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:35:10.965 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:35:11.034 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:35:11.087 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:35:11.117 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:35:11.153 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:35:11.231 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:35:11.264 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:35:11.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:35:11.327 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:35:11.400 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:35:11.445 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:35:11.521 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:35:11.659 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:35:11.714 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:35:12.235 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:35:12.327 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:35:12.356 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:35:12.398 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:35:12.437 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:35:12.479 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:35:12.654 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:35:12.715 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:35:12.805 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:35:12.871 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:35:12.945 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:35:12.975 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:35:13.011 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:35:13.058 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:35:13.094 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:35:13.137 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:35:13.180 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:35:13.316 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:35:13.389 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:35:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:35:13.575 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:35:13.682 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:35:13.849 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:35:13.874 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:35:13.953 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:35:14.076 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:35:14.115 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:35:14.142 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:35:14.216 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:35:14.247 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:35:14.278 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:35:14.312 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:35:14.345 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:35:14.387 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:35:14.421 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:35:14.508 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:35:14.560 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:35:14.591 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:35:14.695 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:35:14.751 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:35:15.467 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:35:15.515 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:35:15.548 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:35:15.646 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:35:15.704 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:35:15.748 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:35:15.826 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:35:15.868 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:35:16.050 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:35:16.082 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:35:16.160 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:35:16.412 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-10T22:35:16.529 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:35:16.568 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:35:16.630 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:35:16.663 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:35:16.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:35:16.740 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:35:16.801 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:35:16.819 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-10T22:35:16.915 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:35:16.935 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:35:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:35:17.082 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:35:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:35:17.198 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:35:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:35:17.322 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:35:17.328 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:17.449 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:17.479 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:35:17.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:35:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:35:17.663 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:35:17.709 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:35:17.738 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:17.763 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:17.825 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:17.855 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:17.885 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:17.901 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:17.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:17.915 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:17.940 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:18.044 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:35:18.073 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:35:18.128 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:35:18.133 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:18.205 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:35:18.233 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:35:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:35:18.266 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:35:18.274 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:35:18.328 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:18.388 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:35:18.432 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:18.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:18.500 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:35:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:35:18.581 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:18.585 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:35:18.620 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:18.710 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:18.742 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:35:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:18.777 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:35:18.793 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:18.924 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:35:18.952 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:35:18.956 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:35:18.995 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:35:19.069 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:35:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:35:19.121 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:35:19.161 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:35:19.194 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:19.205 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:35:19.237 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:35:19.265 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:35:19.270 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:35:19.275 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:19.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:19.335 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:35:19.369 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:35:19.400 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:35:19.418 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:35:19.445 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:35:19.505 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:35:19.523 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:35:19.556 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:35:19.599 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:35:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:35:19.628 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:35:19.655 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:35:19.690 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:35:19.705 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:35:19.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:35:19.763 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:35:19.808 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:35:19.820 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:35:19.848 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:35:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:35:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:35:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:35:19.952 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:35:20.006 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:35:20.038 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:35:20.099 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:35:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:35:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:35:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:35:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:35:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:35:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:35:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:35:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:35:20.525 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:35:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:35:20.612 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:35:20.640 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:35:20.683 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:35:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:35:20.720 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:35:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:35:20.762 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:35:20.939 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:35:20.998 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:35:21.089 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:35:21.154 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:35:21.227 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:35:21.259 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:35:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:35:21.294 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:35:21.341 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:35:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:35:21.377 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:35:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:35:21.420 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:35:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:35:21.462 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:35:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:35:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:35:21.598 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:35:21.648 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:35:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:35:21.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:35:21.727 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:35:21.763 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:21.803 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:35:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:35:21.840 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:35:21.880 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:35:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:35:21.906 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:35:21.930 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:35:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:35:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:35:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:35:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:35:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:35:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:35:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:35:22.214 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:35:22.315 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:22.350 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:35:22.358 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:22.382 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:35:22.391 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:35:22.429 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:35:22.432 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:35:22.464 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:22.465 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:35:22.465 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:35:22.465 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:22.472 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:35:22.492 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:22.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:35:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:35:22.548 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:35:22.587 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:35:22.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:35:22.652 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:35:22.654 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:35:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:35:22.708 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:35:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:35:22.741 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:35:22.761 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:35:22.774 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:35:22.847 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:35:22.877 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:35:22.891 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:35:22.922 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:35:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:35:23.025 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:35:23.039 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:35:23.068 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:35:23.075 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:35:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:35:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:35:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:35:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:35:23.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:35:23.295 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:35:23.328 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:35:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:35:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:35:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:35:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:35:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:35:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:35:23.790 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:35:23.839 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:35:23.873 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:35:23.970 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:35:24.027 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:35:24.072 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:35:24.150 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:35:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:35:24.376 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:35:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:35:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:35:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:35:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:35:24.483 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:35:24.568 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:35:24.624 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:35:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:35:24.757 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:35:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:35:24.971 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:35:24.994 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:35:25.071 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:35:25.111 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:35:25.229 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:35:25.726 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:35:25.837 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:35:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:35:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:35:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:35:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:35:26.915 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:35:26.959 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:27.050 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:27.075 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:35:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:35:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:35:27.171 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:35:27.207 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:35:27.241 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:35:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:35:27.608 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:35:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:27.637 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:35:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:27.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:35:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:35:27.770 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:35:28.260 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:35:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:28.362 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:35:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:35:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:30.660 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:30.696 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:30.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:30.795 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:30.795 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:35:30.795 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:35:30.795 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:30.823 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:31.292 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:35:31.334 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:31.407 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:35:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:35:31.408 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:35:31.408 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:31.443 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-09-10T22:35:34.092 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-09-10T22:35:34.093 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-09-10T22:35:34.093 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:34.263 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:34.279 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:34.279 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:34.279 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:35:34.279 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:35.799 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:35.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:36.058 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:36.081 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:36.534 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:36.557 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:36.557 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:36.557 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:35:36.558 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:36.558 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:36.558 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:36.656 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:35:36.712 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:35:36.768 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:35:36.952 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:35:37.001 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:35:37.071 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:35:37.160 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:35:37.299 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:35:37.335 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:35:37.367 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:35:37.403 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:35:37.833 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:37.860 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:37.892 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:35:37.957 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:35:37.983 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:41.175 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:41.231 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:42.223 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:42.336 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:42.502 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:35:42.533 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-09-10T22:35:42.549 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-09-10T22:35:42.550 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-09-10T22:35:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:42.557 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:42.600 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:42.612 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:42.710 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:35:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:42.749 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:42.749 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:35:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:42.756 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:35:43.190 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-09-10T22:35:43.191 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:35:43.381 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:43.381 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:35:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:43.706 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:35:43.736 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:43.759 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:43.803 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:43.825 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:43.826 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:43.826 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:35:43.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:43.871 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:43.894 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:44.205 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:44.228 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:44.228 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:44.229 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:35:44.229 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:44.229 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:44.229 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:35:44.236 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:44.263 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:44.263 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:44.263 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:35:44.263 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:44.263 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:44.264 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:44.511 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:44.535 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:35:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:44.979 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:45.002 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:45.002 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:45.002 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:35:45.002 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:45.002 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:45.003 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:45.100 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:35:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:45.157 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:35:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:45.221 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:35:45.406 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:35:45.446 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:35:45.524 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:35:45.611 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:35:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:35:45.643 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:45.643 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:35:45.643 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:45.643 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:35:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:35:45.768 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:35:45.805 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:35:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:35:45.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:35:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:35:45.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:35:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:35:46.095 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:35:46.165 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:35:46.231 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:46.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:35:46.271 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:46.303 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:35:46.368 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:35:46.394 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:46.409 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:35:46.446 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:35:46.477 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:35:46.505 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:35:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:35:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:35:47.101 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:35:47.127 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:48.893 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:35:48.934 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:35:48.974 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:35:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:35:49.033 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:35:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:35:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:35:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:35:49.315 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:35:49.361 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:49.624 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:50.026 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:50.046 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:50.283 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:35:50.350 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:50.582 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:50.608 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:50.724 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:50.867 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:35:50.886 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:35:50.910 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:50.935 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:50.935 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:50.935 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:35:50.935 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:50.935 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:50.936 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:50.948 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:35:50.986 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:50.988 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:35:50.998 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:51.069 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:35:51.095 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:35:51.118 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:35:51.142 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:35:51.143 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:35:51.169 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:35:51.169 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:35:51.245 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:35:51.245 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:35:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:35:51.329 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:35:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:51.357 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:35:51.359 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:51.359 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:35:51.539 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:51.539 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:35:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:35:51.623 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:35:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:35:51.668 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:51.668 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:35:51.668 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:51.668 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:35:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:51.686 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:35:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:35:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:35:51.890 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:35:52.094 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:35:52.138 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:52.162 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:52.162 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.162 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:35:52.163 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:52.163 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:52.163 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:52.205 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:52.228 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:52.228 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.229 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:35:52.229 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:52.273 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:52.296 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:35:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:52.297 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:52.607 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:52.630 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:52.630 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.630 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:35:52.630 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:52.631 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:52.631 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:35:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:35:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:35:52.896 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.897 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:35:52.897 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:52.897 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:35:52.897 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:35:52.962 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:52.962 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:35:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:53.007 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:35:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:35:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:35:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:53.355 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:35:53.355 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:35:53.355 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:35:53.355 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:53.355 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:35:53.356 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:35:57.319 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:35:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:35:57.401 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:35:57.433 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:35:57.460 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:35:57.640 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:35:57.690 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:35:57.720 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:35:57.750 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:35:57.796 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:35:58.033 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:35:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:35:58.086 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:35:58.086 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:35:58.086 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:35:58.086 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:35:58.086 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:35:58.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:35:58.088 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:35:58.089 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:35:58.090 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:35:58.091 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:35:58.092 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:35:58.093 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:35:58.094 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:35:58.095 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:35:58.096 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:35:58.097 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:35:58.098 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:35:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:35:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:35:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:35:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:35:58.390 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:58.427 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:35:58.447 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:35:58.492 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:35:58.531 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:35:59.141 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:59.179 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:35:59.272 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:35:59.357 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:35:59.389 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:35:59.479 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:35:59.518 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:35:59.543 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:35:59.569 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:35:59.570 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:35:59.662 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:35:59.662 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:35:59.746 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:35:59.771 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:35:59.776 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:59.777 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:59.956 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:35:59.956 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:36:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:36:00.040 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:36:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:36:00.103 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:36:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:36:00.271 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:36:00.296 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:36:00.322 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:36:00.322 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:36:00.406 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:36:00.406 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:36:00.490 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:36:00.515 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:36:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:36:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.531 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.532 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:36:00.533 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:36:00.534 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:36:00.535 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:36:00.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:36:00.537 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:36:00.538 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:36:00.539 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:36:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:36:00.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:36:00.542 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:36:00.543 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:36:00.543 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:36:00.543 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T22:36:00.701 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:36:00.701 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:36:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:36:00.850 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:00.953 DEBUG:teuthology.parallel:result is None 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:36:06.811 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:36:06.812 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:36:06.813 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:36:06.814 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:36:06.814 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:36:06.814 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:36:06.818 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:36:06.818 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:36:06.818 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:36:06.819 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:36:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:36:06.821 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:36:06.822 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:36:06.823 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:36:06.824 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:36:06.825 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:36:06.826 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:36:06.827 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:36:06.828 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:36:07.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:36:07.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:36:07.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:36:07.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:36:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:36:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:36:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:36:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:36:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:36:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:36:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:36:07.119 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:36:07.120 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:36:07.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:36:09.584 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:09.584 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:36:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:36:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:36:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:36:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:36:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:36:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:36:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:36:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:36:09.598 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:36:09.598 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:36:09.598 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.790 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.791 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.792 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:36:09.793 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:36:09.794 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:36:09.795 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:36:09.796 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:36:09.797 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:36:09.798 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:36:09.799 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.800 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:36:09.801 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:36:09.802 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:36:09.803 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:36:09.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:36:09.803 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-10T22:36:09.980 DEBUG:teuthology.parallel:result is None 2024-09-10T22:36:10.210 DEBUG:teuthology.parallel:result is None 2024-09-10T22:36:10.210 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:10.369 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:36:10.391 INFO:teuthology.orchestra.run.smithi096.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.391 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.392 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:36:10.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:10.558 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:36:10.582 INFO:teuthology.orchestra.run.smithi117.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.582 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.582 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:36:10.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:10.736 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:36:10.762 INFO:teuthology.orchestra.run.smithi121.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.763 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:36:10.763 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:36:10.765 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T22:36:10.765 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:36:10.765 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:36:10.792 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:36:10.792 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:36:10.823 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:36:10.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:36:10.855 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T22:36:10.855 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:36:10.855 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:36:10.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:36:10.946 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:36:10.946 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:36:10.973 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:36:11.039 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:36:11.039 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:36:11.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:36:11.137 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T22:36:11.138 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:36:11.138 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:36:11.162 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:36:11.226 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:36:11.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:36:11.253 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:36:11.319 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:36:11.319 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:36:11.348 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:36:11.417 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T22:36:11.417 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:36:11.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:36:11.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:36:11.506 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:36:11.506 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:36:11.533 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:36:11.600 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:36:11.600 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:36:11.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:36:11.698 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T22:36:11.806 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T22:36:11.807 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:11.807 INFO:tasks.cephadm:Cluster fsid is 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:36:11.807 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T22:36:11.807 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.96', 'mon.b': '172.21.15.117', 'mon.c': '172.21.15.121'} 2024-09-10T22:36:11.808 INFO:tasks.cephadm:First mon is mon.a on smithi096 2024-09-10T22:36:11.808 INFO:tasks.cephadm:First mgr is x 2024-09-10T22:36:11.808 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T22:36:11.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-09-10T22:36:11.834 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-09-10T22:36:11.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname $(hostname -s) 2024-09-10T22:36:11.893 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T22:36:11.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:12.045 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T22:36:12.194 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:36:12.195 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:36:12.196 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T22:36:12.196 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:12.616 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:36 /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:12.617 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.024 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:36 /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.024 DEBUG:teuthology.orchestra.run.smithi121:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.428 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 22:36 /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.428 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.448 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.468 DEBUG:teuthology.orchestra.run.smithi121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T22:36:13.502 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T22:36:13.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:36:13.505 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:36:13.510 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T22:36:15.106 INFO:teuthology.orchestra.run.smithi121.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:36:15.166 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:36:15.188 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:36:49.536 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T22:36:49.536 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:36:49.536 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:36:49.537 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-09-10T22:36:49.537 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:36:49.537 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-10T22:36:49.537 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T22:36:49.722 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T22:36:49.723 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout: "repo_digests": [ 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T22:36:50.583 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T22:36:50.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-09-10T22:36:50.643 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-09-10T22:36:50.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph 2024-09-10T22:36:50.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-09-10T22:36:50.733 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-09-10T22:36:50.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /etc/ceph 2024-09-10T22:36:50.787 INFO:tasks.cephadm:Writing seed config... 2024-09-10T22:36:50.788 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T22:36:50.788 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-10T22:36:50.788 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-10T22:36:50.788 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T22:36:50.788 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T22:36:50.789 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T22:36:50.790 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-10T22:36:50.791 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:36:50.791 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T22:36:50.808 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 14afb286-6fc5-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-09-10T22:36:50.809 DEBUG:teuthology.orchestra.run.smithi096:mon.a> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T22:36:50.851 DEBUG:teuthology.orchestra.run.smithi096:mgr.x> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T22:36:50.895 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T22:36:50.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.96 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:36:51.099 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:36:51.099 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.96', '--skip-admin-label'] 2024-09-10T22:36:51.100 INFO:teuthology.orchestra.run.smithi096.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T22:36:51.100 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying podman|docker is present... 2024-09-10T22:36:51.134 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:36:51.134 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying lvm2 is present... 2024-09-10T22:36:51.134 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying time synchronization is in place... 2024-09-10T22:36:51.141 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:36:51.142 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:36:51.148 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:36:51.148 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:36:51.155 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-09-10T22:36:51.161 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-09-10T22:36:51.161 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:36:51.162 INFO:teuthology.orchestra.run.smithi096.stdout:Repeating the final host check... 2024-09-10T22:36:51.183 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 5.2.2 2024-09-10T22:36:51.183 INFO:teuthology.orchestra.run.smithi096.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T22:36:51.183 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl is present 2024-09-10T22:36:51.183 INFO:teuthology.orchestra.run.smithi096.stdout:lvcreate is present 2024-09-10T22:36:51.190 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T22:36:51.190 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T22:36:51.196 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T22:36:51.196 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:36:51.203 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-09-10T22:36:51.209 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-09-10T22:36:51.209 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-09-10T22:36:51.209 INFO:teuthology.orchestra.run.smithi096.stdout:Host looks OK 2024-09-10T22:36:51.209 INFO:teuthology.orchestra.run.smithi096.stdout:Cluster fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:36:51.210 INFO:teuthology.orchestra.run.smithi096.stdout:Acquiring lock 140448018518896 on /run/cephadm/14afb286-6fc5-11ef-bcea-c7b262605968.lock 2024-09-10T22:36:51.210 INFO:teuthology.orchestra.run.smithi096.stdout:Lock 140448018518896 acquired on /run/cephadm/14afb286-6fc5-11ef-bcea-c7b262605968.lock 2024-09-10T22:36:51.210 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 3300 ... 2024-09-10T22:36:51.210 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 6789 ... 2024-09-10T22:36:51.210 INFO:teuthology.orchestra.run.smithi096.stdout:Base mon IP(s) is [172.21.15.96:3300, 172.21.15.96:6789], mon addrv is [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-09-10T22:36:51.213 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.96 metric 100 2024-09-10T22:36:51.213 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.96 metric 100 2024-09-10T22:36:51.216 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T22:36:51.216 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T22:36:51.216 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T22:36:51.218 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link noprefixroute 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-09-10T22:36:51.219 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-09-10T22:36:51.220 INFO:teuthology.orchestra.run.smithi096.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T22:36:51.220 INFO:teuthology.orchestra.run.smithi096.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T22:36:51.220 INFO:teuthology.orchestra.run.smithi096.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:36:54.674 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:36:54.674 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T22:36:54.674 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T22:36:54.674 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T22:36:54.674 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T22:36:54.675 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T22:36:54.675 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T22:37:03.757 INFO:teuthology.orchestra.run.smithi096.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:37:03.758 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T22:37:03.758 INFO:teuthology.orchestra.run.smithi096.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T22:37:06.046 INFO:teuthology.orchestra.run.smithi096.stdout:stat: stdout 167 167 2024-09-10T22:37:06.046 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial keys... 2024-09-10T22:37:06.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQASyuBmKqR6FhAA+EQLTrvFBIJ2jxha3oilXg== 2024-09-10T22:37:07.856 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQATyuBmtONGDRAAYvc2uMwF/DKXpm9xKnfBxA== 2024-09-10T22:37:08.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQAUyuBmingpChAACqS28huIv3Iu9gBuPBEGyA== 2024-09-10T22:37:08.754 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial monmap... 2024-09-10T22:37:09.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:37:09.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T22:37:09.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:09.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool for a [v2:172.21.15.96:3300,v1:172.21.15.96:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: set fsid to 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:37:09.711 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mon... 2024-09-10T22:37:11.378 INFO:teuthology.orchestra.run.smithi096.stdout:create mon.a on 2024-09-10T22:37:11.600 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T22:37:11.741 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T22:37:11.912 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target. 2024-09-10T22:37:11.912 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target. 2024-09-10T22:37:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a 2024-09-10T22:37:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T22:37:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target.wants/ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968@.service. 2024-09-10T22:37:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:37:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-09-10T22:37:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:37:12.835 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:37:12.836 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-09-10T22:37:12.836 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:37:12.836 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon to start... 2024-09-10T22:37:12.836 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon... 2024-09-10T22:37:14.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:13 smithi096 ceph-mon[30505]: from='client.? 172.21.15.96:0/2419915244' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T22:37:14.100 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T22:37:14.100 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout id: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:14.100 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout services: 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.543963s) 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout data: 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:14.101 INFO:teuthology.orchestra.run.smithi096.stdout:mon is available 2024-09-10T22:37:14.102 INFO:teuthology.orchestra.run.smithi096.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T22:37:14.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:14 smithi096 ceph-mon[30505]: from='client.? 172.21.15.96:0/3096279449' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:37:14.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:14 smithi096 ceph-mon[30505]: from='client.? 172.21.15.96:0/3096279449' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:37:15.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:37:15.182 INFO:teuthology.orchestra.run.smithi096.stdout:Generating new minimal ceph.conf... 2024-09-10T22:37:16.174 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:15 smithi096 ceph-mon[30505]: from='client.? 172.21.15.96:0/1228061500' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:37:16.177 INFO:teuthology.orchestra.run.smithi096.stdout:Restarting the monitor... 2024-09-10T22:37:16.445 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:16 smithi096 systemd[1]: Stopping Ceph mon.a for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:37:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a[30501]: 2024-09-10T22:37:16.443+0000 7efe2f53d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:37:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a[30501]: 2024-09-10T22:37:16.443+0000 7efe2f53d640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T22:37:16.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:16 smithi096 podman[30746]: 2024-09-10 22:37:16.679127607 +0000 UTC m=+0.343856718 container died 44bafe62d0a63b2612162d4f2196fc7f234c9c5e0e3f08792f2a2dc503c6b32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:37:17.378 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30746]: 2024-09-10 22:37:17.176537543 +0000 UTC m=+0.841266654 container cleanup 44bafe62d0a63b2612162d4f2196fc7f234c9c5e0e3f08792f2a2dc503c6b32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:37:17.378 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 bash[30746]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a 2024-09-10T22:37:17.707 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30759]: 2024-09-10 22:37:17.377493459 +0000 UTC m=+0.702310546 container remove 44bafe62d0a63b2612162d4f2196fc7f234c9c5e0e3f08792f2a2dc503c6b32f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T22:37:17.707 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T22:37:17.707 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 systemd[1]: Stopped Ceph mon.a for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:37:17.707 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 systemd[1]: Starting Ceph mon.a for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:37:17.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30850]: 2024-09-10 22:37:17.60916764 +0000 UTC m=+0.016028433 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:37:17.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30850]: 2024-09-10 22:37:17.711506091 +0000 UTC m=+0.118366918 container create d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T22:37:17.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30850]: 2024-09-10 22:37:17.938822127 +0000 UTC m=+0.345682914 container init d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T22:37:17.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 podman[30850]: 2024-09-10 22:37:17.941887783 +0000 UTC m=+0.348748570 container start d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T22:37:18.052 INFO:teuthology.orchestra.run.smithi096.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T22:37:18.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:37:18.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:37:18.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: pidfile_write: ignore empty --pid-file 2024-09-10T22:37:18.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: load: jerasure load: lrc 2024-09-10T22:37:18.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Git sha 0 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: DB SUMMARY 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: DB Session ID: 3J2DKIY431J31RPD230L 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 96274 ; 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:37:18.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.env: 0x55ba21672180 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.info_log: 0x55ba21f27700 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.statistics: (nil) 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.use_fsync: 0 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:37:18.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.db_log_dir: 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.wal_dir: 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:37:18.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.write_buffer_manager: 0x55ba21f2bb80 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:37:18.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.unordered_write: 0 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.row_cache: None 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.wal_filter: None 2024-09-10T22:37:18.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.wal_compression: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:37:18.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:37:18.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:37:18.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:37:18.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:37:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_open_files: -1 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Compression algorithms supported: 2024-09-10T22:37:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kZSTD supported: 0 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.merge_operator: 2024-09-10T22:37:18.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_filter: None 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba21f27320) 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_type: 0 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_index_type: 0 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_shortening: 1 2024-09-10T22:37:18.253 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: checksum: 4 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: no_block_cache: 0 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache: 0x55ba21f1b610 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_options: 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: capacity : 536870912 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_shard_bits : 4 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: strict_capacity_limit : 0 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_compressed: (nil) 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: persistent_cache: (nil) 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size: 4096 2024-09-10T22:37:18.254 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size_deviation: 10 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_restart_interval: 16 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_block_restart_interval: 1 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: metadata_block_size: 4096 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: partition_filters: 0 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: use_delta_encoding: 1 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: filter_policy: bloomfilter 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: whole_key_filtering: 1 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: verify_compression: 0 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: format_version: 5 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_index_compression: 1 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_align: 0 2024-09-10T22:37:18.255 INFO:journalctl@ceph.mon.a.smithi096.stdout: max_auto_readahead_size: 262144 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout: prepopulate_block_cache: 0 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression: NoCompression 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.num_levels: 7 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:37:18.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:37:18.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:37:18.258 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:37:18.259 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:37:18.260 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.ttl: 2592000 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:37:18.261 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0cbded4e-136d-4e30-8d78-3d970b1ccd20 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007837968199, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007837969083, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92325, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 289, "table_properties": {"data_size": 90372, "index_size": 273, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11960, "raw_average_key_size": 48, "raw_value_size": 83641, "raw_average_value_size": 337, "num_data_blocks": 12, "num_entries": 248, "num_filter_entries": 248, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726007837, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0cbded4e-136d-4e30-8d78-3d970b1ccd20", "db_session_id": "3J2DKIY431J31RPD230L", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:37:18.262 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007837969176, "job": 1, "event": "recovery_finished"} 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ba21f4ae00 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: DB pointer 0x55ba2204c000 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** DB Stats ** 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:37:18.263 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: L0 2/0 92.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Sum 2/0 92.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.264 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative compaction: 0.00 GB write, 29.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval compaction: 0.00 GB write, 29.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache BinnedLRUCache@0x55ba21f1b610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: starting mon.a rank 0 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:18.265 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???) e1 preinit fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).mds e1 new map 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).mds e1 print_map 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: e1 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: btime 2024-09-10T22:37:12:788415+0000 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: legacy client fscid: -1 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout: No filesystems configured 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:37:18.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:17 smithi096 ceph-mon[30869]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: monmap epoch 1 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: last_changed 2024-09-10T22:37:09.093247+0000 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: min_mon_release 19 (squid) 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: election_strategy: 1 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: fsmap 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 ceph-mon[30869]: mgrmap e1: no daemons active 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 bash[30850]: d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 2024-09-10T22:37:18.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:18 smithi096 systemd[1]: Started Ceph mon.a for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:37:19.023 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T22:37:19.045 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:37:19.045 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mgr... 2024-09-10T22:37:19.045 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T22:37:19.046 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T22:37:19.288 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x 2024-09-10T22:37:19.288 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T22:37:19.506 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968.target.wants/ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-14afb286-6fc5-11ef-bcea-c7b262605968@.service. 2024-09-10T22:37:19.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:19 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2294110831' entity='client.admin' 2024-09-10T22:37:19.737 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:19 smithi096 systemd[1]: Starting Ceph mgr.x for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:37:20.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:19 smithi096 podman[31081]: 2024-09-10 22:37:19.638306486 +0000 UTC m=+0.015520576 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:37:20.350 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:20 smithi096 podman[31081]: 2024-09-10 22:37:20.063820284 +0000 UTC m=+0.441034415 container create 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T22:37:20.350 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:20 smithi096 podman[31081]: 2024-09-10 22:37:20.274556904 +0000 UTC m=+0.651770988 container init 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:37:20.350 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:20 smithi096 podman[31081]: 2024-09-10 22:37:20.277545057 +0000 UTC m=+0.654759137 container start 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:37:20.364 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:37:20.364 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-09-10T22:37:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:37:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:37:20.372 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-09-10T22:37:20.372 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T22:37:20.380 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:37:20.380 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr to start... 2024-09-10T22:37:20.388 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr... 2024-09-10T22:37:20.614 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:20 smithi096 bash[31081]: 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 2024-09-10T22:37:20.614 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:20 smithi096 systemd[1]: Started Ceph mgr.x for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:37:21.190 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3266467982' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:37:21.448 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:37:21.449 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:37:21.450 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:37:21.451 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:37:12:788415+0000", 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:37:21.452 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:37:21.453 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:37:12.789092+0000", 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:21.454 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (1/15)... 2024-09-10T22:37:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:23 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2050574644' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:37:24.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:37:24.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:37:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:37:24.372 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:37:24.373 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:37:12:788415+0000", 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:37:12.789092+0000", 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (2/15)... 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: Activating manager daemon x 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: mgrmap e2: x(active, starting, since 0.00334833s) 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: Manager daemon x is now available 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:37:25.652 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' 2024-09-10T22:37:25.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' 2024-09-10T22:37:25.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:25 smithi096 ceph-mon[30869]: from='mgr.14100 172.21.15.96:0/651780434' entity='mgr.x' 2024-09-10T22:37:27.668 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:27 smithi096 ceph-mon[30869]: mgrmap e3: x(active, since 1.00692s) 2024-09-10T22:37:27.668 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:27 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/52338710' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T22:37:27.670 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:27.672 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:27.672 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T22:37:27.674 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T22:37:27.678 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T22:37:12:788415+0000", 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T22:37:27.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T22:37:12.789092+0000", 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:27.680 INFO:teuthology.orchestra.run.smithi096.stdout:mgr is available 2024-09-10T22:37:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:28 smithi096 ceph-mon[30869]: mgrmap e4: x(active, since 2s) 2024-09-10T22:37:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:28 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3369310047' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T22:37:28.779 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:28.800 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T22:37:28.800 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T22:37:28.801 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T22:37:28.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T22:37:28.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T22:37:28.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T22:37:28.802 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling cephadm module... 2024-09-10T22:37:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:29 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/657634013' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T22:37:29.750 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:29 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ignoring --setuser ceph since I am not root 2024-09-10T22:37:29.750 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:29 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ignoring --setgroup ceph since I am not root 2024-09-10T22:37:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:30 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/657634013' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T22:37:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:30 smithi096 ceph-mon[30869]: mgrmap e5: x(active, since 4s) 2024-09-10T22:37:31.073 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-09-10T22:37:31.074 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 5... 2024-09-10T22:37:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:31 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1239684460' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: Active manager daemon x restarted 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: Activating manager daemon x 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: mgrmap e6: x(active, starting, since 0.00361495s) 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:37:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:37:34.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:37:34.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:34 smithi096 ceph-mon[30869]: Manager daemon x is now available 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 5 is available 2024-09-10T22:37:35.913 INFO:teuthology.orchestra.run.smithi096.stdout:Setting orchestrator backend to cephadm... 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: Found migration_current of "None". Setting to last migration. 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:36.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: mgrmap e7: x(active, since 1.00576s) 2024-09-10T22:37:36.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-mon[30869]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:37:36.750 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:37:36.435+0000 7fefd5603640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:37:36] ENGINE Error in HTTPServer.serve 2024-09-10T22:37:36.750 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: Traceback (most recent call last): 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._connections.run(self.expiration_interval) 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._run(expiration_interval) 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: s = self.context.wrap_socket( 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: return self.sslsocket_class._create( 2024-09-10T22:37:36.751 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:37:36.752 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self.do_handshake() 2024-09-10T22:37:36.752 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:37:36.752 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._sslobj.do_handshake() 2024-09-10T22:37:36.752 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:37:36.752 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:37:37.141 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:36] ENGINE Bus STARTING 2024-09-10T22:37:37.141 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:36] ENGINE Serving on https://172.21.15.96:7150 2024-09-10T22:37:37.141 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:36] ENGINE Error in HTTPServer.serve 2024-09-10T22:37:37.141 INFO:journalctl@ceph.mon.a.smithi096.stdout: Traceback (most recent call last): 2024-09-10T22:37:37.141 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._run(expiration_interval) 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: s = self.context.wrap_socket( 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: return self.sslsocket_class._create( 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:37:37.142 INFO:journalctl@ceph.mon.a.smithi096.stdout: self.do_handshake() 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._sslobj.do_handshake() 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:37.143 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:37 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:38.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T22:37:38.249 INFO:teuthology.orchestra.run.smithi096.stdout:Generating ssh key... 2024-09-10T22:37:38.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:38 smithi096 ceph-mon[30869]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:38.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:38 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:36] ENGINE Serving on http://172.21.15.96:8765 2024-09-10T22:37:38.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:38 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:36] ENGINE Bus STARTED 2024-09-10T22:37:38.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:38 smithi096 ceph-mon[30869]: mgrmap e8: x(active, since 2s) 2024-09-10T22:37:39.310 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-mon[30869]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:39.310 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: Generating public/private rsa key pair. 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: Your identification has been saved in /tmp/tmpstwvby8b/key 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: Your public key has been saved in /tmp/tmpstwvby8b/key.pub 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: The key fingerprint is: 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: SHA256:xKfVwvDznMnK9EExVuSHO00UKREqJijMsMvfQoxlAEI ceph-14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: The key's randomart image is: 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: +---[RSA 3072]----+ 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: |=E . +==.+| 2024-09-10T22:37:39.311 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: |. * .. + o.= + | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | . * . .ooB.o + o| 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: |. * . .o+.B o = | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | + o S . B o .| 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | o . o o . . | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | o . o . | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | . | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: | | 2024-09-10T22:37:39.312 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: +----[SHA256]-----+ 2024-09-10T22:37:40.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:40 smithi096 ceph-mon[30869]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:40.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:40 smithi096 ceph-mon[30869]: Generating ssh key... 2024-09-10T22:37:40.736 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:37:40.736 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:37:40.736 INFO:teuthology.orchestra.run.smithi096.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T22:37:40.745 INFO:teuthology.orchestra.run.smithi096.stdout:Adding host smithi096... 2024-09-10T22:37:41.136 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:41 smithi096 ceph-mon[30869]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:42.073 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:42 smithi096 ceph-mon[30869]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:43.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:43 smithi096 ceph-mon[30869]: Deploying cephadm binary to smithi096 2024-09-10T22:37:45.387 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Added host 'smithi096' with addr '172.21.15.96' 2024-09-10T22:37:45.387 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mon service... 2024-09-10T22:37:45.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:45 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:45.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:45 smithi096 ceph-mon[30869]: Added host smithi096 2024-09-10T22:37:45.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:45 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:46.905 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T22:37:46.905 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mgr service... 2024-09-10T22:37:47.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:47 smithi096 ceph-mon[30869]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:47.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:47 smithi096 ceph-mon[30869]: Saving service mon spec with placement count:5 2024-09-10T22:37:47.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:47 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:48.138 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T22:37:48.715 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:48 smithi096 ceph-mon[30869]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:37:48.715 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:48 smithi096 ceph-mon[30869]: Saving service mgr spec with placement count:2 2024-09-10T22:37:48.715 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:48 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:48.715 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:48 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:49.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:49 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:49.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2702741785' entity='client.admin' 2024-09-10T22:37:50.584 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling the dashboard module... 2024-09-10T22:37:50.834 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:50 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1906070001' entity='client.admin' 2024-09-10T22:37:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:52 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:52 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1686951654' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T22:37:52.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:52 smithi096 ceph-mon[30869]: from='mgr.14118 172.21.15.96:0/4240384082' entity='mgr.x' 2024-09-10T22:37:52.501 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ignoring --setuser ceph since I am not root 2024-09-10T22:37:52.501 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ignoring --setgroup ceph since I am not root 2024-09-10T22:37:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:53 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1686951654' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T22:37:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:53 smithi096 ceph-mon[30869]: mgrmap e9: x(active, since 17s) 2024-09-10T22:37:54.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:54 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/974050523' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T22:37:54.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-09-10T22:37:54.748 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 9... 2024-09-10T22:37:57.279 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: Active manager daemon x restarted 2024-09-10T22:37:57.279 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: Activating manager daemon x 2024-09-10T22:37:57.279 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T22:37:57.279 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: mgrmap e10: x(active, starting, since 0.00339654s) 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T22:37:57.280 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:57 smithi096 ceph-mon[30869]: Manager daemon x is now available 2024-09-10T22:37:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:37:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T22:37:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T22:37:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T22:37:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: mgrmap e11: x(active, since 1.00586s) 2024-09-10T22:37:58.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-mon[30869]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T22:37:58.837 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-09-10T22:37:58.837 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T22:37:58.838 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T22:37:58.838 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-09-10T22:37:58.838 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 9 is available 2024-09-10T22:37:58.838 INFO:teuthology.orchestra.run.smithi096.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T22:37:59.246 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:37:58.971+0000 7f5ce8665640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:22:37:58] ENGINE Error in HTTPServer.serve 2024-09-10T22:37:59.246 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: Traceback (most recent call last): 2024-09-10T22:37:59.246 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:37:59.246 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._connections.run(self.expiration_interval) 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._run(expiration_interval) 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: s = self.context.wrap_socket( 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: return self.sslsocket_class._create( 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self.do_handshake() 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: self._sslobj.do_handshake() 2024-09-10T22:37:59.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:37:59.248 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:37:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:59 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:58] ENGINE Bus STARTING 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:59 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:58] ENGINE Serving on http://172.21.15.96:8765 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:59 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:58] ENGINE Serving on https://172.21.15.96:7150 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:59 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:58] ENGINE Bus STARTED 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:37:59 smithi096 ceph-mon[30869]: [10/Sep/2024:22:37:58] ENGINE Error in HTTPServer.serve 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout: Traceback (most recent call last): 2024-09-10T22:37:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._connections.run(self.expiration_interval) 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._run(expiration_interval) 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: s = self.context.wrap_socket( 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: return self.sslsocket_class._create( 2024-09-10T22:37:59.501 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T22:37:59.502 INFO:journalctl@ceph.mon.a.smithi096.stdout: self.do_handshake() 2024-09-10T22:37:59.502 INFO:journalctl@ceph.mon.a.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T22:37:59.502 INFO:journalctl@ceph.mon.a.smithi096.stdout: self._sslobj.do_handshake() 2024-09-10T22:37:59.502 INFO:journalctl@ceph.mon.a.smithi096.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T22:38:00.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:00 smithi096 ceph-mon[30869]: mgrmap e12: x(active, since 2s) 2024-09-10T22:38:00.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:00 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:00.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:00 smithi096 ceph-mon[30869]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:00.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:00 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:01.802 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUaIb5Yca8LctzpDDeMTVwBaBYB3wwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIyMzgwMFoXDTM0\nMDkxMTIyMzgwMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1Ljk2MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAsq5Ck2SilrJCiwgwBhVcvbMe5bco+4BMj+8/\nKDByoa/zPrTqu/rN/NjtioEHpp0JffogLXY7rKxZ3z7zZa/oIlzf2gzklbY0+3a0\nRCNWAUa1YWji7Jow7hu+vqnGCz9vN4V/EC0IwYgNzBhERFvkecdwCAA6Nk/crkNh\nXXngbyWUSGs0+T0vF4FhSeOMSQg52Wyvv9oKpLxfwMKnDPZYGwCgyBTex1/MPC+L\nPlTUHSOJCyi+noF72NgJTjsFzsfLZHvsF/qzJCzuX4YvaAFQmqHA6dpUT/+crGto\nGSoB3BbSvHFv8PYctJgsdTuPC7va0+erOduFjCWld9Gv50OIu5wr8D9P2ajJe25v\nOq+ZdoaRmqIv+BKm14eiKNwmfIpiIMkX7p+QbTE57MdzxUw+MUTvwJ6+hoO9R/0Y\nDYhUf/Px6YEAvZvjnProWfIcjhkjAeef9U97Mh81QkqkgdtuppXM54nBQccDViH0\n3mFhQV55hgE1raveQGbQx98TXvSwJRVMINDb1TdPSFNULPweznDWbcz7AJmXwh4P\nshPqCfCRe0YsywOO5sBgPfJIJigGRbwR91TEVrdGbIrBEV98PYt4+V36k3VnIB6Q\n1C/KrYGCgfFWOLSgxhyun5UPt6edeXdZI7bGcgGebLpA0q6hI93oNEE6NmDDx2TT\nFDmgz2kCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDk2LmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPYDAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQCzs0HT1oSRfWNbbfQ8UmGrGpn5dhGZ5UAzrEIU\nzn2go1xG6RlBRvqsrkmF6du3IW5qj8QUPvAaVy1ChEz5grDd9oOO1Fv7fFVBT0lB\nAFS8HhQlY9kJpCSI2xArqjBIHoPxFze3pzyySWuxlT5N/yTp8QDuSHO+5Nf1jHPN\nKB619b0qAxK/RfTh8jcbx6ZxD5YMx9+c/A1cjk5NJE+bXQXES0G8sX4hcGh7RXfR\nx3SmvgDPdWYeRf3nG9tUnPguw41YUAYUnKbW56tODjRnguMGCM1NJkfNBxHJS/hv\nmhsZ4Qz8LE3ld9zJYHXsAKRtVQtJ17Kpvi0dL5PS+y8fNiV/v0ho6xXOCoaTtFSd\nvse2h1Simf+EWLgskhvaCB0GmIv0Le9QP/A61P/YoFB1wk3AeZbTz8yc/NLtku9p\nQfPBxe28aPwUZidoaJFwzwB3MKJCqcKleunODD2robKLdGEQ5KVVPkKhN1BzT8hf\nD780VflXHsrY3Isl5mPr3r6ny2jFUjtIeWoJZYBg/H69QUo6qdB5MlMbU1Q93RDo\n8C06k50TlISmwZv/rFFks/ah9dT0p/r/rUhojT/ueDzdqqAK/Mt4stn6pGDL7AVe\n4CNPr/079oeah9g22+aqomgZ69bH0bZqFkOxC55NMSV+WWF1aR/Af18IuM96tLLK\n2dbjzw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAsq5Ck2SilrJCiwgwBhVcvbMe5bco+4BMj+8/KDByoa/zPrTq\nu/rN/NjtioEHpp0JffogLXY7rKxZ3z7zZa/oIlzf2gzklbY0+3a0RCNWAUa1YWji\n7Jow7hu+vqnGCz9vN4V/EC0IwYgNzBhERFvkecdwCAA6Nk/crkNhXXngbyWUSGs0\n+T0vF4FhSeOMSQg52Wyvv9oKpLxfwMKnDPZYGwCgyBTex1/MPC+LPlTUHSOJCyi+\nnoF72NgJTjsFzsfLZHvsF/qzJCzuX4YvaAFQmqHA6dpUT/+crGtoGSoB3BbSvHFv\n8PYctJgsdTuPC7va0+erOduFjCWld9Gv50OIu5wr8D9P2ajJe25vOq+ZdoaRmqIv\n+BKm14eiKNwmfIpiIMkX7p+QbTE57MdzxUw+MUTvwJ6+hoO9R/0YDYhUf/Px6YEA\nvZvjnProWfIcjhkjAeef9U97Mh81QkqkgdtuppXM54nBQccDViH03mFhQV55hgE1\nraveQGbQx98TXvSwJRVMINDb1TdPSFNULPweznDWbcz7AJmXwh4PshPqCfCRe0Ys\nywOO5sBgPfJIJigGRbwR91TEVrdGbIrBEV98PYt4+V36k3VnIB6Q1C/KrYGCgfFW\nOLSgxhyun5UPt6edeXdZI7bGcgGebLpA0q6hI93oNEE6NmDDx2TTFDmgz2kCAwEA\nAQKCAgBDtjTljT/uEqyT6mzkiUjlXJFLZT8jG+onq3cIIF/4dOidrUq9JNzVwlE2\naRJAqgcUVerGv6T6mS/PoZf0Q5aZGw/7nmIrd+qZFtFWdvubLTQEL467Me1z9lCx\nCT5TOd1IMY9GzoIgcRIlc1a/fwFUwubqKUfmGBqdcUxe85ZBGIEadoITJ4QGiNGZ\nAiHB9VuS0RTDA/p/liFFN569k0ctKG5FlF8eN+f6MNw4KUZeqn4/G9gs0g3gTXIt\nUsGvLFigAjQknC0x+tL0oTGq3MCycLTeCg5kaI3zaxp+WsRspruCjQxqSKH1MCu8\n226oeC6DminxmYaA5y1O/EgIQDf6mPLv9UutdkKRqjMrSRgWbSeQCO6NO7hBhPLs\nowrewvZI3K14G7mDpWZGr4qt0gx8BJM/naG0TyDJ7fZqN1Noqi5IUTTvhXJlZcGt\nfXa1kTq9UakqXYKtKEFIUXMZBHfSWbGOyRHnfY57xy2k+mpTHh3bzPCK468K2iyL\nkoPN4w4m7PPJhm/dg7qF50A7H9v/T0WNRBZmygqqfrUzMmk9peRBIGFSloz7a5w2\n1XLeYNqKfq4GBwEdN3Ofzv6wXMQtbDjk393yzFAylI/hTdFvYsFF5/e3UlZ3TlW8\nK24KQbFOjMoh6nRlfJoDedXK/xxz7eUEG/qexriTjFzmDtmzZwKCAQEA90x/KHUF\nEYUFhF259TcJxG2GrpPEnDgz90z+gHYcUBleIEKZkHuRnohdwzmk5PSmrAoo15gi\njTnJDZcVHpHEbtRQLhDw9gFHPc1SZ9NfUE5nBHtdic5TOSyG25FHcimzOypCcHHZ\nB0JhntXXL6ZOfMenfaSr76lo6GWNuqtOBi4EXtmeW3O2cyNm9z34jAdUEil7IjqK\nyFW9hfV1P19Am9Xs3ActPfgb2+tSOOBM4rqrhLIBaNN1A8BHI7VrbV2A0gn8NPzj\n6WVXk9G9YfnHlYH5Wv5Jfb9AFuJgD/69BDpFgwTt520KOY1Ld5ssT6DoCRHLmMgo\no5ZkTEmA5/hldwKCAQEAuPeycEDQOzfdfrGG3mqyWBAuj7INSjooyQF0aPCJ+eFl\nNDj+5NAfDdjd3jvaxEAamNJrGZK8lL+B5X8aUXgQDbSFXj3mQk0S9TcoLwC735ze\nv/e3rE6qbrTI05Zr3hL5mGP1RZXjaEwY1dZPV+BcVM7EnhsFH8aK4kLyXkKs7odN\ncUJM1cKfll5CKa9r8H1pDMazniTDBLmuXYa5KtBqSp3UDt3T6OsbD4GHIOTmXqCs\nqO5J+KtuLkgTKV5gCd/BIxsqJ/+CLfFlIB9pdImXdj++n6J+PCKN2n6G7kK7pl2m\nUZjwzvhD2oXoDwHa6TwKD8c7AoShgilOvQ+nwxEqHwKCAQAEjuxpGn8Va5ZGclPT\nKUiMzlgFL8QX/RE4JbsrMwIil9UcUx3xg9WTToAr8QwRDFFmKmnGT4+TetUx4u8I\nuTcds9wmfTlzz2I4yWgTcdn36PoNIxj+SUdPbnqt4Wenuw0KsiS3XeHaOGcEPnzQ\n7AfWVPfJYReqOz0ry3NpVW8CviZmeSxqEx/xz10+gW3rZtct3oXeg6W7O5latsOK\nf375YGAVTiyPT9tzmkrRsZC5PSecMovEGuG1R/4Hsnhgc7DELlrvvxmUW8HZf6w1\n+hyrrNGj07wXDUdpJJMQuGlTgp91Ysnf0iq2FNynP7z5WsXsJN/T2Rymp0cCAPjX\nTcTRAoIBACogT8V9HW+/Q3TtXbgeOMwRs2kKtjgnks5kxGx6DBKbIP+J05OtSsVO\nlQlePG7h4qzXj4jdgJGoJB9umUHgxFc6u/1Av8Zt41lx3HcIBcHh6mFyO6TErdBU\n4EBATATh8XKhoWTdxFMmsalSC/QJpz2xcdpIcRxRjaQGd0i8isIwkpSu8uzkTJBk\nm1D/IOg+x8QfJR1pZfkXiPzICDgWC38+/uL75p1vnlQjvfDh8Q2wLmKzEqnN8WQd\nCjArzFezXDtwnmNJsA5iohN01AG0rIhD8FhPG8z6WbBfjL+K903mmlCPMpkQtfN4\n9+cjFwZzpB7/0xK5vkb0vXgVrrGgoSUCggEAB3ndvANxGN5hDRskhPJjCizqAhPB\nQEyHdJdNat3UNACBtnDOaki4Y5VhXVoQK/Z8bA9FTdxRQbBrZqPgbRCboH/4Fol9\ns3RJIHMXGn34Nm8qAgs0TZF5tWuabmmQnSRFExFlIYJkAdqhb5a01fl0LHrnTVau\nccTn53z5PbpRUbxctz6+oK8/f3azylRuc6uJbl9HOmZ0pZaSMulfaCC4jg9Z1VYd\n5NdR5c2xbXRxLStXybdveamE6t2AvWPVxum2dabrGHURZ5lKWEvJzWNztlJzAtvu\n7KYyXiZoCAkwE/QX/YlwLoK6b1uAUsQyla+XFyGmpSgwJ0eS2ZQFlPo1fg==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T22:38:02.899 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T22:38:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:03 smithi096 ceph-mon[30869]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:03 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:04.022 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T22:38:04.022 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial admin user... 2024-09-10T22:38:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:04 smithi096 ceph-mon[30869]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:05.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$tXFIqJ1Dh3siPdR0nNnHnur7rhWaqE5Fqt1c0UDO/6XrAH7R3euaK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726007884, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T22:38:05.370 INFO:teuthology.orchestra.run.smithi096.stdout:Fetching dashboard port number... 2024-09-10T22:38:06.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:05 smithi096 ceph-mon[30869]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:06.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T22:38:06.552 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T22:38:06.553 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-09-10T22:38:06.560 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T22:38:06.560 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-09-10T22:38:06.560 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-09-10T22:38:06.560 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph Dashboard is now available at: 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: URL: https://smithi096.front.sepia.ceph.com:8443/ 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: User: admin 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: Password: qmg1or68iv 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:06.562 INFO:teuthology.orchestra.run.smithi096.stdout:Saving cluster configuration to /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config directory 2024-09-10T22:38:07.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:06 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2638530799' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T22:38:07.776 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: ceph telemetry on 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout:For more information see: 2024-09-10T22:38:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T22:38:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:07.778 INFO:teuthology.orchestra.run.smithi096.stdout:Bootstrap complete. 2024-09-10T22:38:07.815 INFO:tasks.cephadm:Fetching config... 2024-09-10T22:38:07.816 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:07.816 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T22:38:07.832 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T22:38:07.832 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:07.832 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T22:38:07.889 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T22:38:07.889 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:07.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T22:38:07.955 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T22:38:07.955 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:07.955 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T22:38:08.010 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T22:38:08.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:38:08.084 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:08.094 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:38:08.135 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:08.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T22:38:08.190 INFO:teuthology.orchestra.run.smithi121.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6YFKXCaFYnI5P2NhKgNQEyxsRVQBbx7IS8pfeerrMtnBjN5zPtbsqKljDEt2qRjKgV7sGfUqWlbAnvD+jYNqRHINnprnKDUd98YUeRLp58AUIzyXV7r69sD36BcJvttM0hAhfRgGGvyFGU/iBaTNRIEjVvJxlFJuSurXu5+AQ1Emb6CFWvCqO1jqbiZRQatN/h89Zi8o0vm8uWDpJQ5yGgtv9Y13k2Uz65SuVQMxX5WEN8XOc2Ut1qlTXIuxSAid85rW4jMC3NxuT8+VWGgagGbsq6KD7achh9n5GAtUyFxKh1H1eWNrHwi9kA1kwwxcIWJ+aAslvvTG6spxpRc7gRTjRn5YyZdf1R7cysU3ngM9qeobtkQiv9FGFnXoLA8ZOA8OmwhRSc6ReIKe1eXEjk2OigSQlHaMuHHoFf/Ctwvu5oxEtrQq4/9JPfhI0uh11n8ZF8DC7gYA0ish/IxVdZRlKV8R+4QSEy82Eed97yBqdfhpBvnlzQIMEmzvRhr8= ceph-14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:08.203 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T22:38:08.470 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:08 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/370727274' entity='client.admin' 2024-09-10T22:38:09.628 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T22:38:09.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T22:38:10.017 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:10.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2400050169' entity='client.admin' 2024-09-10T22:38:10.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:10.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:10.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"} : dispatch 2024-09-10T22:38:10.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:10.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:10.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:11 smithi096 ceph-mon[30869]: mgrmap e13: x(active, since 12s) 2024-09-10T22:38:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:11 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:11 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:11 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:11 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:11.959 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-09-10T22:38:11.960 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:38:11.960 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:38:11.979 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:38:11.979 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:12.037 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-09-10T22:38:12.038 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch host add smithi117 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: Updating smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:12.389 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:13 smithi096 ceph-mon[30869]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:13 smithi096 ceph-mon[30869]: Updating smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:38:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:14 smithi096 ceph-mon[30869]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:15 smithi096 ceph-mon[30869]: Deploying cephadm binary to smithi117 2024-09-10T22:38:16.238 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-09-10T22:38:16.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:38:17.068 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:17 smithi096 ceph-mon[30869]: Added host smithi117 2024-09-10T22:38:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:17.697 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-09-10T22:38:18.237 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:18 smithi096 ceph-mon[30869]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:18.237 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:18.297 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi121 2024-09-10T22:38:18.298 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:38:18.298 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.conf 2024-09-10T22:38:18.317 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:38:18.317 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:18.375 INFO:tasks.cephadm:Adding host smithi121 to orchestrator... 2024-09-10T22:38:18.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch host add smithi121 2024-09-10T22:38:18.614 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:19 smithi096 ceph-mon[30869]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:38:20.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:20 smithi096 ceph-mon[30869]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:20.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:20 smithi096 ceph-mon[30869]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: Deploying cephadm binary to smithi121 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:22.420 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi121' with addr '172.21.15.121' 2024-09-10T22:38:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T22:38:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:22.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:22.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:22.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:23.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T22:38:23.242 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:23 smithi096 ceph-mon[30869]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:38:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:23.873 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:38:23.873 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}] 2024-09-10T22:38:24.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:24 smithi096 ceph-mon[30869]: Added host smithi121 2024-09-10T22:38:24.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:24 smithi096 ceph-mon[30869]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:24.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:24 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:24.443 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T22:38:24.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T22:38:24.683 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:25.362 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:25 smithi096 ceph-mon[30869]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:38:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-09-10T22:38:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:26 smithi096 ceph-mon[30869]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:26 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1226101164' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:38:26.866 INFO:tasks.cephadm:Adding mon.a on smithi096 2024-09-10T22:38:26.866 INFO:tasks.cephadm:Adding mon.b on smithi117 2024-09-10T22:38:26.866 INFO:tasks.cephadm:Adding mon.c on smithi121 2024-09-10T22:38:26.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c' 2024-09-10T22:38:27.117 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1226101164' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:38:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"} : dispatch 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:38:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:27 smithi096 ceph-mon[30869]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:27.823 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mon update... 2024-09-10T22:38:28.370 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T22:38:28.372 DEBUG:teuthology.orchestra.run.smithi121:mon.c> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T22:38:28.374 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:38:28.375 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:38:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:28 smithi096 ceph-mon[30869]: Deploying daemon mon.c on smithi121 2024-09-10T22:38:29.073 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:29.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:38:29.749 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:37:09.093247Z","created":"2024-09-10T22:37:09.093247Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T22:38:29.750 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-09-10T22:38:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:29 smithi096 ceph-mon[30869]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:30 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/2989781764' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:31 smithi096 ceph-mon[30869]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:32.595 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:38:32.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:38:32.857 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:38:37.955 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:38:37.955 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":2,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:38:32.537146Z","created":"2024-09-10T22:37:09.093247Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:38:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 2 2024-09-10T22:38:37.956 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:38:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: mon.a calling monitor election 2024-09-10T22:38:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: mon.c calling monitor election 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: monmap epoch 2 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:38.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: last_changed 2024-09-10T22:38:32.537146+0000 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: min_mon_release 19 (squid) 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: election_strategy: 1 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: 1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.c 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: fsmap 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: mgrmap e13: x(active, since 40s) 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: overall HEALTH_OK 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:38.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:38.853 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:38 smithi121 ceph-mon[35119]: Deploying daemon mon.b on smithi117 2024-09-10T22:38:38.853 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:38 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/3087313719' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:38.854 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:39.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:38 smithi096 ceph-mon[30869]: Deploying daemon mon.b on smithi117 2024-09-10T22:38:39.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:38 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/3087313719' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:39.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:39.525 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:38:39.525 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:38:39.787 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:38:39.813 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:39 smithi121 ceph-mon[35119]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:40.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:39 smithi096 ceph-mon[30869]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:40.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:38:40.477 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":2,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:38:32.537146Z","created":"2024-09-10T22:37:09.093247Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T22:38:40.477 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 2 2024-09-10T22:38:40.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:40 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/1396186268' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:40 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/1396186268' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:41.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 podman[35062]: 2024-09-10 22:38:41.165486349 +0000 UTC m=+0.118432884 container create fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 podman[35062]: 2024-09-10 22:38:41.400914583 +0000 UTC m=+0.353861074 container init fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 podman[35062]: 2024-09-10 22:38:41.404224393 +0000 UTC m=+0.357170885 container start fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pidfile_write: ignore empty --pid-file 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: load: jerasure load: lrc 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: RocksDB version: 7.9.2 2024-09-10T22:38:41.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Git sha 0 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: DB SUMMARY 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: DB Session ID: Z6YN0UWLDO6BTARDYPR8 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: CURRENT file: CURRENT 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.error_if_exists: 0 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.create_if_missing: 0 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.paranoid_checks: 1 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T22:38:41.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.env: 0x56488e31c180 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.info_log: 0x56488f463700 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.statistics: (nil) 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.use_fsync: 0 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_log_file_size: 0 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T22:38:41.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_fallocate: 1 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.use_direct_reads: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.db_log_dir: 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.wal_dir: 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T22:38:41.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.write_buffer_manager: 0x56488f467b80 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.unordered_write: 0 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T22:38:41.781 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T22:38:41.782 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T22:38:41.782 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.row_cache: None 2024-09-10T22:38:41.782 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.wal_filter: None 2024-09-10T22:38:41.783 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T22:38:41.783 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.two_write_queues: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.wal_compression: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.atomic_flush: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.log_readahead_size: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T22:38:41.784 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_background_jobs: 2 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_background_compactions: -1 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_subcompactions: 1 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T22:38:41.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_open_files: -1 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T22:38:41.786 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_background_flushes: -1 2024-09-10T22:38:41.787 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Compression algorithms supported: 2024-09-10T22:38:41.787 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kZSTD supported: 0 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kXpressCompression supported: 0 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kBZip2Compression supported: 0 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kLZ4Compression supported: 1 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kZlibCompression supported: 1 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: kSnappyCompression supported: 1 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.merge_operator: 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_filter: None 2024-09-10T22:38:41.788 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56488f463320) 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_type: 0 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_index_type: 0 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_shortening: 1 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: checksum: 4 2024-09-10T22:38:41.789 INFO:journalctl@ceph.mon.b.smithi117.stdout: no_block_cache: 0 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache: 0x56488f457610 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_options: 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: capacity : 536870912 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: num_shard_bits : 4 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: strict_capacity_limit : 0 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_compressed: (nil) 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: persistent_cache: (nil) 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size: 4096 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size_deviation: 10 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_restart_interval: 16 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_block_restart_interval: 1 2024-09-10T22:38:41.790 INFO:journalctl@ceph.mon.b.smithi117.stdout: metadata_block_size: 4096 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: partition_filters: 0 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: use_delta_encoding: 1 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: filter_policy: bloomfilter 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: whole_key_filtering: 1 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: verify_compression: 0 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: format_version: 5 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_index_compression: 1 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_align: 0 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: max_auto_readahead_size: 262144 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: prepopulate_block_cache: 0 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: initial_auto_readahead_size: 8192 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T22:38:41.791 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression: NoCompression 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.num_levels: 7 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T22:38:41.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T22:38:41.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T22:38:41.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T22:38:41.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.inplace_update_support: 0 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T22:38:41.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.bloom_locality: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.max_successive_merges: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.ttl: 2592000 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enable_blob_files: false 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.min_blob_size: 0 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T22:38:41.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9e710d6b-115b-4859-b0c4-8d06c9b6c456 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007921432219, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T22:38:41.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007921432782, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726007921, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9e710d6b-115b-4859-b0c4-8d06c9b6c456", "db_session_id": "Z6YN0UWLDO6BTARDYPR8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726007921432909, "job": 1, "event": "recovery_finished"} 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56488f486e00 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: DB pointer 0x56488f588000 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** DB Stats ** 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:38:41.799 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T22:38:41.800 INFO:journalctl@ceph.mon.b.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T22:38:41.801 INFO:journalctl@ceph.mon.b.smithi117.stdout: Block cache BinnedLRUCache@0x56488f457610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: starting mon.b rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(???) e0 preinit fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: e1 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: btime 2024-09-10T22:37:12:788415+0000 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:38:41.802 INFO:journalctl@ceph.mon.b.smithi117.stdout: legacy client fscid: -1 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout: No filesystems configured 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Deploying cephadm binary to smithi121 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.803 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Added host smithi121 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1226101164' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1226101164' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"} : dispatch 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Deploying daemon mon.c on smithi121 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/2989781764' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.a calling monitor election 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.c calling monitor election 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: monmap epoch 2 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:41.808 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: last_changed 2024-09-10T22:38:32.537146+0000 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: min_mon_release 19 (squid) 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: election_strategy: 1 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: 1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.c 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: fsmap 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mgrmap e13: x(active, since 40s) 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: overall HEALTH_OK 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:41.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: Deploying daemon mon.b on smithi117 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/3087313719' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/1396186268' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 ceph-mon[35081]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 bash[35062]: fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 2024-09-10T22:38:41.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:41 smithi117 systemd[1]: Started Ceph mon.b for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:38:42.122 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T22:38:42.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T22:38:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:38:47.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: mon.a calling monitor election 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: mon.c calling monitor election 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: mon.b calling monitor election 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: monmap epoch 3 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: last_changed 2024-09-10T22:38:41.461922+0000 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: min_mon_release 19 (squid) 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: election_strategy: 1 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:38:47.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: 1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.c 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: fsmap 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: mgrmap e13: x(active, since 49s) 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: overall HEALTH_OK 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: mon.a calling monitor election 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: mon.c calling monitor election 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: mon.b calling monitor election 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: monmap epoch 3 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: last_changed 2024-09-10T22:38:41.461922+0000 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: min_mon_release 19 (squid) 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: election_strategy: 1 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: 1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.c 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-10T22:38:47.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: fsmap 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: mgrmap e13: x(active, since 49s) 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: overall HEALTH_OK 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:38:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: mon.a calling monitor election 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: mon.c calling monitor election 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: mon.b calling monitor election 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: monmap epoch 3 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: last_changed 2024-09-10T22:38:41.461922+0000 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: created 2024-09-10T22:37:09.093247+0000 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: min_mon_release 19 (squid) 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: election_strategy: 1 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: 1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.c 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: fsmap 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: mgrmap e13: x(active, since 49s) 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: overall HEALTH_OK 2024-09-10T22:38:47.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.500 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:38:47.500 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":3,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","modified":"2024-09-10T22:38:41.461922Z","created":"2024-09-10T22:37:09.093247Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T22:38:47.501 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 3 2024-09-10T22:38:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:47 smithi121 ceph-mon[35119]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:47.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:47.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:47 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/2713572884' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:47 smithi096 ceph-mon[30869]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:47 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/2713572884' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:48.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:48.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:47 smithi117 ceph-mon[35081]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:48.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:38:48.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:47 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/2713572884' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T22:38:48.398 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T22:38:48.398 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T22:38:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: Updating smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:48.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: Updating smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:49.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: Updating smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:49.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.590 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:38:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: Reconfiguring daemon mon.a on smithi096 2024-09-10T22:38:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:49.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:49.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: Reconfiguring daemon mon.a on smithi096 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:49.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:50.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: Reconfiguring daemon mon.a on smithi096 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:50.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:50.194 INFO:teuthology.orchestra.run.smithi096.stdout:# minimal ceph.conf for 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:50.195 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-09-10T22:38:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:38:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: mon_host = [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2024-09-10T22:38:50.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:50 smithi096 ceph-mon[30869]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:38:50.799 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:50 smithi096 ceph-mon[30869]: Reconfiguring daemon mon.b on smithi117 2024-09-10T22:38:50.799 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:50 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/433157455' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:50.800 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T22:38:50.800 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:50.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:38:50.828 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:38:50.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:50.895 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:38:50.895 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:38:50.918 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:50 smithi117 ceph-mon[35081]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:38:50.918 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:50 smithi117 ceph-mon[35081]: Reconfiguring daemon mon.b on smithi117 2024-09-10T22:38:50.918 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:50 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/433157455' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:50.922 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:38:50.923 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:50.989 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:38:50.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T22:38:51.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:50 smithi121 ceph-mon[35119]: Reconfiguring mon.b (monmap changed)... 2024-09-10T22:38:51.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:50 smithi121 ceph-mon[35119]: Reconfiguring daemon mon.b on smithi117 2024-09-10T22:38:51.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:50 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/433157455' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:51.016 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:38:51.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T22:38:51.084 INFO:tasks.cephadm:Adding mgr.x on smithi096 2024-09-10T22:38:51.084 INFO:tasks.cephadm:Adding mgr.y on smithi117 2024-09-10T22:38:51.084 INFO:tasks.cephadm:Adding mgr.z on smithi121 2024-09-10T22:38:51.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi096=x;smithi117=y;smithi121=z' 2024-09-10T22:38:51.696 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:38:52.081 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: Reconfiguring daemon mon.c on smithi121 2024-09-10T22:38:52.082 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:51 smithi121 ceph-mon[35119]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:52.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: Reconfiguring daemon mon.c on smithi121 2024-09-10T22:38:52.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:51 smithi096 ceph-mon[30869]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: Reconfiguring mon.c (monmap changed)... 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: Reconfiguring daemon mon.c on smithi121 2024-09-10T22:38:52.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:51 smithi117 ceph-mon[35081]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:52.622 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mgr update... 2024-09-10T22:38:53.144 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T22:38:53.146 DEBUG:teuthology.orchestra.run.smithi121:mgr.z> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T22:38:53.149 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T22:38:53.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T22:38:53.152 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T22:38:53.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T22:38:53.195 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T22:38:53.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T22:38:53.238 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T22:38:53.238 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T22:38:53.281 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T22:38:53.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T22:38:53.324 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T22:38:53.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T22:38:53.367 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T22:38:53.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T22:38:53.411 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T22:38:53.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: Saving service mgr spec with placement smithi096=x;smithi117=y;smithi121=z;count:3 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:53.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: Deploying daemon mgr.z on smithi121 2024-09-10T22:38:53.447 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:53 smithi096 ceph-mon[30869]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:53.449 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T22:38:53.449 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T22:38:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:53.465 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T22:38:53.465 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: Saving service mgr spec with placement smithi096=x;smithi117=y;smithi121=z;count:3 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:53.468 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.469 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: Deploying daemon mgr.z on smithi121 2024-09-10T22:38:53.469 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:53 smithi121 ceph-mon[35119]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:53.519 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T22:38:53.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: Saving service mgr spec with placement smithi096=x;smithi117=y;smithi121=z;count:3 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: Deploying daemon mgr.z on smithi121 2024-09-10T22:38:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:53 smithi117 ceph-mon[35081]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:53.565 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T22:38:53.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T22:38:53.613 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T22:38:53.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T22:38:53.656 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T22:38:53.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T22:38:53.701 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T22:38:53.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T22:38:53.747 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T22:38:53.747 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T22:38:53.790 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T22:38:53.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T22:38:53.834 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T22:38:53.834 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T22:38:53.888 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T22:38:53.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T22:38:53.937 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T22:38:53.937 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T22:38:53.981 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T22:38:53.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T22:38:54.033 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T22:38:54.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T22:38:54.084 DEBUG:tasks.cephadm:set 22 configs 2024-09-10T22:38:54.084 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi096') 2024-09-10T22:38:54.229 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:54.320 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:54.909 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:54 smithi121 systemd[1]: Starting Ceph mgr.z for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:38:55.169 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:54 smithi121 podman[36699]: 2024-09-10 22:38:54.907607432 +0000 UTC m=+0.112309939 container create 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T22:38:55.169 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:54 smithi121 podman[36699]: 2024-09-10 22:38:54.810976336 +0000 UTC m=+0.015678799 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:38:55.169 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:55 smithi121 podman[36699]: 2024-09-10 22:38:55.101469402 +0000 UTC m=+0.306171873 container init 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:38:55.170 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:55 smithi121 podman[36699]: 2024-09-10 22:38:55.104251677 +0000 UTC m=+0.308954139 container start 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:38:55.170 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mgr[36718]: -- 172.21.15.121:0/2871235529 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55644de210e0 con 0x55644ddfe800 2024-09-10T22:38:55.170 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:55 smithi121 bash[36699]: 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 2024-09-10T22:38:55.387 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:55.402 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:55.513 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 22:38:55 smithi121 systemd[1]: Started Ceph mgr.z for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:38:55.766 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi096') 2024-09-10T22:38:55.838 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2121368217' entity='client.admin' 2024-09-10T22:38:55.838 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/31393783' entity='client.admin' 2024-09-10T22:38:55.838 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:55.838 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.838 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:55.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:55.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:55.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:55 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2121368217' entity='client.admin' 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/31393783' entity='client.admin' 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:55.896 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:55.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:55.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:55 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:56.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2121368217' entity='client.admin' 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/31393783' entity='client.admin' 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:56.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:55 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:56.159 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi096') 2024-09-10T22:38:56.739 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:56 smithi096 ceph-mon[30869]: Deploying daemon mgr.y on smithi117 2024-09-10T22:38:56.867 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:56 smithi117 ceph-mon[35081]: Deploying daemon mgr.y on smithi117 2024-09-10T22:38:57.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:56 smithi121 ceph-mon[35119]: Deploying daemon mgr.y on smithi117 2024-09-10T22:38:57.243 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 systemd[1]: Starting Ceph mgr.y for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:38:57.424 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 podman[36203]: 2024-09-10 22:38:57.242302347 +0000 UTC m=+0.108135344 container create f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 podman[36203]: 2024-09-10 22:38:57.14987677 +0000 UTC m=+0.015709743 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 podman[36203]: 2024-09-10 22:38:57.436335958 +0000 UTC m=+0.302168927 container init f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 podman[36203]: 2024-09-10 22:38:57.439334227 +0000 UTC m=+0.305167183 container start f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mgr[36222]: -- 172.21.15.117:0/2399398255 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f60a6d50e0 con 0x55f60a6b2800 2024-09-10T22:38:57.512 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 bash[36203]: f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad 2024-09-10T22:38:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:57.627 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:57.627 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:57.629 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:38:57.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.666+0000 7f9370ff9640 1 -- 172.21.15.96:0/2876155987 <== mon.1 v2:172.21.15.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v13)=0 v13) ==== 109+0+0 (secure 0 0 0) 0x7f9364016590 con 0x7f9374101920 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f935c041720 msgr2=0x7f935c043be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f935c041720 0x7f935c043be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9368005ab0 tx=0x7f9368005750 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9374101920 msgr2=0x7f9374196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9374101920 0x7f9374196a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f936400b590 tx=0x7f936400ba50 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 shutdown_connections 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9374103440 0x7f9374197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f935c041720 0x7f935c043be0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9374102a80 0x7f93741970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 --2- 172.21.15.96:0/2876155987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9374101920 0x7f9374196a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 >> 172.21.15.96:0/2876155987 conn(0x7f93740fd1f0 msgr2=0x7f9374105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 shutdown_connections 2024-09-10T22:38:57.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.670+0000 7f937a19e640 1 -- 172.21.15.96:0/2876155987 wait complete. 2024-09-10T22:38:57.776 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 22:38:57 smithi117 systemd[1]: Started Ceph mgr.y for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:38:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1791726114' entity='client.admin' 2024-09-10T22:38:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:57.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:57 smithi117 ceph-mon[35081]: from='client.? ' entity='client.admin' 2024-09-10T22:38:57.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.858+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f4538119b30 con 0x7f45380740c0 2024-09-10T22:38:57.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.864+0000 7f452e7fc640 1 -- 172.21.15.96:0/808299400 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f4534017de0 con 0x7f45380740c0 2024-09-10T22:38:57.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f452403f590 msgr2=0x7f4524041a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:57.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f452403f590 0x7f4524041a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4530009aa0 tx=0x7f4530005dd0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45380740c0 msgr2=0x7f4538116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:57.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45380740c0 0x7f4538116930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f453400b880 tx=0x7f453400bd40 comp rx=0 tx=0).stop 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 shutdown_connections 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4538074a80 0x7f4538116f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f452403f590 0x7f4524041a50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f453810a450 0x7f4538117bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 --2- 172.21.15.96:0/808299400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45380740c0 0x7f4538116930 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 >> 172.21.15.96:0/808299400 conn(0x7f453806fcb0 msgr2=0x7f4538072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 shutdown_connections 2024-09-10T22:38:57.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:57.866+0000 7f453f9e1640 1 -- 172.21.15.96:0/808299400 wait complete. 2024-09-10T22:38:57.868 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1791726114' entity='client.admin' 2024-09-10T22:38:57.868 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:57.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:57.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:57 smithi096 ceph-mon[30869]: from='client.? ' entity='client.admin' 2024-09-10T22:38:58.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1791726114' entity='client.admin' 2024-09-10T22:38:58.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:58.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:58.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:58.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:58.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:58.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:57 smithi121 ceph-mon[35119]: from='client.? ' entity='client.admin' 2024-09-10T22:38:59.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:59.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:58 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/808299400' entity='client.admin' 2024-09-10T22:38:59.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:59.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:58 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/808299400' entity='client.admin' 2024-09-10T22:38:59.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:38:59.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:58 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/808299400' entity='client.admin' 2024-09-10T22:38:59.103 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.477+0000 7fcb00397640 1 -- 172.21.15.96:0/1658507843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8104570 msgr2=0x7fcaf810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.477+0000 7fcb00397640 1 --2- 172.21.15.96:0/1658507843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8104570 0x7fcaf810aef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009930 tx=0x7fcaf002a420 comp rx=0 tx=0).stop 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- 172.21.15.96:0/1658507843 shutdown_connections 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- 172.21.15.96:0/1658507843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- 172.21.15.96:0/1658507843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8104570 0x7fcaf810aef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- 172.21.15.96:0/1658507843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8102a50 0x7fcaf8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- 172.21.15.96:0/1658507843 >> 172.21.15.96:0/1658507843 conn(0x7fcaf80fe320 msgr2=0x7fcaf8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- 172.21.15.96:0/1658507843 shutdown_connections 2024-09-10T22:38:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- 172.21.15.96:0/1658507843 wait complete. 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 Processor -- start 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- start start 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8104570 0x7fcaf8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810e450 con 0x7fcaf8102a50 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810e2d0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcb00397640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810e5d0 con 0x7fcaf8104570 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.478+0000 7fcafe133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafe133640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38554/0 (socket says 172.21.15.96:38554) 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafe133640 1 -- 172.21.15.96:0/3830611177 learned_addr learned my addr 172.21.15.96:0/3830611177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:38:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafe934640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8104570 0x7fcaf8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 -- 172.21.15.96:0/3830611177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8104570 msgr2=0x7fcaf8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8104570 0x7fcaf8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 -- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 msgr2=0x7fcaf8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 -- 172.21.15.96:0/3830611177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf819a7b0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafe133640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcafd932640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf81992d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae800ccc0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fcae80108a0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf819aaa0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8018640 con 0x7fcaf8103bb0 2024-09-10T22:38:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.479+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81a1180 con 0x7fcaf8103bb0 2024-09-10T22:38:59.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.480+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fcae80187e0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.481+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf8102e50 con 0x7fcaf8103bb0 2024-09-10T22:38:59.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.481+0000 7fcaef7fe640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 0x7fcae40419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.481+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcae8054410 con 0x7fcaf8103bb0 2024-09-10T22:38:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.481+0000 7fcafe133640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 0x7fcae40419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:59.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.481+0000 7fcafe133640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 0x7fcae40419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40049d0 tx=0x7fcaf4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:59.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.486+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcae8014d70 con 0x7fcaf8103bb0 2024-09-10T22:38:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.668+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fcaf802e8c0 con 0x7fcaf8103bb0 2024-09-10T22:38:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.673+0000 7fcaef7fe640 1 -- 172.21.15.96:0/3830611177 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v15)=0 v15) ==== 115+0+0 (secure 0 0 0) 0x7fcae801fe00 con 0x7fcaf8103bb0 2024-09-10T22:38:59.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 msgr2=0x7fcae40419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 0x7fcae40419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40049d0 tx=0x7fcaf4009500 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 msgr2=0x7fcaf81992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf81992d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcae800e9f0 tx=0x7fcae800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 shutdown_connections 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaf8103bb0 0x7fcaf81992d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcae403f510 0x7fcae40419d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaf8104570 0x7fcaf8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 --2- 172.21.15.96:0/3830611177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcaf8102a50 0x7fcaf8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 >> 172.21.15.96:0/3830611177 conn(0x7fcaf80fe320 msgr2=0x7fcaf8107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:59.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 shutdown_connections 2024-09-10T22:38:59.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:38:59.675+0000 7fcb00397640 1 -- 172.21.15.96:0/3830611177 wait complete. 2024-09-10T22:38:59.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:59.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:59.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: Reconfiguring daemon mgr.x on smithi096 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:38:59.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:38:59 smithi096 ceph-mon[30869]: from='client.? ' entity='client.admin' 2024-09-10T22:39:00.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:39:00.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:39:00.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:00.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: Reconfiguring daemon mgr.x on smithi096 2024-09-10T22:39:00.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:00.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:38:59 smithi121 ceph-mon[35119]: from='client.? ' entity='client.admin' 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:00.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: Reconfiguring daemon mgr.x on smithi096 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:00.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:38:59 smithi117 ceph-mon[35081]: from='client.? ' entity='client.admin' 2024-09-10T22:39:00.327 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:00.327 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:00.328 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:00.329 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:00.330 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:01.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:00 smithi096 ceph-mon[30869]: Standby manager daemon z started 2024-09-10T22:39:01.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:00 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:39:01.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:00 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:01.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:00 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:39:01.153 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:00 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:01.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:00 smithi121 ceph-mon[35119]: Standby manager daemon z started 2024-09-10T22:39:01.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:00 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:39:01.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:00 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:01.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:00 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:39:01.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:00 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:00 smithi117 ceph-mon[35081]: Standby manager daemon z started 2024-09-10T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:00 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:00 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:00 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:00 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.121:0/927323056' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 -- 172.21.15.96:0/3827827689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120103460 msgr2=0x7f3120109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 --2- 172.21.15.96:0/3827827689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120103460 0x7f3120109de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3118009cd0 tx=0x7f311802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 -- 172.21.15.96:0/3827827689 shutdown_connections 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 --2- 172.21.15.96:0/3827827689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120103460 0x7f3120109de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 --2- 172.21.15.96:0/3827827689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 0x7f3120102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 --2- 172.21.15.96:0/3827827689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120101940 0x7f3120101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 -- 172.21.15.96:0/3827827689 >> 172.21.15.96:0/3827827689 conn(0x7f31200fd1f0 msgr2=0x7f31200ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 -- 172.21.15.96:0/3827827689 shutdown_connections 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 -- 172.21.15.96:0/3827827689 wait complete. 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.652+0000 7f3126cd8640 1 Processor -- start 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 -- start start 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 0x7f3120198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 0x7f3120198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f312010d3a0 con 0x7f3120101940 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f312010d220 con 0x7f3120103460 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f312010d520 con 0x7f3120102aa0 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38586/0 (socket says 172.21.15.96:38586) 2024-09-10T22:39:01.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 -- 172.21.15.96:0/2596462070 learned_addr learned my addr 172.21.15.96:0/2596462070 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3125275640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 0x7f3120198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f311ffff640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 0x7f3120198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 -- 172.21.15.96:0/2596462070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 msgr2=0x7f3120198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 0x7f3120198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 -- 172.21.15.96:0/2596462070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 msgr2=0x7f3120198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 0x7f3120198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 -- 172.21.15.96:0/2596462070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3120199660 con 0x7f3120101940 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3125275640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 0x7f3120198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3124a74640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114017030 con 0x7f3120101940 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f3114004020 con 0x7f3120101940 2024-09-10T22:39:01.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3120199950 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.653+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114005580 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.654+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201a0030 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.655+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3120101d40 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.655+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f3114009a30 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.655+0000 7f311dffb640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 0x7f311007c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.656+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f311409b820 con 0x7f3120101940 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.656+0000 7f311ffff640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 0x7f311007c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.656+0000 7f311ffff640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 0x7f311007c870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f310c009890 tx=0x7f310c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:01.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.660+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3114064550 con 0x7f3120101940 2024-09-10T22:39:01.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.850+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f31201a0320 con 0x7f3120101940 2024-09-10T22:39:01.854 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:01.854 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:01.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.854+0000 7f311dffb640 1 -- 172.21.15.96:0/2596462070 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f3114063e50 con 0x7f3120101940 2024-09-10T22:39:01.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.856+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 msgr2=0x7f311007c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:01.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 0x7f311007c870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f310c009890 tx=0x7f310c008040 comp rx=0 tx=0).stop 2024-09-10T22:39:01.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 msgr2=0x7f3120197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:01.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400de10 comp rx=0 tx=0).stop 2024-09-10T22:39:01.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 shutdown_connections 2024-09-10T22:39:01.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3120103460 0x7f3120198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f311007a3b0 0x7f311007c870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3120102aa0 0x7f3120198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 --2- 172.21.15.96:0/2596462070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3120101940 0x7f3120197b30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 >> 172.21.15.96:0/2596462070 conn(0x7f31200fd1f0 msgr2=0x7f3120103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 shutdown_connections 2024-09-10T22:39:01.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.857+0000 7f3126cd8640 1 -- 172.21.15.96:0/2596462070 wait complete. 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 -- 172.21.15.96:0/1601571680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102bd0 msgr2=0x7f023c109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 --2- 172.21.15.96:0/1601571680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102bd0 0x7f023c109550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0234009930 tx=0x7f023402a420 comp rx=0 tx=0).stop 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 -- 172.21.15.96:0/1601571680 shutdown_connections 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 --2- 172.21.15.96:0/1601571680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102210 0x7f023c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 --2- 172.21.15.96:0/1601571680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c102bd0 0x7f023c109550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 --2- 172.21.15.96:0/1601571680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c1010b0 0x7f023c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 -- 172.21.15.96:0/1601571680 >> 172.21.15.96:0/1601571680 conn(0x7f023c0fc9a0 msgr2=0x7f023c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 -- 172.21.15.96:0/1601571680 shutdown_connections 2024-09-10T22:39:02.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.014+0000 7f024360f640 1 -- 172.21.15.96:0/1601571680 wait complete. 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 Processor -- start 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 -- start start 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c102210 0x7f023c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102bd0 0x7f023c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10d390 con 0x7f023c1010b0 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10d210 con 0x7f023c102bd0 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f024360f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10d510 con 0x7f023c102210 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f02413ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f02413ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38606/0 (socket says 172.21.15.96:38606) 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f02413ab640 1 -- 172.21.15.96:0/3712542748 learned_addr learned my addr 172.21.15.96:0/3712542748 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:02.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.015+0000 7f0241bac640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102bd0 0x7f023c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f0240baa640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c102210 0x7f023c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 -- 172.21.15.96:0/3712542748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c102210 msgr2=0x7f023c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c102210 0x7f023c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 -- 172.21.15.96:0/3712542748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102bd0 msgr2=0x7f023c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102bd0 0x7f023c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 -- 172.21.15.96:0/3712542748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c19a700 con 0x7f023c1010b0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f02413ab640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cce0 tx=0x7f022c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c004b20 con 0x7f023c1010b0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f022c004cc0 con 0x7f023c1010b0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c19a9f0 con 0x7f023c1010b0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c00f6b0 con 0x7f023c1010b0 2024-09-10T22:39:02.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.016+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c1a10d0 con 0x7f023c1010b0 2024-09-10T22:39:02.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.017+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c1014b0 con 0x7f023c1010b0 2024-09-10T22:39:02.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.018+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f022c005020 con 0x7f023c1010b0 2024-09-10T22:39:02.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.018+0000 7f023a7fc640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 0x7f022407c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.018+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f022c09be60 con 0x7f023c1010b0 2024-09-10T22:39:02.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.018+0000 7f0240baa640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 0x7f022407c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.019+0000 7f0240baa640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 0x7f022407c870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0230005fd0 tx=0x7f023000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.022+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f022c014030 con 0x7f023c1010b0 2024-09-10T22:39:02.024 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:01 smithi096 ceph-mon[30869]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:39:02.024 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:39:02.024 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:01 smithi096 ceph-mon[30869]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:02.024 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:01 smithi096 ceph-mon[30869]: from='client.? ' entity='client.admin' 2024-09-10T22:39:02.024 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:01 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2596462070' entity='client.admin' 2024-09-10T22:39:02.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- 172.21.15.96:0/2603863198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 msgr2=0x7f4884102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 --2- 172.21.15.96:0/2603863198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f4884102f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4878009930 tx=0x7f487802a420 comp rx=0 tx=0).stop 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- 172.21.15.96:0/2603863198 shutdown_connections 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 --2- 172.21.15.96:0/2603863198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884103460 0x7f4884109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 --2- 172.21.15.96:0/2603863198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f4884102f20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 --2- 172.21.15.96:0/2603863198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884101940 0x7f4884101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- 172.21.15.96:0/2603863198 >> 172.21.15.96:0/2603863198 conn(0x7f48840fd210 msgr2=0x7f48840ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- 172.21.15.96:0/2603863198 shutdown_connections 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- 172.21.15.96:0/2603863198 wait complete. 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 Processor -- start 2024-09-10T22:39:02.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.183+0000 7f488a914640 1 -- start start 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884101940 0x7f4884198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884103460 0x7f4884199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f488410d370 con 0x7f4884102aa0 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f488410d1f0 con 0x7f4884101940 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f488a914640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f488410d4f0 con 0x7f4884103460 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38622/0 (socket says 172.21.15.96:38622) 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 -- 172.21.15.96:0/2931145647 learned_addr learned my addr 172.21.15.96:0/2931145647 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:02.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f4883fff640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884101940 0x7f4884198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f4888eb1640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884103460 0x7f4884199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 -- 172.21.15.96:0/2931145647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884103460 msgr2=0x7f4884199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884103460 0x7f4884199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 -- 172.21.15.96:0/2931145647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884101940 msgr2=0x7f4884198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884101940 0x7f4884198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 -- 172.21.15.96:0/2931145647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488419a7c0 con 0x7f4884102aa0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48837fe640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f487802a940 tx=0x7f4878005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.184+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487803f070 con 0x7f4884102aa0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.185+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f4878003c60 con 0x7f4884102aa0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.185+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488419aa50 con 0x7f4884102aa0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.185+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878035030 con 0x7f4884102aa0 2024-09-10T22:39:02.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.185+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841a0fc0 con 0x7f4884102aa0 2024-09-10T22:39:02.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.186+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4884101d40 con 0x7f4884102aa0 2024-09-10T22:39:02.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.187+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f487803bb00 con 0x7f4884102aa0 2024-09-10T22:39:02.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.187+0000 7f48817fa640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 0x7f486c07c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.187+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f48780bc920 con 0x7f4884102aa0 2024-09-10T22:39:02.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.187+0000 7f4883fff640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 0x7f486c07c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.188+0000 7f4883fff640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 0x7f486c07c8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48740058c0 tx=0x7f487400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.191+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f487803be20 con 0x7f4884102aa0 2024-09-10T22:39:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.212+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f023c1a16c0 con 0x7f023c1010b0 2024-09-10T22:39:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.215+0000 7f023a7fc640 1 -- 172.21.15.96:0/3712542748 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v18)=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f022c0644c0 con 0x7f023c1010b0 2024-09-10T22:39:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 msgr2=0x7f022407c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 0x7f022407c870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0230005fd0 tx=0x7f023000b590 comp rx=0 tx=0).stop 2024-09-10T22:39:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 msgr2=0x7f023c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cce0 tx=0x7f022c00a620 comp rx=0 tx=0).stop 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 shutdown_connections 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f023c102bd0 0x7f023c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f022407a3b0 0x7f022407c870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f023c102210 0x7f023c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 --2- 172.21.15.96:0/3712542748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f023c1010b0 0x7f023c198bd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 >> 172.21.15.96:0/3712542748 conn(0x7f023c0fc9a0 msgr2=0x7f023c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 shutdown_connections 2024-09-10T22:39:02.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.218+0000 7f024360f640 1 -- 172.21.15.96:0/3712542748 wait complete. 2024-09-10T22:39:02.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:01 smithi121 ceph-mon[35119]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:39:02.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:39:02.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:01 smithi121 ceph-mon[35119]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:02.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:01 smithi121 ceph-mon[35119]: from='client.? ' entity='client.admin' 2024-09-10T22:39:02.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:01 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2596462070' entity='client.admin' 2024-09-10T22:39:02.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:01 smithi117 ceph-mon[35081]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T22:39:02.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T22:39:02.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:01 smithi117 ceph-mon[35081]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:02.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:01 smithi117 ceph-mon[35081]: from='client.? ' entity='client.admin' 2024-09-10T22:39:02.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:01 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2596462070' entity='client.admin' 2024-09-10T22:39:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 -- 172.21.15.96:0/1494539132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034101910 msgr2=0x7f6034101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 --2- 172.21.15.96:0/1494539132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034101910 0x7f6034101d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6024009930 tx=0x7f602402a420 comp rx=0 tx=0).stop 2024-09-10T22:39:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 -- 172.21.15.96:0/1494539132 shutdown_connections 2024-09-10T22:39:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 --2- 172.21.15.96:0/1494539132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f6034102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 --2- 172.21.15.96:0/1494539132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034103430 0x7f6034109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 --2- 172.21.15.96:0/1494539132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034101910 0x7f6034101d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 -- 172.21.15.96:0/1494539132 >> 172.21.15.96:0/1494539132 conn(0x7f60340fd1e0 msgr2=0x7f60340ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 -- 172.21.15.96:0/1494539132 shutdown_connections 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.233+0000 7f603906f640 1 -- 172.21.15.96:0/1494539132 wait complete. 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 Processor -- start 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 -- start start 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034101910 0x7f603406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f603410d2f0 con 0x7f6034103430 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f603410d170 con 0x7f6034102a70 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603906f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f603410d470 con 0x7f6034101910 2024-09-10T22:39:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60676/0 (socket says 172.21.15.96:60676) 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 -- 172.21.15.96:0/3414324384 learned_addr learned my addr 172.21.15.96:0/3414324384 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38568/0 (socket says 172.21.15.96:38568) 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f6032d9d640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034101910 0x7f603406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 -- 172.21.15.96:0/3414324384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034101910 msgr2=0x7f603406b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034101910 0x7f603406b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 -- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 msgr2=0x7f603419ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.234+0000 7f603259c640 1 -- 172.21.15.96:0/3414324384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60341a0820 con 0x7f6034102a70 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f603359e640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:39:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f603259c640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f602800d950 tx=0x7f602800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028017070 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f6028009c30 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60341a0b10 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602800f620 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.235+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a10a0 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.236+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6034101d10 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.237+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f6028010460 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.237+0000 7f601bfff640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 0x7f601c07c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.237+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f602809ba00 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.237+0000 7f6032d9d640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 0x7f601c07c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.238+0000 7f6032d9d640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 0x7f601c07c870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6024004770 tx=0x7f60240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.242+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6028010e20 con 0x7f6034102a70 2024-09-10T22:39:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.422+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f6034063d50 con 0x7f6034102a70 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.427+0000 7f601bfff640 1 -- 172.21.15.96:0/3414324384 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f6028064040 con 0x7f6034102a70 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 msgr2=0x7f601c07c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 0x7f601c07c870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6024004770 tx=0x7f60240058d0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 msgr2=0x7f603419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f602800d950 tx=0x7f602800de10 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 shutdown_connections 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6034102a70 0x7f603419d970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f601c07a3b0 0x7f601c07c870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6034103430 0x7f603419ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 --2- 172.21.15.96:0/3414324384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6034101910 0x7f603406b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 >> 172.21.15.96:0/3414324384 conn(0x7f60340fd1e0 msgr2=0x7f6034105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 shutdown_connections 2024-09-10T22:39:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:01.429+0000 7f603906f640 1 -- 172.21.15.96:0/3414324384 wait complete. 2024-09-10T22:39:02.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.375+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f48841a1310 con 0x7f4884102aa0 2024-09-10T22:39:02.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.378+0000 7f48817fa640 1 -- 172.21.15.96:0/2931145647 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f4878089b10 con 0x7f4884102aa0 2024-09-10T22:39:02.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 msgr2=0x7f486c07c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 0x7f486c07c8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48740058c0 tx=0x7f487400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 msgr2=0x7f48841992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f487802a940 tx=0x7f4878005280 comp rx=0 tx=0).stop 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 shutdown_connections 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f486c07a400 0x7f486c07c8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4884103460 0x7f4884199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4884102aa0 0x7f48841992e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 --2- 172.21.15.96:0/2931145647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4884101940 0x7f4884198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 >> 172.21.15.96:0/2931145647 conn(0x7f48840fd210 msgr2=0x7f48840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 shutdown_connections 2024-09-10T22:39:02.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:02.381+0000 7f488a914640 1 -- 172.21.15.96:0/2931145647 wait complete. 2024-09-10T22:39:03.301 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3712542748' entity='client.admin' 2024-09-10T22:39:03.301 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2931145647' entity='client.admin' 2024-09-10T22:39:03.301 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: Standby manager daemon y started 2024-09-10T22:39:03.301 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:39:03.301 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:03.302 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:39:03.302 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:03.302 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:03 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.299+0000 7f1c6629a640 1 -- 172.21.15.96:0/4055489001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60101910 msgr2=0x7f1c60101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.299+0000 7f1c6629a640 1 --2- 172.21.15.96:0/4055489001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60101910 0x7f1c60101d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009930 tx=0x7f1c5002a420 comp rx=0 tx=0).stop 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 -- 172.21.15.96:0/4055489001 shutdown_connections 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 --2- 172.21.15.96:0/4055489001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c60102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 --2- 172.21.15.96:0/4055489001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60103430 0x7f1c60109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 --2- 172.21.15.96:0/4055489001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60101910 0x7f1c60101d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 -- 172.21.15.96:0/4055489001 >> 172.21.15.96:0/4055489001 conn(0x7f1c600fd1e0 msgr2=0x7f1c600ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 -- 172.21.15.96:0/4055489001 shutdown_connections 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 -- 172.21.15.96:0/4055489001 wait complete. 2024-09-10T22:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 Processor -- start 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.300+0000 7f1c6629a640 1 -- start start 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60101910 0x7f1c60197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 0x7f1c60198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010d360 con 0x7f1c60103430 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010d1e0 con 0x7f1c60102a70 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c6629a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010d4e0 con 0x7f1c60101910 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c64837640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 0x7f1c60198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60768/0 (socket says 172.21.15.96:60768) 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 -- 172.21.15.96:0/3753388649 learned_addr learned my addr 172.21.15.96:0/3753388649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:03.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5ffff640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60101910 0x7f1c60197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 -- 172.21.15.96:0/3753388649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60101910 msgr2=0x7f1c60197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60101910 0x7f1c60197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.301+0000 7f1c5f7fe640 1 -- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 msgr2=0x7f1c60198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5f7fe640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 0x7f1c60198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5f7fe640 1 -- 172.21.15.96:0/3753388649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c60199680 con 0x7f1c60102a70 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c64837640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 0x7f1c60198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5f7fe640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017030 con 0x7f1c60102a70 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f1c54004020 con 0x7f1c60102a70 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c60199970 con 0x7f1c60102a70 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54005580 con 0x7f1c60102a70 2024-09-10T22:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.302+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6006ac80 con 0x7f1c60102a70 2024-09-10T22:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.303+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c60101d10 con 0x7f1c60102a70 2024-09-10T22:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.304+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f1c54009a30 con 0x7f1c60102a70 2024-09-10T22:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.305+0000 7f1c5d7fa640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 0x7f1c4807c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.305+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1c5409b3d0 con 0x7f1c60102a70 2024-09-10T22:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.305+0000 7f1c5ffff640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 0x7f1c4807c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:03.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.305+0000 7f1c5ffff640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 0x7f1c4807c870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005ab0 tx=0x7f1c50005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:03.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.308+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1c54063c50 con 0x7f1c60102a70 2024-09-10T22:39:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.385+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f1c5405fec0 con 0x7f1c60102a70 2024-09-10T22:39:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.505+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f1c6006b700 con 0x7f1c60102a70 2024-09-10T22:39:03.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3712542748' entity='client.admin' 2024-09-10T22:39:03.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2931145647' entity='client.admin' 2024-09-10T22:39:03.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: Standby manager daemon y started 2024-09-10T22:39:03.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:39:03.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:03.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:39:03.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:03.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:03 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.512+0000 7f1c5d7fa640 1 -- 172.21.15.96:0/3753388649 <== mon.2 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v21)=0 v21) ==== 106+0+0 (secure 0 0 0) 0x7f1c54027b60 con 0x7f1c60102a70 2024-09-10T22:39:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 msgr2=0x7f1c4807c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 0x7f1c4807c870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005ab0 tx=0x7f1c50005a40 comp rx=0 tx=0).stop 2024-09-10T22:39:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 msgr2=0x7f1c601981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).stop 2024-09-10T22:39:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 shutdown_connections 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c60102a70 0x7f1c601981a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1c4807a3b0 0x7f1c4807c870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c60103430 0x7f1c60198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 --2- 172.21.15.96:0/3753388649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c60101910 0x7f1c60197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 >> 172.21.15.96:0/3753388649 conn(0x7f1c600fd1e0 msgr2=0x7f1c60105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.514+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 shutdown_connections 2024-09-10T22:39:03.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.515+0000 7f1c6629a640 1 -- 172.21.15.96:0/3753388649 wait complete. 2024-09-10T22:39:03.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3712542748' entity='client.admin' 2024-09-10T22:39:03.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2931145647' entity='client.admin' 2024-09-10T22:39:03.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: Standby manager daemon y started 2024-09-10T22:39:03.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T22:39:03.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T22:39:03.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T22:39:03.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='mgr.? 172.21.15.117:0/449673774' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T22:39:03.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:03 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.044+0000 7f5ee04fd640 1 -- 172.21.15.96:0/1838849851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8104550 msgr2=0x7f5ed810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.044+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/1838849851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8104550 0x7f5ed810aed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009930 tx=0x7f5ed002a420 comp rx=0 tx=0).stop 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- 172.21.15.96:0/1838849851 shutdown_connections 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/1838849851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8104550 0x7f5ed810aed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/1838849851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8103b90 0x7f5ed8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/1838849851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8102a30 0x7f5ed8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- 172.21.15.96:0/1838849851 >> 172.21.15.96:0/1838849851 conn(0x7f5ed80fe320 msgr2=0x7f5ed8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- 172.21.15.96:0/1838849851 shutdown_connections 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- 172.21.15.96:0/1838849851 wait complete. 2024-09-10T22:39:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 Processor -- start 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- start start 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8102a30 0x7f5ed8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8104550 0x7f5ed8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed810e490 con 0x7f5ed8103b90 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed810e310 con 0x7f5ed8104550 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.045+0000 7f5ee04fd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed810e610 con 0x7f5ed8102a30 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38668/0 (socket says 172.21.15.96:38668) 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 -- 172.21.15.96:0/2958241458 learned_addr learned my addr 172.21.15.96:0/2958241458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edea9a640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8104550 0x7f5ed8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ede299640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8102a30 0x7f5ed8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 -- 172.21.15.96:0/2958241458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8102a30 msgr2=0x7f5ed8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8102a30 0x7f5ed8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 -- 172.21.15.96:0/2958241458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8104550 msgr2=0x7f5ed8199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8104550 0x7f5ed8199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 -- 172.21.15.96:0/2958241458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed819a790 con 0x7f5ed8103b90 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5edda98640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b4a0 tx=0x7f5ecc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc014070 con 0x7f5ed8103b90 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed819aa80 con 0x7f5ed8103b90 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5ecc010a80 con 0x7f5ed8103b90 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc017ad0 con 0x7f5ed8103b90 2024-09-10T22:39:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.046+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed81a1160 con 0x7f5ed8103b90 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.047+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed8102e30 con 0x7f5ed8103b90 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.048+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f5ecc017c70 con 0x7f5ed8103b90 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.049+0000 7f5ed77fe640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 0x7f5ec00b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.050+0000 7f5ede299640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 0x7f5ec00b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.050+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ecc0e3760 con 0x7f5ed8103b90 2024-09-10T22:39:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.050+0000 7f5ede299640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 0x7f5ec00b7710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005fd0 tx=0x7f5ec8009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:04.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.052+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ecc010c30 con 0x7f5ed8103b90 2024-09-10T22:39:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.196+0000 7f8952b1d640 1 -- 172.21.15.96:0/2857319958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 msgr2=0x7f894c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.196+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2857319958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c101d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8940009930 tx=0x7f894002a420 comp rx=0 tx=0).stop 2024-09-10T22:39:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- 172.21.15.96:0/2857319958 shutdown_connections 2024-09-10T22:39:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2857319958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 0x7f894c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2857319958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2857319958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c101d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- 172.21.15.96:0/2857319958 >> 172.21.15.96:0/2857319958 conn(0x7f894c0fd1e0 msgr2=0x7f894c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- 172.21.15.96:0/2857319958 shutdown_connections 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- 172.21.15.96:0/2857319958 wait complete. 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 Processor -- start 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- start start 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 0x7f894c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10e4e0 con 0x7f894c101910 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10e360 con 0x7f894c102a70 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f8952b1d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f894c10e660 con 0x7f894c103430 2024-09-10T22:39:04.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f89508b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f89508b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38636/0 (socket says 172.21.15.96:38636) 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.197+0000 7f89508b9640 1 -- 172.21.15.96:0/2029923697 learned_addr learned my addr 172.21.15.96:0/2029923697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f894bfff640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 0x7f894c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89510ba640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 -- 172.21.15.96:0/2029923697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 msgr2=0x7f894c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 -- 172.21.15.96:0/2029923697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 msgr2=0x7f894c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 0x7f894c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 -- 172.21.15.96:0/2029923697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c19a7a0 con 0x7f894c101910 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89510ba640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f89508b9640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f894002a940 tx=0x7f8940005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894003f070 con 0x7f894c101910 2024-09-10T22:39:04.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f8940003c60 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c19aa30 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940035030 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.198+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c1a0fa0 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.199+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c101d10 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.200+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f89400037f0 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.200+0000 7f8949ffb640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 0x7f893c07c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.200+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f89400bca60 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.200+0000 7f894bfff640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 0x7f893c07c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.201+0000 7f894bfff640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 0x7f893c07c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f893800bbe0 tx=0x7f893800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.205+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f89400850c0 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.386+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f8940081330 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.396+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f894c063d50 con 0x7f894c101910 2024-09-10T22:39:04.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.400+0000 7f8949ffb640 1 -- 172.21.15.96:0/2029923697 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f89400b9e80 con 0x7f894c101910 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 msgr2=0x7f893c07c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 0x7f893c07c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f893800bbe0 tx=0x7f893800b390 comp rx=0 tx=0).stop 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 msgr2=0x7f894c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f894002a940 tx=0x7f8940005280 comp rx=0 tx=0).stop 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 shutdown_connections 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f894c102a70 0x7f894c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f893c07a3b0 0x7f893c07c870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f894c103430 0x7f894c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 --2- 172.21.15.96:0/2029923697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f894c101910 0x7f894c198c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 >> 172.21.15.96:0/2029923697 conn(0x7f894c0fd1e0 msgr2=0x7f894c105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:04.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 shutdown_connections 2024-09-10T22:39:04.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:03.402+0000 7f8952b1d640 1 -- 172.21.15.96:0/2029923697 wait complete. 2024-09-10T22:39:04.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.234+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f5ed81a1330 con 0x7f5ed8103b90 2024-09-10T22:39:04.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.236+0000 7f5ed77fe640 1 -- 172.21.15.96:0/2958241458 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f5ecc0ac2d0 con 0x7f5ed8103b90 2024-09-10T22:39:04.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 msgr2=0x7f5ec00b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 0x7f5ec00b7710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005fd0 tx=0x7f5ec8009870 comp rx=0 tx=0).stop 2024-09-10T22:39:04.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 msgr2=0x7f5ed81992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:04.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b4a0 tx=0x7f5ecc00b960 comp rx=0 tx=0).stop 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 shutdown_connections 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ed8104550 0x7f5ed8199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5ec00b5250 0x7f5ec00b7710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ed8103b90 0x7f5ed81992b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 --2- 172.21.15.96:0/2958241458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ed8102a30 0x7f5ed8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 >> 172.21.15.96:0/2958241458 conn(0x7f5ed80fe320 msgr2=0x7f5ed8104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 shutdown_connections 2024-09-10T22:39:04.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:04.239+0000 7f5ee04fd640 1 -- 172.21.15.96:0/2958241458 wait complete. 2024-09-10T22:39:04.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:04 smithi096 ceph-mon[30869]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:04.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:04 smithi096 ceph-mon[30869]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:39:04.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:39:04.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:04 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2029923697' entity='client.admin' 2024-09-10T22:39:04.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:04 smithi096 ceph-mon[30869]: from='client.? ' entity='client.admin' 2024-09-10T22:39:04.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:04 smithi121 ceph-mon[35119]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:04.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:04 smithi121 ceph-mon[35119]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:39:04.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:39:04.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:04 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2029923697' entity='client.admin' 2024-09-10T22:39:04.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:04 smithi121 ceph-mon[35119]: from='client.? ' entity='client.admin' 2024-09-10T22:39:04.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:04 smithi117 ceph-mon[35081]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:04.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:04 smithi117 ceph-mon[35081]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-10T22:39:04.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T22:39:04.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:04 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2029923697' entity='client.admin' 2024-09-10T22:39:04.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:04 smithi117 ceph-mon[35081]: from='client.? ' entity='client.admin' 2024-09-10T22:39:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:05 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2958241458' entity='client.admin' 2024-09-10T22:39:05.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:05 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2958241458' entity='client.admin' 2024-09-10T22:39:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:05 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2958241458' entity='client.admin' 2024-09-10T22:39:06.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:06 smithi096 ceph-mon[30869]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:06.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:06 smithi121 ceph-mon[35119]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:06.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:06 smithi117 ceph-mon[35081]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:06.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 -- 172.21.15.96:0/1578296311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220102a30 msgr2=0x7f8220102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:06.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 --2- 172.21.15.96:0/1578296311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220102a30 0x7f8220102e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f821c009930 tx=0x7f821c02a420 comp rx=0 tx=0).stop 2024-09-10T22:39:06.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 -- 172.21.15.96:0/1578296311 shutdown_connections 2024-09-10T22:39:06.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 --2- 172.21.15.96:0/1578296311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220104550 0x7f822010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:06.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 --2- 172.21.15.96:0/1578296311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220103b90 0x7f8220104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 --2- 172.21.15.96:0/1578296311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220102a30 0x7f8220102e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.947+0000 7f8228125640 1 -- 172.21.15.96:0/1578296311 >> 172.21.15.96:0/1578296311 conn(0x7f82200fe320 msgr2=0x7f8220100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- 172.21.15.96:0/1578296311 shutdown_connections 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- 172.21.15.96:0/1578296311 wait complete. 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 Processor -- start 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- start start 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102a30 0x7f8220076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220104550 0x7f8220075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f822010e460 con 0x7f8220103b90 2024-09-10T22:39:06.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f822010e2e0 con 0x7f8220102a30 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f8228125640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f822010e5e0 con 0x7f8220104550 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f82256c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f82256c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41940/0 (socket says 172.21.15.96:41940) 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.948+0000 7f82256c0640 1 -- 172.21.15.96:0/651009917 learned_addr learned my addr 172.21.15.96:0/651009917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8225ec1640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102a30 0x7f8220076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82266c2640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220104550 0x7f8220075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 -- 172.21.15.96:0/651009917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220104550 msgr2=0x7f8220075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220104550 0x7f8220075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 -- 172.21.15.96:0/651009917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102a30 msgr2=0x7f8220076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102a30 0x7f8220076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 -- 172.21.15.96:0/651009917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220076340 con 0x7f8220103b90 2024-09-10T22:39:06.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f82256c0640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821000d950 tx=0x7f821000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:06.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210017030 con 0x7f8220103b90 2024-09-10T22:39:06.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f8210004020 con 0x7f8220103b90 2024-09-10T22:39:06.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82201a2d50 con 0x7f8220103b90 2024-09-10T22:39:06.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210005580 con 0x7f8220103b90 2024-09-10T22:39:06.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.949+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201a3200 con 0x7f8220103b90 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.950+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8220102e30 con 0x7f8220103b90 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.951+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f8210009a30 con 0x7f8220103b90 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.952+0000 7f8216ffd640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 0x7f820c0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.952+0000 7f8225ec1640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 0x7f820c0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.952+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f82100e4000 con 0x7f8220103b90 2024-09-10T22:39:06.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.953+0000 7f8225ec1640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 0x7f820c0b7710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:06.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:06.955+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f82100acb90 con 0x7f8220103b90 2024-09-10T22:39:07.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.134+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f822002e5c0 con 0x7f8220103b90 2024-09-10T22:39:07.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.138+0000 7f8216ffd640 1 -- 172.21.15.96:0/651009917 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v23)=0 v23) ==== 117+0+0 (secure 0 0 0) 0x7f82100ac490 con 0x7f8220103b90 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 msgr2=0x7f820c0b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 0x7f820c0b7710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 msgr2=0x7f8220075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821000d950 tx=0x7f821000de10 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 shutdown_connections 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f820c0b5250 0x7f820c0b7710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8220104550 0x7f8220075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8220103b90 0x7f8220075540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 --2- 172.21.15.96:0/651009917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8220102a30 0x7f8220076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:07.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 >> 172.21.15.96:0/651009917 conn(0x7f82200fe320 msgr2=0x7f8220107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:07.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 shutdown_connections 2024-09-10T22:39:07.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:07.140+0000 7f8228125640 1 -- 172.21.15.96:0/651009917 wait complete. 2024-09-10T22:39:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:07 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/651009917' entity='client.admin' 2024-09-10T22:39:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:07 smithi096 ceph-mon[30869]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:07.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:07.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:07 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/651009917' entity='client.admin' 2024-09-10T22:39:07.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:07 smithi121 ceph-mon[35119]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:07.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:07.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:07.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:07 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/651009917' entity='client.admin' 2024-09-10T22:39:07.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:07 smithi117 ceph-mon[35081]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:08.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 -- 172.21.15.96:0/2056184992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 msgr2=0x7f9f28102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 --2- 172.21.15.96:0/2056184992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28102ef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009930 tx=0x7f9f1802a420 comp rx=0 tx=0).stop 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 -- 172.21.15.96:0/2056184992 shutdown_connections 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 --2- 172.21.15.96:0/2056184992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28103430 0x7f9f28109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 --2- 172.21.15.96:0/2056184992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28102ef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 --2- 172.21.15.96:0/2056184992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28101910 0x7f9f28101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 -- 172.21.15.96:0/2056184992 >> 172.21.15.96:0/2056184992 conn(0x7f9f280fd1e0 msgr2=0x7f9f280ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 -- 172.21.15.96:0/2056184992 shutdown_connections 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.044+0000 7f9f307b0640 1 -- 172.21.15.96:0/2056184992 wait complete. 2024-09-10T22:39:08.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 Processor -- start 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 -- start start 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101910 0x7f9f28197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28103430 0x7f9f28198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810d2f0 con 0x7f9f28102a70 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810d170 con 0x7f9f28101910 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f307b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810d470 con 0x7f9f28103430 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41964/0 (socket says 172.21.15.96:41964) 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 -- 172.21.15.96:0/1801996242 learned_addr learned my addr 172.21.15.96:0/1801996242 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2ed4d640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28103430 0x7f9f28198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2e54c640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101910 0x7f9f28197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:08.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 -- 172.21.15.96:0/1801996242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28103430 msgr2=0x7f9f28198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28103430 0x7f9f28198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 -- 172.21.15.96:0/1801996242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101910 msgr2=0x7f9f28197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101910 0x7f9f28197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.045+0000 7f9f2dd4b640 1 -- 172.21.15.96:0/1801996242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f28199600 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f2dd4b640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1803f070 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f28199890 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f9f18031d40 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18035030 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.046+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2819fe00 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.047+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f28101d10 con 0x7f9f28102a70 2024-09-10T22:39:08.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.049+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f9f18003b30 con 0x7f9f28102a70 2024-09-10T22:39:08.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.049+0000 7f9f1f7fe640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 0x7f9f140b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:08.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.050+0000 7f9f2e54c640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 0x7f9f140b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:08.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.050+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9f181050d0 con 0x7f9f28102a70 2024-09-10T22:39:08.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.050+0000 7f9f2e54c640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 0x7f9f140b7710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005e10 tx=0x7f9f24005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:08.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.052+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9f180cdc60 con 0x7f9f28102a70 2024-09-10T22:39:08.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.234+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f9f2819ffd0 con 0x7f9f28102a70 2024-09-10T22:39:08.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.240+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f9f180cd560 con 0x7f9f28102a70 2024-09-10T22:39:08.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.241+0000 7f9f1f7fe640 1 -- 172.21.15.96:0/1801996242 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f180047a0 con 0x7f9f28102a70 2024-09-10T22:39:08.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.242+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 msgr2=0x7f9f140b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:08.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.242+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 0x7f9f140b7710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005e10 tx=0x7f9f24005da0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.242+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 msgr2=0x7f9f28198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.242+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f180042f0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.242+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 shutdown_connections 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f140b5250 0x7f9f140b7710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f28103430 0x7f9f28198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28102a70 0x7f9f28198120 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 --2- 172.21.15.96:0/1801996242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101910 0x7f9f28197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 >> 172.21.15.96:0/1801996242 conn(0x7f9f280fd1e0 msgr2=0x7f9f28103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 shutdown_connections 2024-09-10T22:39:08.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:08.243+0000 7f9f307b0640 1 -- 172.21.15.96:0/1801996242 wait complete. 2024-09-10T22:39:08.784 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-10T22:39:09.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 -- 172.21.15.96:0/1374836716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c103430 msgr2=0x7fc71c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 --2- 172.21.15.96:0/1374836716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c103430 0x7fc71c109db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc714009cd0 tx=0x7fc71402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 -- 172.21.15.96:0/1374836716 shutdown_connections 2024-09-10T22:39:09.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 --2- 172.21.15.96:0/1374836716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c102a70 0x7fc71c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 --2- 172.21.15.96:0/1374836716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c103430 0x7fc71c109db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 --2- 172.21.15.96:0/1374836716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c101910 0x7fc71c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 -- 172.21.15.96:0/1374836716 >> 172.21.15.96:0/1374836716 conn(0x7fc71c0fd1e0 msgr2=0x7fc71c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 -- 172.21.15.96:0/1374836716 shutdown_connections 2024-09-10T22:39:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 -- 172.21.15.96:0/1374836716 wait complete. 2024-09-10T22:39:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.006+0000 7fc721fac640 1 Processor -- start 2024-09-10T22:39:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 -- start start 2024-09-10T22:39:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c101910 0x7fc71c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 0x7fc71c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d430 con 0x7fc71c102a70 2024-09-10T22:39:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d2b0 con 0x7fc71c101910 2024-09-10T22:39:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc721fac640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10d5b0 con 0x7fc71c103430 2024-09-10T22:39:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41982/0 (socket says 172.21.15.96:41982) 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 -- 172.21.15.96:0/1008252464 learned_addr learned my addr 172.21.15.96:0/1008252464 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71b7fe640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c101910 0x7fc71c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71bfff640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 0x7fc71c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 -- 172.21.15.96:0/1008252464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 msgr2=0x7fc71c19feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 0x7fc71c19feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 -- 172.21.15.96:0/1008252464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c101910 msgr2=0x7fc71c06b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c101910 0x7fc71c06b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 -- 172.21.15.96:0/1008252464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c1a0770 con 0x7fc71c102a70 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71bfff640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 0x7fc71c19feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.007+0000 7fc71affd640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c920 tx=0x7fc71000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.008+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710007bb0 con 0x7fc71c102a70 2024-09-10T22:39:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.008+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc710007d50 con 0x7fc71c102a70 2024-09-10T22:39:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.008+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c1a0a60 con 0x7fc71c102a70 2024-09-10T22:39:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.008+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710018660 con 0x7fc71c102a70 2024-09-10T22:39:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.008+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a0fa0 con 0x7fc71c102a70 2024-09-10T22:39:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.009+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c108bc0 con 0x7fc71c102a70 2024-09-10T22:39:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.010+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fc71000f450 con 0x7fc71c102a70 2024-09-10T22:39:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.011+0000 7fc718ff9640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 0x7fc7040b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.011+0000 7fc71b7fe640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 0x7fc7040b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.011+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc7100e4290 con 0x7fc71c102a70 2024-09-10T22:39:09.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.011+0000 7fc71b7fe640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 0x7fc7040b7710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00abe0 tx=0x7fc70c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:09.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.014+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc710014030 con 0x7fc71c102a70 2024-09-10T22:39:09.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.195+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fc71c02e5c0 con 0x7fc71c102a70 2024-09-10T22:39:09.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.199+0000 7fc718ff9640 1 -- 172.21.15.96:0/1008252464 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7fc71000f770 con 0x7fc71c102a70 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 msgr2=0x7fc7040b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 0x7fc7040b7710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00abe0 tx=0x7fc70c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 msgr2=0x7fc71c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c920 tx=0x7fc71000cd20 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 shutdown_connections 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7040b5250 0x7fc7040b7710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc71c103430 0x7fc71c19feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71c102a70 0x7fc71c19d8c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 --2- 172.21.15.96:0/1008252464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c101910 0x7fc71c06b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 >> 172.21.15.96:0/1008252464 conn(0x7fc71c0fd1e0 msgr2=0x7fc71c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:09.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 shutdown_connections 2024-09-10T22:39:09.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.201+0000 7fc721fac640 1 -- 172.21.15.96:0/1008252464 wait complete. 2024-09-10T22:39:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:09 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1801996242' entity='client.admin' 2024-09-10T22:39:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:09 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1008252464' entity='client.admin' 2024-09-10T22:39:09.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:09 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1801996242' entity='client.admin' 2024-09-10T22:39:09.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:09 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1008252464' entity='client.admin' 2024-09-10T22:39:09.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:09 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1801996242' entity='client.admin' 2024-09-10T22:39:09.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:09 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1008252464' entity='client.admin' 2024-09-10T22:39:09.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 -- 172.21.15.96:0/3534069259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178102aa0 msgr2=0x7fe178102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 --2- 172.21.15.96:0/3534069259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178102aa0 0x7fe178102f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe168009930 tx=0x7fe16802a420 comp rx=0 tx=0).stop 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 -- 172.21.15.96:0/3534069259 shutdown_connections 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 --2- 172.21.15.96:0/3534069259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178103460 0x7fe178109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 --2- 172.21.15.96:0/3534069259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178102aa0 0x7fe178102f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 --2- 172.21.15.96:0/3534069259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178101940 0x7fe178101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 -- 172.21.15.96:0/3534069259 >> 172.21.15.96:0/3534069259 conn(0x7fe1780fd210 msgr2=0x7fe1780ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 -- 172.21.15.96:0/3534069259 shutdown_connections 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.677+0000 7fe17f73b640 1 -- 172.21.15.96:0/3534069259 wait complete. 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 Processor -- start 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 -- start start 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178102aa0 0x7fe178199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 0x7fe17819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe17810e5d0 con 0x7fe178103460 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe17810e450 con 0x7fe178102aa0 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17f73b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe17810e750 con 0x7fe178101940 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17d4d7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17ccd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178102aa0 0x7fe178199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17dcd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 0x7fe17819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17d4d7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54882/0 (socket says 172.21.15.96:54882) 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.678+0000 7fe17d4d7640 1 -- 172.21.15.96:0/2038377305 learned_addr learned my addr 172.21.15.96:0/2038377305 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 -- 172.21.15.96:0/2038377305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 msgr2=0x7fe178198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 -- 172.21.15.96:0/2038377305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178102aa0 msgr2=0x7fe178199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178102aa0 0x7fe178199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 -- 172.21.15.96:0/2038377305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17819a8e0 con 0x7fe178103460 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17d4d7640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17dcd8640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 0x7fe17819a040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e970 tx=0x7fe17000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000ccc0 con 0x7fe178103460 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe170005020 con 0x7fe178103460 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000ce60 con 0x7fe178103460 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17819abd0 con 0x7fe178103460 2024-09-10T22:39:09.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.679+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781a12b0 con 0x7fe178103460 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.680+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe178108de0 con 0x7fe178103460 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.682+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fe170004a00 con 0x7fe178103460 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.682+0000 7fe16e7fc640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 0x7fe1640b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.682+0000 7fe17d4d7640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 0x7fe1640b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.682+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe1700e3e10 con 0x7fe178103460 2024-09-10T22:39:09.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.683+0000 7fe17d4d7640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 0x7fe1640b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e840 tx=0x7fe174009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:09.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.685+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1700051d0 con 0x7fe178103460 2024-09-10T22:39:09.790 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:09.795 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:09.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.887+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fe17802e5c0 con 0x7fe178103460 2024-09-10T22:39:09.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.890+0000 7fe16e7fc640 1 -- 172.21.15.96:0/2038377305 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v26)=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7fe1700aca30 con 0x7fe178103460 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 msgr2=0x7fe1640b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 0x7fe1640b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e840 tx=0x7fe174009800 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 msgr2=0x7fe17819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 0x7fe17819a040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e970 tx=0x7fe17000ee30 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 shutdown_connections 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178102aa0 0x7fe178199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1640b5250 0x7fe1640b7710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178103460 0x7fe17819a040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 --2- 172.21.15.96:0/2038377305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101940 0x7fe178198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:09.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 >> 172.21.15.96:0/2038377305 conn(0x7fe1780fd210 msgr2=0x7fe178105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:09.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 shutdown_connections 2024-09-10T22:39:09.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:09.893+0000 7fe17f73b640 1 -- 172.21.15.96:0/2038377305 wait complete. 2024-09-10T22:39:10.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- 172.21.15.96:0/3510459273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c103460 msgr2=0x7f520c109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 --2- 172.21.15.96:0/3510459273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c103460 0x7f520c109de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5204009cd0 tx=0x7f520402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- 172.21.15.96:0/3510459273 shutdown_connections 2024-09-10T22:39:10.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 --2- 172.21.15.96:0/3510459273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c103460 0x7f520c109de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 --2- 172.21.15.96:0/3510459273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c102aa0 0x7f520c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 --2- 172.21.15.96:0/3510459273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c101940 0x7f520c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- 172.21.15.96:0/3510459273 >> 172.21.15.96:0/3510459273 conn(0x7f520c0fd210 msgr2=0x7f520c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:10.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- 172.21.15.96:0/3510459273 shutdown_connections 2024-09-10T22:39:10.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- 172.21.15.96:0/3510459273 wait complete. 2024-09-10T22:39:10.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 Processor -- start 2024-09-10T22:39:10.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.055+0000 7f5211d49640 1 -- start start 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 0x7f520c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c103460 0x7f520c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10d480 con 0x7f520c101940 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10d300 con 0x7f520c102aa0 2024-09-10T22:39:10.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10d600 con 0x7f520c103460 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42014/0 (socket says 172.21.15.96:42014) 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 -- 172.21.15.96:0/1585851940 learned_addr learned my addr 172.21.15.96:0/1585851940 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520affd640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 0x7f520c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520bfff640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c103460 0x7f520c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 -- 172.21.15.96:0/1585851940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c103460 msgr2=0x7f520c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c103460 0x7f520c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 -- 172.21.15.96:0/1585851940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 msgr2=0x7f520c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 0x7f520c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 -- 172.21.15.96:0/1585851940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c199760 con 0x7f520c101940 2024-09-10T22:39:10.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520affd640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 0x7f520c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f520b7fe640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cd80 con 0x7f520c101940 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51fc0108c0 con 0x7f520c101940 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0187d0 con 0x7f520c101940 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.056+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c199a50 con 0x7f520c101940 2024-09-10T22:39:10.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.057+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c06b660 con 0x7f520c101940 2024-09-10T22:39:10.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.057+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c06c080 con 0x7f520c101940 2024-09-10T22:39:10.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.059+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f51fc010430 con 0x7f520c101940 2024-09-10T22:39:10.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.059+0000 7f5208ff9640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 0x7f51f40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.060+0000 7f520affd640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 0x7f51f40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.060+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51fc0e4680 con 0x7f520c101940 2024-09-10T22:39:10.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.060+0000 7f520affd640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 0x7f51f40b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5200009fc0 tx=0x7f5200005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:10.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.062+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51fc0acbc0 con 0x7f520c101940 2024-09-10T22:39:10.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.244+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f520c06c2a0 con 0x7f520c101940 2024-09-10T22:39:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.254+0000 7f5208ff9640 1 -- 172.21.15.96:0/1585851940 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f51fc0b15f0 con 0x7f520c101940 2024-09-10T22:39:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.256+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 msgr2=0x7f51f40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.256+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 0x7f51f40b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5200009fc0 tx=0x7f5200005f90 comp rx=0 tx=0).stop 2024-09-10T22:39:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.256+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 msgr2=0x7f520c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.256+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 shutdown_connections 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520c102aa0 0x7f520c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51f40b52d0 0x7f51f40b7790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f520c103460 0x7f520c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 --2- 172.21.15.96:0/1585851940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f520c101940 0x7f520c197c30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 >> 172.21.15.96:0/1585851940 conn(0x7f520c0fd210 msgr2=0x7f520c105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 shutdown_connections 2024-09-10T22:39:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.257+0000 7f5211d49640 1 -- 172.21.15.96:0/1585851940 wait complete. 2024-09-10T22:39:10.487 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:10 smithi096 ceph-mon[30869]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:10.487 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:10 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2038377305' entity='client.admin' 2024-09-10T22:39:10.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:10 smithi121 ceph-mon[35119]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:10.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:10 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2038377305' entity='client.admin' 2024-09-10T22:39:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:10 smithi117 ceph-mon[35081]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:10 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2038377305' entity='client.admin' 2024-09-10T22:39:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- 172.21.15.96:0/410069357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8102a30 msgr2=0x7fb9f8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 --2- 172.21.15.96:0/410069357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8102a30 0x7fb9f8102e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009930 tx=0x7fb9e802a420 comp rx=0 tx=0).stop 2024-09-10T22:39:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- 172.21.15.96:0/410069357 shutdown_connections 2024-09-10T22:39:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 --2- 172.21.15.96:0/410069357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8104550 0x7fb9f810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 --2- 172.21.15.96:0/410069357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8103b90 0x7fb9f8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 --2- 172.21.15.96:0/410069357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8102a30 0x7fb9f8102e30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- 172.21.15.96:0/410069357 >> 172.21.15.96:0/410069357 conn(0x7fb9f80fe320 msgr2=0x7fb9f8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- 172.21.15.96:0/410069357 shutdown_connections 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- 172.21.15.96:0/410069357 wait complete. 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 Processor -- start 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 -- start start 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.954+0000 7fb9fe452640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8102a30 0x7fb9f8076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8103b90 0x7fb9f8075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e570 con 0x7fb9f8104550 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e3f0 con 0x7fb9f8103b90 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f810e6f0 con 0x7fb9f8102a30 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42036/0 (socket says 172.21.15.96:42036) 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 -- 172.21.15.96:0/133663434 learned_addr learned my addr 172.21.15.96:0/133663434 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9effff640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8103b90 0x7fb9f8075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9f7fff640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8102a30 0x7fb9f8076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 -- 172.21.15.96:0/133663434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8102a30 msgr2=0x7fb9f8076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8102a30 0x7fb9f8076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 -- 172.21.15.96:0/133663434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8103b90 msgr2=0x7fb9f8075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8103b90 0x7fb9f8075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 -- 172.21.15.96:0/133663434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f8076340 con 0x7fb9f8104550 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fc9ef640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000cce0 tx=0x7fb9f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000f450 con 0x7fb9f8104550 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f81a2de0 con 0x7fb9f8104550 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb9f000fa80 con 0x7fb9f8104550 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0018ae0 con 0x7fb9f8104550 2024-09-10T22:39:10.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.955+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f81a3240 con 0x7fb9f8104550 2024-09-10T22:39:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.956+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f8109db0 con 0x7fb9f8104550 2024-09-10T22:39:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.958+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fb9f000f5f0 con 0x7fb9f8104550 2024-09-10T22:39:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.958+0000 7fb9f5ffb640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 0x7fb9e40b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.958+0000 7fb9f7fff640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 0x7fb9e40b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.959+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb9f00e4620 con 0x7fb9f8104550 2024-09-10T22:39:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.959+0000 7fb9f7fff640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 0x7fb9e40b7710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009900 tx=0x7fb9e8005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:10.961+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb9f0014030 con 0x7fb9f8104550 2024-09-10T22:39:11.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.138+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fb9f802e5c0 con 0x7fb9f8104550 2024-09-10T22:39:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.142+0000 7fb9f5ffb640 1 -- 172.21.15.96:0/133663434 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v28)=0 v28) ==== 125+0+0 (secure 0 0 0) 0x7fb9f00ad240 con 0x7fb9f8104550 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 msgr2=0x7fb9e40b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 0x7fb9e40b7710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009900 tx=0x7fb9e8005720 comp rx=0 tx=0).stop 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 msgr2=0x7fb9f8075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000cce0 tx=0x7fb9f000a620 comp rx=0 tx=0).stop 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 shutdown_connections 2024-09-10T22:39:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9f8103b90 0x7fb9f8075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9e40b5250 0x7fb9e40b7710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9f8104550 0x7fb9f8075a80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 --2- 172.21.15.96:0/133663434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9f8102a30 0x7fb9f8076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 >> 172.21.15.96:0/133663434 conn(0x7fb9f80fe320 msgr2=0x7fb9f8106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 shutdown_connections 2024-09-10T22:39:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:11.144+0000 7fb9fe452640 1 -- 172.21.15.96:0/133663434 wait complete. 2024-09-10T22:39:11.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:11 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1585851940' entity='client.admin' 2024-09-10T22:39:11.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:11 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/133663434' entity='client.admin' 2024-09-10T22:39:11.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:11 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1585851940' entity='client.admin' 2024-09-10T22:39:11.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:11 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/133663434' entity='client.admin' 2024-09-10T22:39:11.593 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:11 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1585851940' entity='client.admin' 2024-09-10T22:39:11.593 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:11 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/133663434' entity='client.admin' 2024-09-10T22:39:12.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:12 smithi121 ceph-mon[35119]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:12.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:12 smithi117 ceph-mon[35081]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:12.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:12 smithi096 ceph-mon[30869]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:13 smithi096 ceph-mon[30869]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:13.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:13.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:13.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:13.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:13 smithi121 ceph-mon[35119]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:13 smithi117 ceph-mon[35081]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:14.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2665361800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec103450 msgr2=0x7f5bec109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2665361800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec103450 0x7f5bec109dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009cd0 tx=0x7f5be402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2665361800 shutdown_connections 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2665361800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec102a90 0x7f5bec102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2665361800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec103450 0x7f5bec109dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2665361800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec101930 0x7f5bec101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2665361800 >> 172.21.15.96:0/2665361800 conn(0x7f5bec0fd220 msgr2=0x7f5bec0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:14.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2665361800 shutdown_connections 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2665361800 wait complete. 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.725+0000 7f5bf3ca1640 1 Processor -- start 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 -- start start 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 0x7f5bec103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec103450 0x7f5bec1044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec10d470 con 0x7f5bec101930 2024-09-10T22:39:14.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec10d2f0 con 0x7f5bec103450 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf3ca1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec10d5f0 con 0x7f5bec102a90 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42070/0 (socket says 172.21.15.96:42070) 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 -- 172.21.15.96:0/2408836240 learned_addr learned my addr 172.21.15.96:0/2408836240 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf123c640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 0x7f5bec103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf223e640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec103450 0x7f5bec1044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 -- 172.21.15.96:0/2408836240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 msgr2=0x7f5bec103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 0x7f5bec103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 -- 172.21.15.96:0/2408836240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec103450 msgr2=0x7f5bec1044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec103450 0x7f5bec1044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 -- 172.21.15.96:0/2408836240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec104d60 con 0x7f5bec101930 2024-09-10T22:39:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf123c640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 0x7f5bec103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5bf1a3d640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5be800d770 tx=0x7f5be800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.726+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8017070 con 0x7f5bec101930 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.727+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec1a2e50 con 0x7f5bec101930 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.727+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5be80054b0 con 0x7f5bec101930 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.727+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8004c70 con 0x7f5bec101930 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.727+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1a3250 con 0x7f5bec101930 2024-09-10T22:39:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.728+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec1a3480 con 0x7f5bec101930 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.730+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f5be8004e10 con 0x7f5bec101930 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.730+0000 7f5be2ffd640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 0x7f5bd80b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.731+0000 7f5bf123c640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 0x7f5bd80b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.731+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5be80e3d50 con 0x7f5bec101930 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.731+0000 7f5bf123c640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 0x7f5bd80b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0058c0 tx=0x7f5bdc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:14.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.733+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5be80ac8e0 con 0x7f5bec101930 2024-09-10T22:39:14.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 -- 172.21.15.96:0/3291246763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8102a30 msgr2=0x7f6fb8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 --2- 172.21.15.96:0/3291246763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8102a30 0x7f6fb8102e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009930 tx=0x7f6fb402a420 comp rx=0 tx=0).stop 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 -- 172.21.15.96:0/3291246763 shutdown_connections 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 --2- 172.21.15.96:0/3291246763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8103b90 0x7f6fb8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 --2- 172.21.15.96:0/3291246763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fb8104550 0x7f6fb810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 --2- 172.21.15.96:0/3291246763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8102a30 0x7f6fb8102e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 -- 172.21.15.96:0/3291246763 >> 172.21.15.96:0/3291246763 conn(0x7f6fb80fe320 msgr2=0x7f6fb8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 -- 172.21.15.96:0/3291246763 shutdown_connections 2024-09-10T22:39:14.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.544+0000 7f6fbf509640 1 -- 172.21.15.96:0/3291246763 wait complete. 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 Processor -- start 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 -- start start 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8102a30 0x7f6fb8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fb8103b90 0x7f6fb8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb810e620 con 0x7f6fb8104550 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb810e4a0 con 0x7f6fb8102a30 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbf509640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb810e7a0 con 0x7f6fb8103b90 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42044/0 (socket says 172.21.15.96:42044) 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 -- 172.21.15.96:0/2724651059 learned_addr learned my addr 172.21.15.96:0/2724651059 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:14.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbd2a5640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8102a30 0x7f6fb8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 -- 172.21.15.96:0/2724651059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fb8103b90 msgr2=0x7f6fb8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fb8103b90 0x7f6fb8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 -- 172.21.15.96:0/2724651059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8102a30 msgr2=0x7f6fb8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8102a30 0x7f6fb8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.545+0000 7f6fbdaa6640 1 -- 172.21.15.96:0/2724651059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb819a930 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fbdaa6640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b970 tx=0x7f6fb000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0019070 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6fb0014440 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00075d0 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb819ac20 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.546+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb81a1300 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.547+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb81a1590 con 0x7f6fb8104550 2024-09-10T22:39:14.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.548+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f6fb0014930 con 0x7f6fb8104550 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.549+0000 7f6fae7fc640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 0x7f6fa40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.549+0000 7f6fbd2a5640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 0x7f6fa40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.549+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6fb00e3ad0 con 0x7f6fb8104550 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.549+0000 7f6fbd2a5640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 0x7f6fa40b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.552+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6fb00ac660 con 0x7f6fb8104550 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.731+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f6fb81a1760 con 0x7f6fb8104550 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.734+0000 7f6fae7fc640 1 -- 172.21.15.96:0/2724651059 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v29)=0 v29) ==== 127+0+0 (secure 0 0 0) 0x7f6fb00abf60 con 0x7f6fb8104550 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 msgr2=0x7f6fa40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 0x7f6fa40b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb4005a20 comp rx=0 tx=0).stop 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 msgr2=0x7f6fb819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b970 tx=0x7f6fb000be30 comp rx=0 tx=0).stop 2024-09-10T22:39:14.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 shutdown_connections 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6fa40b52d0 0x7f6fa40b7790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fb8104550 0x7f6fb819a090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fb8103b90 0x7f6fb8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 --2- 172.21.15.96:0/2724651059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fb8102a30 0x7f6fb8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 >> 172.21.15.96:0/2724651059 conn(0x7f6fb80fe320 msgr2=0x7f6fb8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 shutdown_connections 2024-09-10T22:39:14.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:13.737+0000 7f6fbf509640 1 -- 172.21.15.96:0/2724651059 wait complete. 2024-09-10T22:39:14.883 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:14 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2724651059' entity='client.admin' 2024-09-10T22:39:14.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.912+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5bec1a39b0 con 0x7f5bec101930 2024-09-10T22:39:14.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.915+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7f5be80ac1e0 con 0x7f5bec101930 2024-09-10T22:39:14.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.916+0000 7f5be2ffd640 1 -- 172.21.15.96:0/2408836240 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be80abcf0 con 0x7f5bec101930 2024-09-10T22:39:14.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 msgr2=0x7f5bd80b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 0x7f5bd80b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0058c0 tx=0x7f5bdc005850 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 msgr2=0x7f5bec1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5be800d770 tx=0x7f5be800dc30 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 shutdown_connections 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bec103450 0x7f5bec1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bd80b5250 0x7f5bd80b7710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bec102a90 0x7f5bec103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 --2- 172.21.15.96:0/2408836240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec101930 0x7f5bec1056f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 >> 172.21.15.96:0/2408836240 conn(0x7f5bec0fd220 msgr2=0x7f5bec0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 shutdown_connections 2024-09-10T22:39:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:14.918+0000 7f5bf3ca1640 1 -- 172.21.15.96:0/2408836240 wait complete. 2024-09-10T22:39:15.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:14 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2724651059' entity='client.admin' 2024-09-10T22:39:15.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:14 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2724651059' entity='client.admin' 2024-09-10T22:39:15.645 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:16.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2408836240' entity='client.admin' 2024-09-10T22:39:16.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:15 smithi096 ceph-mon[30869]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 -- 172.21.15.96:0/2672152963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc3006c0a0 msgr2=0x7efc30105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 --2- 172.21.15.96:0/2672152963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc3006c0a0 0x7efc30105670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc20009900 tx=0x7efc2002a410 comp rx=0 tx=0).stop 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 -- 172.21.15.96:0/2672152963 shutdown_connections 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 --2- 172.21.15.96:0/2672152963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 0x7efc30108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 --2- 172.21.15.96:0/2672152963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc30108740 0x7efc3010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 --2- 172.21.15.96:0/2672152963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc3006c0a0 0x7efc30105670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 -- 172.21.15.96:0/2672152963 >> 172.21.15.96:0/2672152963 conn(0x7efc300fd200 msgr2=0x7efc300ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 -- 172.21.15.96:0/2672152963 shutdown_connections 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 -- 172.21.15.96:0/2672152963 wait complete. 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.041+0000 7efc3552e640 1 Processor -- start 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 -- start start 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 0x7efc30198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 0x7efc301994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010e680 con 0x7efc30108740 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010e500 con 0x7efc30105bb0 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010e800 con 0x7efc3006c0a0 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:16.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42098/0 (socket says 172.21.15.96:42098) 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 -- 172.21.15.96:0/2685147326 learned_addr learned my addr 172.21.15.96:0/2685147326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2e7fc640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 0x7efc301994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2effd640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 0x7efc30198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 -- 172.21.15.96:0/2685147326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 msgr2=0x7efc30198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 0x7efc30198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 -- 172.21.15.96:0/2685147326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 msgr2=0x7efc301994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 0x7efc301994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 -- 172.21.15.96:0/2685147326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3019a990 con 0x7efc30108740 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2effd640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 0x7efc30198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc2f7fe640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc28009fd0 tx=0x7efc2800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28017070 con 0x7efc30108740 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc28003e20 con 0x7efc30108740 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3019ac80 con 0x7efc30108740 2024-09-10T22:39:16.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.042+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28010460 con 0x7efc30108740 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.043+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a1360 con 0x7efc30108740 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.043+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc300fecf0 con 0x7efc30108740 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.046+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7efc28003970 con 0x7efc30108740 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.046+0000 7efc17fff640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 0x7efc180b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.047+0000 7efc2effd640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 0x7efc180b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.047+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efc280e3bd0 con 0x7efc30108740 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.047+0000 7efc2effd640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 0x7efc180b7790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc200096c0 tx=0x7efc20009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:16.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.049+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efc280ac7e0 con 0x7efc30108740 2024-09-10T22:39:16.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.228+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7efc3002e5c0 con 0x7efc30108740 2024-09-10T22:39:16.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.231+0000 7efc17fff640 1 -- 172.21.15.96:0/2685147326 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7efc280ac0e0 con 0x7efc30108740 2024-09-10T22:39:16.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 msgr2=0x7efc180b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 0x7efc180b7790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc200096c0 tx=0x7efc20009650 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 msgr2=0x7efc3019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc28009fd0 tx=0x7efc2800b260 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 shutdown_connections 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc30105bb0 0x7efc301994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efc180b52d0 0x7efc180b7790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc30108740 0x7efc3019a0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 --2- 172.21.15.96:0/2685147326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3006c0a0 0x7efc30198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 >> 172.21.15.96:0/2685147326 conn(0x7efc300fd200 msgr2=0x7efc30103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 shutdown_connections 2024-09-10T22:39:16.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:16.234+0000 7efc3552e640 1 -- 172.21.15.96:0/2685147326 wait complete. 2024-09-10T22:39:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2408836240' entity='client.admin' 2024-09-10T22:39:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:15 smithi121 ceph-mon[35119]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2408836240' entity='client.admin' 2024-09-10T22:39:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:15 smithi117 ceph-mon[35081]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:16.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi096') 2024-09-10T22:39:16.459 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi096') 2024-09-10T22:39:17.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi096') 2024-09-10T22:39:17.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '14afb286-6fc5-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi096') 2024-09-10T22:39:17.035 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T22:39:17.036 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:39:17.036 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:39:17.052 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:39:17.052 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:17.107 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-10 22:38:11.238532519 +0000 2024-09-10T22:39:17.108 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-10 22:33:36.833408348 +0000 2024-09-10T22:39:17.108 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-10 22:33:36.833408348 +0000 2024-09-10T22:39:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-10 22:33:36.833408348 +0000 2024-09-10T22:39:17.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:39:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-10T22:39:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-10T22:39:17.172 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000297892 s, 1.7 MB/s 2024-09-10T22:39:17.173 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:39:17.229 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-10 22:38:11.272531758 +0000 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-10 22:33:37.161401011 +0000 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-10 22:33:37.161401011 +0000 2024-09-10T22:39:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-10 22:33:37.161401011 +0000 2024-09-10T22:39:17.286 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:39:17.346 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:17 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2685147326' entity='client.admin' 2024-09-10T22:39:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-10T22:39:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-10T22:39:17.349 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000216161 s, 2.4 MB/s 2024-09-10T22:39:17.350 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:39:17.406 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-10T22:39:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:39:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:17.462 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T22:39:17.462 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:17.462 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:17.463 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-10 22:38:11.307530974 +0000 2024-09-10T22:39:17.463 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-10 22:33:37.504393339 +0000 2024-09-10T22:39:17.463 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-10 22:33:37.504393339 +0000 2024-09-10T22:39:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-10 22:33:37.504393339 +0000 2024-09-10T22:39:17.463 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:39:17.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:17 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2685147326' entity='client.admin' 2024-09-10T22:39:17.525 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-10T22:39:17.525 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-10T22:39:17.525 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000208533 s, 2.5 MB/s 2024-09-10T22:39:17.526 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:39:17.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:17 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2685147326' entity='client.admin' 2024-09-10T22:39:17.582 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-10 22:38:11.341530212 +0000 2024-09-10T22:39:17.637 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-10 22:33:37.834385958 +0000 2024-09-10T22:39:17.638 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-10 22:33:37.834385958 +0000 2024-09-10T22:39:17.638 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-10 22:33:37.834385958 +0000 2024-09-10T22:39:17.638 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:39:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-10T22:39:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-10T22:39:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000203707 s, 2.5 MB/s 2024-09-10T22:39:17.702 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:39:17.758 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T22:39:17.758 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:39:17.778 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:39:17.778 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 22:38:56.973638400 +0000 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 22:34:03.487595064 +0000 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 22:34:03.487595064 +0000 2024-09-10T22:39:17.837 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 22:34:03.487595064 +0000 2024-09-10T22:39:17.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:39:17.906 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T22:39:17.907 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T22:39:17.907 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000286798 s, 1.8 MB/s 2024-09-10T22:39:17.908 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:39:17.966 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 22:38:56.973638400 +0000 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 22:34:03.844587795 +0000 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 22:34:03.844587795 +0000 2024-09-10T22:39:18.023 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 22:34:03.844587795 +0000 2024-09-10T22:39:18.024 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:39:18.089 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T22:39:18.089 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T22:39:18.089 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000201368 s, 2.5 MB/s 2024-09-10T22:39:18.090 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:39:18.148 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-10T22:39:18.204 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:39:18.204 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.204 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 22:38:56.974638380 +0000 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 22:34:04.173581095 +0000 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 22:34:04.173581095 +0000 2024-09-10T22:39:18.205 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 22:34:04.173581095 +0000 2024-09-10T22:39:18.205 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:39:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T22:39:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T22:39:18.269 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000222188 s, 2.3 MB/s 2024-09-10T22:39:18.270 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:39:18.327 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 22:38:56.974638380 +0000 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 22:34:04.496574518 +0000 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 22:34:04.496574518 +0000 2024-09-10T22:39:18.384 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 22:34:04.496574518 +0000 2024-09-10T22:39:18.385 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:39:18.448 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T22:39:18.448 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T22:39:18.448 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000170115 s, 3.0 MB/s 2024-09-10T22:39:18.449 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:39:18.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:18 smithi096 ceph-mon[30869]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:18.505 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T22:39:18.506 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:39:18.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:18 smithi121 ceph-mon[35119]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:18.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:39:18.526 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-09-10T22:39:18.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:18 smithi117 ceph-mon[35081]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 22:38:54.638636547 +0000 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 22:33:51.995844801 +0000 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 22:33:51.995844801 +0000 2024-09-10T22:39:18.585 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 22:33:51.995844801 +0000 2024-09-10T22:39:18.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:39:18.655 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T22:39:18.655 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T22:39:18.655 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000296056 s, 1.7 MB/s 2024-09-10T22:39:18.657 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:39:18.716 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-09-10T22:39:18.773 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:39:18.773 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.773 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T22:39:18.773 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.774 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.774 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 22:38:54.639636533 +0000 2024-09-10T22:39:18.774 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 22:33:52.332840428 +0000 2024-09-10T22:39:18.774 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 22:33:52.332840428 +0000 2024-09-10T22:39:18.774 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 22:33:52.332840428 +0000 2024-09-10T22:39:18.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:39:18.841 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T22:39:18.842 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T22:39:18.842 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000210517 s, 2.4 MB/s 2024-09-10T22:39:18.843 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:39:18.901 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-09-10T22:39:18.959 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 22:38:54.639636533 +0000 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 22:33:52.658836198 +0000 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 22:33:52.658836198 +0000 2024-09-10T22:39:18.960 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 22:33:52.658836198 +0000 2024-09-10T22:39:18.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:39:19.026 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T22:39:19.026 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T22:39:19.026 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000279939 s, 1.8 MB/s 2024-09-10T22:39:19.028 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:39:19.086 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-09-10T22:39:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 22:38:54.639636533 +0000 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 22:33:52.976832073 +0000 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 22:33:52.976832073 +0000 2024-09-10T22:39:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 22:33:52.976832073 +0000 2024-09-10T22:39:19.145 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:39:19.212 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T22:39:19.212 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T22:39:19.213 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000202789 s, 2.5 MB/s 2024-09-10T22:39:19.214 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:39:19.273 INFO:tasks.cephadm:Deploying osd.0 on smithi096 with /dev/vg_nvme/lv_4... 2024-09-10T22:39:19.273 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:39:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:20.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:20 smithi096 ceph-mon[30869]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:20.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:20 smithi121 ceph-mon[35119]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:20.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:20 smithi117 ceph-mon[35081]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:39:21.966 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-09-10T22:39:22.213 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:39:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:22 smithi096 ceph-mon[30869]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:22.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:22 smithi121 ceph-mon[35119]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:22.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:22 smithi117 ceph-mon[35081]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:22.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 -- 172.21.15.96:0/1269716844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c10a990 msgr2=0x7fed5c10ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 --2- 172.21.15.96:0/1269716844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c10a990 0x7fed5c10ce30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fed54009ca0 tx=0x7fed5402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 -- 172.21.15.96:0/1269716844 shutdown_connections 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 --2- 172.21.15.96:0/1269716844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c107fe0 0x7fed5c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 --2- 172.21.15.96:0/1269716844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c10a990 0x7fed5c10ce30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 --2- 172.21.15.96:0/1269716844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 0x7fed5c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 -- 172.21.15.96:0/1269716844 >> 172.21.15.96:0/1269716844 conn(0x7fed5c0fd210 msgr2=0x7fed5c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 -- 172.21.15.96:0/1269716844 shutdown_connections 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.653+0000 7fed61817640 1 -- 172.21.15.96:0/1269716844 wait complete. 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 Processor -- start 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 -- start start 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 0x7fed5c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c10a990 0x7fed5c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1108e0 con 0x7fed5c107fe0 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c110760 con 0x7fed5c10a990 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed61817640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c110a60 con 0x7fed5c06bf30 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56272/0 (socket says 172.21.15.96:56272) 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5a7fc640 1 -- 172.21.15.96:0/1027942643 learned_addr learned my addr 172.21.15.96:0/1027942643 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5affd640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 0x7fed5c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5b7fe640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c10a990 0x7fed5c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.654+0000 7fed5a7fc640 1 -- 172.21.15.96:0/1027942643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 msgr2=0x7fed5c19b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed5a7fc640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 0x7fed5c19b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed5a7fc640 1 -- 172.21.15.96:0/1027942643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c10a990 msgr2=0x7fed5c19c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed5a7fc640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c10a990 0x7fed5c19c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:22.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed5a7fc640 1 -- 172.21.15.96:0/1027942643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c19cb40 con 0x7fed5c107fe0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed5a7fc640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed5000d770 tx=0x7fed5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50017070 con 0x7fed5c107fe0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed50009c30 con 0x7fed5c107fe0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c19ce30 con 0x7fed5c107fe0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5000f620 con 0x7fed5c107fe0 2024-09-10T22:39:22.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.655+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1a3510 con 0x7fed5c107fe0 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.656+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c0fed00 con 0x7fed5c107fe0 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.657+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fed50010460 con 0x7fed5c107fe0 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.658+0000 7fed60815640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 0x7fed440b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.658+0000 7fed5affd640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 0x7fed440b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.658+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fed500e3cd0 con 0x7fed5c107fe0 2024-09-10T22:39:22.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.658+0000 7fed5affd640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 0x7fed440b7710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:22.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.662+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fed500ac860 con 0x7fed5c107fe0 2024-09-10T22:39:22.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:22.844+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fed5c063590 con 0x7fed440b5250 2024-09-10T22:39:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:23.517 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:23.517 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:23.517 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:23.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:23.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:23.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:24.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:24 smithi096 ceph-mon[30869]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:24.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:24 smithi096 ceph-mon[30869]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:24.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:24 smithi121 ceph-mon[35119]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:24.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:24 smithi121 ceph-mon[35119]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:24.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:24 smithi117 ceph-mon[35081]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:24.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:24 smithi117 ceph-mon[35081]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:25.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:25 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"} : dispatch 2024-09-10T22:39:25.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:25 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"}]': finished 2024-09-10T22:39:25.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:25 smithi121 ceph-mon[35119]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:39:25.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:25 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:25 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"} : dispatch 2024-09-10T22:39:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:25 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"}]': finished 2024-09-10T22:39:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:25 smithi117 ceph-mon[35081]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:39:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:25 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:25.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:25 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"} : dispatch 2024-09-10T22:39:25.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:25 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2790677236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a28ec19e-b4a4-4a13-8674-1c7d77449432"}]': finished 2024-09-10T22:39:25.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:25 smithi096 ceph-mon[30869]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T22:39:25.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:25 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:26.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:26 smithi121 ceph-mon[35119]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:26.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:26 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1888712990' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:26.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:26 smithi117 ceph-mon[35081]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:26.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:26 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1888712990' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:26 smithi096 ceph-mon[30869]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:26 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1888712990' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:28.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:28 smithi121 ceph-mon[35119]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:28.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:28 smithi117 ceph-mon[35081]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:28 smithi096 ceph-mon[30869]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:30.344 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:30 smithi096 ceph-mon[30869]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:30.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:30 smithi121 ceph-mon[35119]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:30.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:30 smithi117 ceph-mon[35081]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:31 smithi121 ceph-mon[35119]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:31 smithi117 ceph-mon[35081]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:31.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:31 smithi096 ceph-mon[30869]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:33.478 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:39:33.478 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:33.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:39:33.516 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T22:39:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:34.386 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:34 smithi096 ceph-mon[30869]: Deploying daemon osd.0 on smithi096 2024-09-10T22:39:34.386 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:34 smithi096 ceph-mon[30869]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:34 smithi121 ceph-mon[35119]: Deploying daemon osd.0 on smithi096 2024-09-10T22:39:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:34 smithi121 ceph-mon[35119]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:34 smithi117 ceph-mon[35081]: Deploying daemon osd.0 on smithi096 2024-09-10T22:39:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:34 smithi117 ceph-mon[35081]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:35.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:35 smithi096 ceph-mon[30869]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:35 smithi121 ceph-mon[35119]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:35 smithi117 ceph-mon[35081]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:37 smithi096 ceph-mon[30869]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:37 smithi121 ceph-mon[35119]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:37 smithi117 ceph-mon[35081]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:39.566 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:39 smithi121 ceph-mon[35119]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:39.577 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:39 smithi117 ceph-mon[35081]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:39 smithi096 ceph-mon[30869]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:40.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:40.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:40.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:40.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:40.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:40.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:40.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:40.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:40.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:41.657 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:41 smithi096 ceph-mon[30869]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:41 smithi121 ceph-mon[35119]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:41 smithi117 ceph-mon[35081]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:43.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:43.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:43.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.415+0000 7fed60815640 1 -- 172.21.15.96:0/1027942643 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fed5c063590 con 0x7fed440b5250 2024-09-10T22:39:43.416 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 0 on host 'smithi096' 2024-09-10T22:39:43.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 msgr2=0x7fed440b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:43.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 0x7fed440b7710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0058c0 tx=0x7fed4c00a250 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 msgr2=0x7fed5c19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed5000d770 tx=0x7fed5000dc30 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 shutdown_connections 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed5c10a990 0x7fed5c19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed440b5250 0x7fed440b7710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed5c107fe0 0x7fed5c19b660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 --2- 172.21.15.96:0/1027942643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed5c06bf30 0x7fed5c19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 >> 172.21.15.96:0/1027942643 conn(0x7fed5c0fd210 msgr2=0x7fed5c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 shutdown_connections 2024-09-10T22:39:43.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:39:43.417+0000 7fed61817640 1 -- 172.21.15.96:0/1027942643 wait complete. 2024-09-10T22:39:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:43 smithi096 ceph-mon[30869]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:43 smithi121 ceph-mon[35119]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:43 smithi117 ceph-mon[35081]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:39:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:44.029 DEBUG:teuthology.orchestra.run.smithi096:osd.0> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T22:39:44.032 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_4... 2024-09-10T22:39:44.032 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:39:44.282 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:39:45.473 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:45 smithi096 ceph-mon[30869]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:39:45.473 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:39:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T22:39:45.179+0000 7f3c20a5a640 -1 osd.0 0 log_to_monitors true 2024-09-10T22:39:45.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:45 smithi121 ceph-mon[35119]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:39:45.523 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:45 smithi117 ceph-mon[35081]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:39:46.375 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:46 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T22:39:46.241+0000 7f3c1c9dd640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T22:39:46.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T22:39:46.246+0000 7f3c18804640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:39:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:39:46.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:46 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:46.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: from='osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T22:39:46.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:46 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:46.674 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:39:46.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-09-10T22:39:46.951 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:47.357 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:47.358 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2465038382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 msgr2=0x7ff818109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2465038382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818109db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff808009930 tx=0x7ff80802a420 comp rx=0 tx=0).stop 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2465038382 shutdown_connections 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2465038382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818109db0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2465038382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818102a70 0x7ff818102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2465038382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818101910 0x7ff818101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:47.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2465038382 >> 172.21.15.117:0/2465038382 conn(0x7ff8180fd1e0 msgr2=0x7ff8180ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:39:47.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2465038382 shutdown_connections 2024-09-10T22:39:47.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.366+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2465038382 wait complete. 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 Processor -- start 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 -- start start 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818101910 0x7ff818197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818102a70 0x7ff8181982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d5f0 con 0x7ff818102a70 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d470 con 0x7ff818103430 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81fe2f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff81810d770 con 0x7ff818101910 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81e3cc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81e3cc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59168/0 (socket says 172.21.15.117:59168) 2024-09-10T22:39:47.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.367+0000 7ff81e3cc640 1 -- 172.21.15.117:0/2739671345 learned_addr learned my addr 172.21.15.117:0/2739671345 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81dbcb640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818101910 0x7ff818197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81d3ca640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818102a70 0x7ff8181982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 -- 172.21.15.117:0/2739671345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818101910 msgr2=0x7ff818197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818101910 0x7ff818197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 -- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818102a70 msgr2=0x7ff8181982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818102a70 0x7ff8181982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 -- 172.21.15.117:0/2739671345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8181997b0 con 0x7ff818103430 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81e3cc640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff808005ab0 tx=0x7ff8080042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80803f030 con 0x7ff818103430 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff818199a40 con 0x7ff818103430 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff808003cb0 con 0x7ff818103430 2024-09-10T22:39:47.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808035070 con 0x7ff818103430 2024-09-10T22:39:47.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.368+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a0180 con 0x7ff818103430 2024-09-10T22:39:47.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.369+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818101d10 con 0x7ff818103430 2024-09-10T22:39:47.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.371+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7ff80803bbf0 con 0x7ff818103430 2024-09-10T22:39:47.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.371+0000 7ff816ffd640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 0x7ff8000b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:39:47.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.372+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7ff808104de0 con 0x7ff818103430 2024-09-10T22:39:47.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.372+0000 7ff81dbcb640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 0x7ff8000b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:47.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.372+0000 7ff81dbcb640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 0x7ff8000b7790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff810007ec0 tx=0x7ff8100075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:39:47.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.375+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8080cd710 con 0x7ff818103430 2024-09-10T22:39:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:47.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:39:47.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:47.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:39:47.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:39:47.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:39:47.560+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff818063590 con 0x7ff8000b52d0 2024-09-10T22:39:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:39:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:39:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi096 2024-09-10T22:39:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] boot 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:48.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:48.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi096 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] boot 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:48.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi096 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: osd.0 [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] boot 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T22:39:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:39:48.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:39:48.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:49.500 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:49 smithi117 ceph-mon[35081]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:49.500 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:49 smithi117 ceph-mon[35081]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:39:49.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:49 smithi121 ceph-mon[35119]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:49.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:49 smithi121 ceph-mon[35119]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:39:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:49 smithi096 ceph-mon[30869]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:39:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:49 smithi096 ceph-mon[30869]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T22:39:50.390 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:50.390 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/4050463186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.390 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.390 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"}]': finished 2024-09-10T22:39:50.391 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:39:50.391 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:50 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:39:50.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:50.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/4050463186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"}]': finished 2024-09-10T22:39:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:39:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:50 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/4050463186' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"} : dispatch 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afe84389-be35-42dc-bd7b-23672bffb969"}]': finished 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T22:39:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:50 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:39:51.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:51 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/1935409045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:51 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/1935409045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:51 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/1935409045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:39:52.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:52 smithi121 ceph-mon[35119]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:52.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:52 smithi117 ceph-mon[35081]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:52.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:52 smithi096 ceph-mon[30869]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:54.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:54 smithi121 ceph-mon[35119]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:54 smithi117 ceph-mon[35081]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:54 smithi096 ceph-mon[30869]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:55 smithi117 ceph-mon[35081]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:55.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:55 smithi096 ceph-mon[30869]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:55 smithi121 ceph-mon[35119]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:57.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:57 smithi117 ceph-mon[35081]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:57.758 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:57 smithi096 ceph-mon[30869]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:57.778 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:57 smithi121 ceph-mon[35119]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:58.730 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:39:58.730 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:58.730 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:58 smithi117 ceph-mon[35081]: Deploying daemon osd.1 on smithi117 2024-09-10T22:39:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:39:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:58 smithi096 ceph-mon[30869]: Deploying daemon osd.1 on smithi117 2024-09-10T22:39:58.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T22:39:58.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:39:58.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:58 smithi121 ceph-mon[35119]: Deploying daemon osd.1 on smithi117 2024-09-10T22:39:59.433 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:39:59 smithi117 ceph-mon[35081]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:39:59 smithi096 ceph-mon[30869]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:39:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:39:59 smithi121 ceph-mon[35119]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:00.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:00 smithi117 ceph-mon[35081]: overall HEALTH_OK 2024-09-10T22:40:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:00 smithi096 ceph-mon[30869]: overall HEALTH_OK 2024-09-10T22:40:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:00 smithi121 ceph-mon[35119]: overall HEALTH_OK 2024-09-10T22:40:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:01 smithi117 ceph-mon[35081]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:01 smithi096 ceph-mon[30869]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:01 smithi121 ceph-mon[35119]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:03.370 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:03 smithi117 ceph-mon[35081]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:03 smithi096 ceph-mon[30869]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:03.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:03 smithi121 ceph-mon[35119]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:04.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:04.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:04.593 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:05.397 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:05 smithi117 ceph-mon[35081]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:05 smithi096 ceph-mon[30869]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:05 smithi121 ceph-mon[35119]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:07 smithi096 ceph-mon[30869]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:07 smithi121 ceph-mon[35119]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:07 smithi117 ceph-mon[35081]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:08.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.079+0000 7ff816ffd640 1 -- 172.21.15.117:0/2739671345 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff818063590 con 0x7ff8000b52d0 2024-09-10T22:40:08.081 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 msgr2=0x7ff8000b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 0x7ff8000b7790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff810007ec0 tx=0x7ff8100075b0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 msgr2=0x7ff818198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff808005ab0 tx=0x7ff8080042f0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 shutdown_connections 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff818103430 0x7ff818198f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8000b52d0 0x7ff8000b7790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff818102a70 0x7ff8181982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 --2- 172.21.15.117:0/2739671345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff818101910 0x7ff818197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 >> 172.21.15.117:0/2739671345 conn(0x7ff8180fd1e0 msgr2=0x7ff818105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 shutdown_connections 2024-09-10T22:40:08.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:40:08.081+0000 7ff81fe2f640 1 -- 172.21.15.117:0/2739671345 wait complete. 2024-09-10T22:40:08.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:08 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:08.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:08 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:08 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.685 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T22:40:08.688 INFO:tasks.cephadm:Deploying osd.2 on smithi121 with /dev/vg_nvme/lv_4... 2024-09-10T22:40:08.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T22:40:08.715 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:08 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:08.715 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:08 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.715 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:08 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:08 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:08.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:08 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:08 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:08.943 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:40:09.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:09 smithi117 ceph-mon[35081]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:09 smithi096 ceph-mon[30869]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:09 smithi121 ceph-mon[35119]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:10.139 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T22:40:09.933+0000 7f6343049640 -1 osd.1 0 log_to_monitors true 2024-09-10T22:40:10.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:10 smithi121 ceph-mon[35119]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:10.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:10 smithi121 ceph-mon[35119]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:10 smithi096 ceph-mon[30869]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:10 smithi096 ceph-mon[30869]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:10 smithi117 ceph-mon[35081]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:10 smithi117 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T22:40:11.269 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:40:11.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_4 2024-09-10T22:40:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:40:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:40:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-mon[35081]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:11 smithi121 ceph-mon[35119]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:11.606 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: from='osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:40:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:11 smithi096 ceph-mon[30869]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- 172.21.15.121:0/414904900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434103430 msgr2=0x7ff434109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- 172.21.15.121:0/414904900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434103430 0x7ff434109db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009d00 tx=0x7ff42c02a910 comp rx=0 tx=0).stop 2024-09-10T22:40:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- 172.21.15.121:0/414904900 shutdown_connections 2024-09-10T22:40:12.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- 172.21.15.121:0/414904900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 0x7ff434102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- 172.21.15.121:0/414904900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434103430 0x7ff434109db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- 172.21.15.121:0/414904900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434101910 0x7ff434101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- 172.21.15.121:0/414904900 >> 172.21.15.121:0/414904900 conn(0x7ff4340fd1e0 msgr2=0x7ff4340ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- 172.21.15.121:0/414904900 shutdown_connections 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- 172.21.15.121:0/414904900 wait complete. 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 Processor -- start 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 -- start start 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.061+0000 7ff43c238640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 0x7ff43419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434103430 0x7ff4341a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d590 con 0x7ff434103430 2024-09-10T22:40:12.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d410 con 0x7ff434102a70 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410d710 con 0x7ff434101910 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40214/0 (socket says 172.21.15.121:40214) 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 -- 172.21.15.121:0/1861377487 learned_addr learned my addr 172.21.15.121:0/1861377487 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff4397d3640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 0x7ff43419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43a7d5640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434103430 0x7ff4341a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 -- 172.21.15.121:0/1861377487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 msgr2=0x7ff43419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 0x7ff43419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 -- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434103430 msgr2=0x7ff4341a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434103430 0x7ff4341a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 -- 172.21.15.121:0/1861377487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4341a0960 con 0x7ff434101910 2024-09-10T22:40:12.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff439fd4640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff43000d950 tx=0x7ff43000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430017030 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430004020 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4341a0c50 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300047a0 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.062+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a1190 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.063+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff434108c90 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.066+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7ff430009a30 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.066+0000 7ff42affd640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 0x7ff4200b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.067+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7ff4300e3b00 con 0x7ff434101910 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.067+0000 7ff4397d3640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 0x7ff4200b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:12.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.067+0000 7ff4397d3640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 0x7ff4200b7790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff424009b50 tx=0x7ff424005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:12.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.070+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff4300ac3b0 con 0x7ff434101910 2024-09-10T22:40:12.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:12.261+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff434063590 con 0x7ff4200b52d0 2024-09-10T22:40:12.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T22:40:11.824+0000 7f633efcc640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T22:40:12.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T22:40:11.828+0000 7f6336deb640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:40:12.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi117 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:12.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.515 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='osd.1 ' entity='osd.1' 2024-09-10T22:40:12.515 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:12.515 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:12.515 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.515 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:40:12.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi117 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='osd.1 ' entity='osd.1' 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:40:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T22:40:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi117 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:12.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='osd.1 ' entity='osd.1' 2024-09-10T22:40:12.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:12.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:12.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:12.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:14.221 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:40:14.221 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:40:14.222 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:14.222 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] boot 2024-09-10T22:40:14.222 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:40:14.222 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:14.222 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:13 smithi121 ceph-mon[35119]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] boot 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:40:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:14.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:13 smithi096 ceph-mon[30869]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:40:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:40:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: osd.1 [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] boot 2024-09-10T22:40:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T22:40:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T22:40:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:13 smithi117 ceph-mon[35081]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/2329276843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"}]': finished 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:40:15.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/2329276843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"}]': finished 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:40:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:15.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T22:40:15.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/2329276843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"} : dispatch 2024-09-10T22:40:15.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d696b9b-398f-4457-a05b-c14dbb9a42f5"}]': finished 2024-09-10T22:40:15.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T22:40:15.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/2294873540' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:15 smithi096 ceph-mon[30869]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/2294873540' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:15 smithi121 ceph-mon[35119]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/2294873540' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:15 smithi117 ceph-mon[35081]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:17 smithi096 ceph-mon[30869]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:17 smithi121 ceph-mon[35119]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:17 smithi117 ceph-mon[35081]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:19 smithi121 ceph-mon[35119]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:19 smithi096 ceph-mon[30869]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:19 smithi117 ceph-mon[35081]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:21.525 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:21 smithi121 ceph-mon[35119]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:21 smithi096 ceph-mon[30869]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:21 smithi117 ceph-mon[35081]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:40:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:22.888 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:40:22.888 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T22:40:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:23.862 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:23 smithi121 ceph-mon[35119]: Deploying daemon osd.2 on smithi121 2024-09-10T22:40:23.862 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:23 smithi121 ceph-mon[35119]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:24.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:23 smithi096 ceph-mon[30869]: Deploying daemon osd.2 on smithi121 2024-09-10T22:40:24.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:23 smithi096 ceph-mon[30869]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:23 smithi117 ceph-mon[35081]: Deploying daemon osd.2 on smithi121 2024-09-10T22:40:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:23 smithi117 ceph-mon[35081]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:25.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:25 smithi121 ceph-mon[35119]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:25 smithi096 ceph-mon[30869]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:25 smithi117 ceph-mon[35081]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:27.642 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:27 smithi121 ceph-mon[35119]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:27 smithi096 ceph-mon[30869]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:27 smithi117 ceph-mon[35081]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:29.230 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:29.230 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:29.230 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:30.159 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:29 smithi121 ceph-mon[35119]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:29 smithi096 ceph-mon[30869]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:29 smithi117 ceph-mon[35081]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:31.649 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:31 smithi121 ceph-mon[35119]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:31 smithi096 ceph-mon[30869]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:31 smithi117 ceph-mon[35081]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:32.344 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.343+0000 7ff42affd640 1 -- 172.21.15.121:0/1861377487 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff434063590 con 0x7ff4200b52d0 2024-09-10T22:40:32.345 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 2 on host 'smithi121' 2024-09-10T22:40:32.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.345+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 msgr2=0x7ff4200b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:32.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.345+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 0x7ff4200b7790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff424009b50 tx=0x7ff424005f90 comp rx=0 tx=0).stop 2024-09-10T22:40:32.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.345+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 msgr2=0x7ff43406b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:32.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.345+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff43000d950 tx=0x7ff43000de10 comp rx=0 tx=0).stop 2024-09-10T22:40:32.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 shutdown_connections 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff434102a70 0x7ff43419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4200b52d0 0x7ff4200b7790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434103430 0x7ff4341a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 --2- 172.21.15.121:0/1861377487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff434101910 0x7ff43406b3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 >> 172.21.15.121:0/1861377487 conn(0x7ff4340fd1e0 msgr2=0x7ff434105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 shutdown_connections 2024-09-10T22:40:32.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:40:32.346+0000 7ff43c238640 1 -- 172.21.15.121:0/1861377487 wait complete. 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.690 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:32.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:32.985 DEBUG:teuthology.orchestra.run.smithi121:osd.2> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T22:40:32.987 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_3... 2024-09-10T22:40:32.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:40:33.242 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:40:33.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:33 smithi096 ceph-mon[30869]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:33 smithi121 ceph-mon[35119]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:33 smithi117 ceph-mon[35081]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:34 smithi096 ceph-mon[30869]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:34 smithi096 ceph-mon[30869]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:34 smithi121 ceph-mon[35119]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:34 smithi121 ceph-mon[35119]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.264 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:34 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T22:40:34.878+0000 7f01022f2640 -1 osd.2 0 log_to_monitors true 2024-09-10T22:40:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:34 smithi117 ceph-mon[35081]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:34 smithi117 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T22:40:35.783 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:40:35.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-09-10T22:40:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:40:36.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:36.194 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.263 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T22:40:35.948+0000 7f00fe275640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T22:40:36.264 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T22:40:35.953+0000 7f00fa09c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:36.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:36.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:36.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:40:36.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:36.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:36.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- 172.21.15.96:0/1643593143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 msgr2=0x7fb9b8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:36.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1643593143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8102e30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009cd0 tx=0x7fb9a002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- 172.21.15.96:0/1643593143 shutdown_connections 2024-09-10T22:40:36.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1643593143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 0x7fb9b810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1643593143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 0x7fb9b8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1643593143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8102e30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- 172.21.15.96:0/1643593143 >> 172.21.15.96:0/1643593143 conn(0x7fb9b80fe320 msgr2=0x7fb9b8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- 172.21.15.96:0/1643593143 shutdown_connections 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- 172.21.15.96:0/1643593143 wait complete. 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 Processor -- start 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.524+0000 7fb9bf014640 1 -- start start 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 0x7fb9b8075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 0x7fb9b8075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810e790 con 0x7fb9b8102a30 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810e610 con 0x7fb9b8104550 2024-09-10T22:40:36.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810e910 con 0x7fb9b8103b90 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42410/0 (socket says 172.21.15.96:42410) 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 -- 172.21.15.96:0/1756661165 learned_addr learned my addr 172.21.15.96:0/1756661165 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bd5b1640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 0x7fb9b8075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9b7fff640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 0x7fb9b8075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 -- 172.21.15.96:0/1756661165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 msgr2=0x7fb9b8075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 0x7fb9b8075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 -- 172.21.15.96:0/1756661165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 msgr2=0x7fb9b8075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 0x7fb9b8075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 -- 172.21.15.96:0/1756661165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b8075fc0 con 0x7fb9b8102a30 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bcdb0640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009430 tx=0x7fb9a0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:36.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0004510 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b80761f0 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a00046b0 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.525+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0045670 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.526+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b81a7a80 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.527+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b8102eb0 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.528+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fb9a004d070 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.529+0000 7fb9b5ffb640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 0x7fb9a40b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.529+0000 7fb9b7fff640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 0x7fb9a40b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.529+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fb9a002e030 con 0x7fb9b8102a30 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.530+0000 7fb9b7fff640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 0x7fb9a40b7710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009890 tx=0x7fb9a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:36.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.532+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb9a004a060 con 0x7fb9b8102a30 2024-09-10T22:40:36.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:36.717+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb9b8105680 con 0x7fb9a40b5250 2024-09-10T22:40:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi121 2024-09-10T22:40:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-10T22:40:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:40:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:40:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:37.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:37.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:37.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi121 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:37.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi121 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:40:37.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:40:37.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:37.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] boot 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:38.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:38.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] boot 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:38.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: osd.2 [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] boot 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T22:40:38.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:39.233 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:38 smithi096 ceph-mon[30869]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51555]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51555]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51555]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.234 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51555]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.234 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.234 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51543]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51543]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51546]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51546]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51546]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.235 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51546]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.259 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:39.260 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:38 smithi121 ceph-mon[35119]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:40:39.260 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44167]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44167]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44171]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44171]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44171]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.260 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44171]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:39.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:38 smithi117 ceph-mon[35081]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T22:40:39.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:40:39.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43390]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43390]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.277 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.277 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.277 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.277 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43393]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51558]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 sudo[51558]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44174]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44174]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44174]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 sudo[44177]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43397]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43397]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43397]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43400]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43400]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43400]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T22:40:39.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 sudo[43400]: pam_unix(sudo:session): session closed for user root 2024-09-10T22:40:40.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:40.219+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb9a0030070 con 0x7fb9b8102a30 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"} : dispatch 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"}]': finished 2024-09-10T22:40:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/952411155' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"} : dispatch 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"}]': finished 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/952411155' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:40.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:40:40.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T22:40:40.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"} : dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1370670795' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e749818-335f-4852-b188-2898bf14e7ab"}]': finished 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T22:40:40.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/952411155' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:40:41.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:41 smithi096 ceph-mon[30869]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:40:41.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:41 smithi121 ceph-mon[35119]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:40:41.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:41 smithi117 ceph-mon[35081]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T22:40:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:42 smithi096 ceph-mon[30869]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:42.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:42 smithi121 ceph-mon[35119]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:42.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:42 smithi117 ceph-mon[35081]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:43.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:43 smithi096 ceph-mon[30869]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:43 smithi121 ceph-mon[35119]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:43 smithi117 ceph-mon[35081]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:45.721 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:45 smithi096 ceph-mon[30869]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:45 smithi121 ceph-mon[35119]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:45 smithi117 ceph-mon[35081]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:40:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:40:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T22:40:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:48.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:48 smithi096 ceph-mon[30869]: Deploying daemon osd.3 on smithi096 2024-09-10T22:40:48.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:48 smithi096 ceph-mon[30869]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:48 smithi121 ceph-mon[35119]: Deploying daemon osd.3 on smithi096 2024-09-10T22:40:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:48 smithi121 ceph-mon[35119]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:48 smithi117 ceph-mon[35081]: Deploying daemon osd.3 on smithi096 2024-09-10T22:40:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:48 smithi117 ceph-mon[35081]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:50.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:50 smithi096 ceph-mon[30869]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:50.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:50 smithi121 ceph-mon[35119]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:50.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:50 smithi117 ceph-mon[35081]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:51 smithi096 ceph-mon[30869]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:51 smithi121 ceph-mon[35119]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:51 smithi117 ceph-mon[35081]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:53 smithi096 ceph-mon[30869]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:53 smithi121 ceph-mon[35119]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:53 smithi117 ceph-mon[35081]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:55.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:55 smithi096 ceph-mon[30869]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:55 smithi121 ceph-mon[35119]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:55 smithi117 ceph-mon[35081]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:57.427 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.427 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.427 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:57.428 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:57.428 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.477 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-09-10T22:40:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.474+0000 7fb9b5ffb640 1 -- 172.21.15.96:0/1756661165 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9b8105680 con 0x7fb9a40b5250 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 msgr2=0x7fb9a40b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 0x7fb9a40b7710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009890 tx=0x7fb9a8008040 comp rx=0 tx=0).stop 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 msgr2=0x7fb9b8076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009430 tx=0x7fb9a0003f90 comp rx=0 tx=0).stop 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 shutdown_connections 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b8104550 0x7fb9b8075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9a40b5250 0x7fb9a40b7710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9b8103b90 0x7fb9b8075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 --2- 172.21.15.96:0/1756661165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9b8102a30 0x7fb9b8076cd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 >> 172.21.15.96:0/1756661165 conn(0x7fb9b80fe320 msgr2=0x7fb9b8104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 shutdown_connections 2024-09-10T22:40:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:40:57.477+0000 7fb9bf014640 1 -- 172.21.15.96:0/1756661165 wait complete. 2024-09-10T22:40:57.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:57.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:57.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:40:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:40:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.216 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T22:40:58.218 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-09-10T22:40:58.218 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:40:58.473 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:58 smithi096 ceph-mon[30869]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:58.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:58.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.477 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:40:58.516 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:58 smithi121 ceph-mon[35119]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:58.516 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:58.516 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.517 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:58 smithi117 ceph-mon[35081]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:58.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:40:58.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:58.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:40:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:59 smithi096 ceph-mon[30869]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:40:59 smithi096 ceph-mon[30869]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:40:59.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 22:40:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T22:40:59.324+0000 7f99dd835640 -1 osd.3 0 log_to_monitors true 2024-09-10T22:40:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:59 smithi117 ceph-mon[35081]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:40:59 smithi117 ceph-mon[35081]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:40:59.768 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:59 smithi121 ceph-mon[35119]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:40:59.768 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:40:59 smithi121 ceph-mon[35119]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T22:41:01.124 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:41:01.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-09-10T22:41:01.405 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:41:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:41:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:41:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:41:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi096 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:01 smithi117 ceph-mon[35081]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:41:01.715 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi096 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-mon[30869]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:41:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi096 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:01.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-10T22:41:01.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:01.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:01.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:01.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:01 smithi121 ceph-mon[35119]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T22:41:01.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 -- 172.21.15.117:0/627209445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010104550 msgr2=0x7f801010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 --2- 172.21.15.117:0/627209445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010104550 0x7f801010aed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8008009cd0 tx=0x7f800802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 -- 172.21.15.117:0/627209445 shutdown_connections 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 --2- 172.21.15.117:0/627209445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010104550 0x7f801010aed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 --2- 172.21.15.117:0/627209445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 0x7f8010104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 --2- 172.21.15.117:0/627209445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010102a30 0x7f8010102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.913+0000 7f8017c6b640 1 -- 172.21.15.117:0/627209445 >> 172.21.15.117:0/627209445 conn(0x7f80100fe320 msgr2=0x7f8010100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- 172.21.15.117:0/627209445 shutdown_connections 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- 172.21.15.117:0/627209445 wait complete. 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 Processor -- start 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- start start 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 0x7f8010075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010104550 0x7f8010075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f801010e690 con 0x7f8010104550 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f801010e510 con 0x7f8010102a30 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8017c6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f801010e810 con 0x7f8010103b90 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8015a07640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8015a07640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59624/0 (socket says 172.21.15.117:59624) 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8015a07640 1 -- 172.21.15.117:0/2613358220 learned_addr learned my addr 172.21.15.117:0/2613358220 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8015206640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 0x7f8010075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.914+0000 7f8016208640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010104550 0x7f8010075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 -- 172.21.15.117:0/2613358220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 msgr2=0x7f8010075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 0x7f8010075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 -- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010104550 msgr2=0x7f8010075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010104550 0x7f8010075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 -- 172.21.15.117:0/2613358220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8010076340 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8015a07640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c890 tx=0x7f800c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c004220 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800c0043c0 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101a2f00 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c004a00 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.915+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101a3350 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.916+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010109d50 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.917+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f800c0079b0 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.918+0000 7f8006ffd640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 0x7f7ffc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.918+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f800c0e48e0 con 0x7f8010102a30 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.918+0000 7f8015206640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 0x7f7ffc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.919+0000 7f8015206640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 0x7f7ffc0b77a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f800000bbe0 tx=0x7f800000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:01.921+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f800c014030 con 0x7f8010102a30 2024-09-10T22:41:02.000 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T22:41:01.713+0000 7f99d97b8640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T22:41:02.000 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 22:41:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T22:41:01.718+0000 7f99d15d7640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:41:02.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:02.104+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8010063590 con 0x7f7ffc0b52e0 2024-09-10T22:41:02.509 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] boot 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:41:02.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:41:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:41:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] boot 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:41:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556]' entity='osd.3' 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: osd.3 [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] boot 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T22:41:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T22:41:03.514 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:03 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:41:03.514 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:03 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:41:03.514 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:03 smithi117 ceph-mon[35081]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:41:03.514 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:03 smithi117 ceph-mon[35081]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:03 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:41:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:03 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:41:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:03 smithi096 ceph-mon[30869]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:41:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:03 smithi096 ceph-mon[30869]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:03 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:41:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:03 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:41:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:03 smithi121 ceph-mon[35119]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T22:41:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:03 smithi121 ceph-mon[35119]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:05.139 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:41:05.139 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/3375878629' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.139 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.139 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"}]': finished 2024-09-10T22:41:05.140 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:41:05.140 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:41:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/3375878629' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"}]': finished 2024-09-10T22:41:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:41:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:05.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T22:41:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/3375878629' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"} : dispatch 2024-09-10T22:41:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43f285c1-14f0-4b31-a928-1705584f5f0b"}]': finished 2024-09-10T22:41:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T22:41:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:06 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/3588140886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:06 smithi117 ceph-mon[35081]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:06.380 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:06 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/3588140886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:06.380 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:06 smithi121 ceph-mon[35119]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:06.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:06 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/3588140886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:06.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:06 smithi096 ceph-mon[30869]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:07 smithi096 ceph-mon[30869]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:07 smithi121 ceph-mon[35119]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:07 smithi117 ceph-mon[35081]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 110 MiB used, 358 GiB / 358 GiB avail 2024-09-10T22:41:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:09 smithi096 ceph-mon[30869]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:41:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:09 smithi121 ceph-mon[35119]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:41:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:09 smithi117 ceph-mon[35081]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-10T22:41:11.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:11 smithi117 ceph-mon[35081]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:41:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:11 smithi096 ceph-mon[30869]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:41:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:11 smithi121 ceph-mon[35119]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:41:12.973 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:41:12.973 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:41:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T22:41:13.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:13.975 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:13 smithi117 ceph-mon[35081]: Deploying daemon osd.4 on smithi117 2024-09-10T22:41:13.975 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:13 smithi117 ceph-mon[35081]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:41:14.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:13 smithi096 ceph-mon[30869]: Deploying daemon osd.4 on smithi117 2024-09-10T22:41:14.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:13 smithi096 ceph-mon[30869]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:41:14.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:13 smithi121 ceph-mon[35119]: Deploying daemon osd.4 on smithi117 2024-09-10T22:41:14.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:13 smithi121 ceph-mon[35119]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:41:15.743 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:15 smithi117 ceph-mon[35081]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T22:41:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:15 smithi096 ceph-mon[30869]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T22:41:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:15 smithi121 ceph-mon[35119]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T22:41:17.630 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:17 smithi117 ceph-mon[35081]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:17 smithi096 ceph-mon[30869]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:17 smithi121 ceph-mon[35119]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:19.131 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:19.131 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:19.131 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:19 smithi096 ceph-mon[30869]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:20.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:19 smithi121 ceph-mon[35119]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:20.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:19 smithi117 ceph-mon[35081]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:41:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:21 smithi096 ceph-mon[30869]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:21 smithi121 ceph-mon[35119]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:21 smithi117 ceph-mon[35081]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:22.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:22.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:22.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:22.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:22.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:23.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.003 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-09-10T22:41:23.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.001+0000 7f8006ffd640 1 -- 172.21.15.117:0/2613358220 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8010063590 con 0x7f7ffc0b52e0 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 msgr2=0x7f7ffc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 0x7f7ffc0b77a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f800000bbe0 tx=0x7f800000b390 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 msgr2=0x7f8010076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c890 tx=0x7f800c00cd50 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 shutdown_connections 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7ffc0b52e0 0x7f7ffc0b77a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8010104550 0x7f8010075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8010103b90 0x7f8010075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 --2- 172.21.15.117:0/2613358220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8010102a30 0x7f8010076cd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 >> 172.21.15.117:0/2613358220 conn(0x7f80100fe320 msgr2=0x7f8010106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:23.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 shutdown_connections 2024-09-10T22:41:23.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:41:23.004+0000 7f8017c6b640 1 -- 172.21.15.117:0/2613358220 wait complete. 2024-09-10T22:41:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.700 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T22:41:23.702 INFO:tasks.cephadm:Deploying osd.5 on smithi121 with /dev/vg_nvme/lv_3... 2024-09-10T22:41:23.702 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T22:41:23.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:23.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:23.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:23 smithi117 ceph-mon[35081]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:23.956 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:41:24.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:24.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:24.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:24.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:23 smithi096 ceph-mon[30869]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:23 smithi121 ceph-mon[35119]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:25.414 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:25 smithi121 ceph-mon[35119]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:25.414 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:25 smithi121 ceph-mon[35119]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:25 smithi117 ceph-mon[35081]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:25 smithi117 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:25.527 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 22:41:25 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T22:41:25.235+0000 7ff3864c4640 -1 osd.4 0 log_to_monitors true 2024-09-10T22:41:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:25 smithi096 ceph-mon[30869]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:25 smithi096 ceph-mon[30869]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T22:41:26.436 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T22:41:26.299+0000 7ff382447640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T22:41:26.436 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T22:41:26.312+0000 7ff37e26e640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:41:26.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:26.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:41:26.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:41:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:26 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.713 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:26.713 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:41:26.713 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:41:26.714 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:26 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:41:26.739 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_3 2024-09-10T22:41:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T22:41:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T22:41:26.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:41:26.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:41:26.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T22:41:26.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:26 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:26.991 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:41:27.399 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi117 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.400 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:27 smithi121 ceph-mon[35119]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 -- 172.21.15.121:0/125675909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c105c00 msgr2=0x7f763c108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 --2- 172.21.15.121:0/125675909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c105c00 0x7f763c108010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7630009900 tx=0x7f763002a410 comp rx=0 tx=0).stop 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 -- 172.21.15.121:0/125675909 shutdown_connections 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 --2- 172.21.15.121:0/125675909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c108750 0x7f763c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 --2- 172.21.15.121:0/125675909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c105c00 0x7f763c108010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 --2- 172.21.15.121:0/125675909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 0x7f763c1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 -- 172.21.15.121:0/125675909 >> 172.21.15.121:0/125675909 conn(0x7f763c0fd210 msgr2=0x7f763c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:27.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 -- 172.21.15.121:0/125675909 shutdown_connections 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.480+0000 7f76412df640 1 -- 172.21.15.121:0/125675909 wait complete. 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 Processor -- start 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 -- start start 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 0x7f763c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c105c00 0x7f763c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e720 con 0x7f763c105c00 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e5a0 con 0x7f763c06c080 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f76412df640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f763c10e8a0 con 0x7f763c108750 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.481+0000 7f763b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47712/0 (socket says 172.21.15.121:47712) 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 -- 172.21.15.121:0/335105062 learned_addr learned my addr 172.21.15.121:0/335105062 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763a7fc640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c105c00 0x7f763c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763affd640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 0x7f763c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 -- 172.21.15.121:0/335105062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 msgr2=0x7f763c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 0x7f763c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 -- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c105c00 msgr2=0x7f763c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c105c00 0x7f763c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 -- 172.21.15.121:0/335105062 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c19a9e0 con 0x7f763c108750 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f763b7fe640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7634009fd0 tx=0x7f763400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634011830 con 0x7f763c108750 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c19acd0 con 0x7f763c108750 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634016070 con 0x7f763c108750 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634010de0 con 0x7f763c108750 2024-09-10T22:41:27.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.482+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c1a13b0 con 0x7f763c108750 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.483+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763c106000 con 0x7f763c108750 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.484+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f76340119d0 con 0x7f763c108750 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.485+0000 7f7623fff640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 0x7f76240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.485+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4101+0+0 (secure 0 0 0) 0x7f76340e41d0 con 0x7f763c108750 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.485+0000 7f763affd640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 0x7f76240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.485+0000 7f763affd640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 0x7f76240b77a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f762c005fd0 tx=0x7f762c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:27.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.489+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76340ac060 con 0x7f763c108750 2024-09-10T22:41:27.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:27.677+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f763c063590 con 0x7f76240b52e0 2024-09-10T22:41:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi117 2024-09-10T22:41:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:27 smithi096 ceph-mon[30869]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi117 2024-09-10T22:41:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:27.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:27 smithi117 ceph-mon[35081]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T22:41:28.427 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:41:28.428 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:41:28.428 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.428 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] boot 2024-09-10T22:41:28.428 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:41:28.429 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.429 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:28.429 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:28.429 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:28.429 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:41:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] boot 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:28.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:41:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:41:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: osd.4 [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] boot 2024-09-10T22:41:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T22:41:28.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T22:41:28.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:28.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:28.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:28.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:29.601 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:29 smithi121 ceph-mon[35119]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:41:29.601 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:29 smithi121 ceph-mon[35119]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:29 smithi096 ceph-mon[30869]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:41:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:29 smithi096 ceph-mon[30869]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:29 smithi117 ceph-mon[35081]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T22:41:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:29 smithi117 ceph-mon[35081]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:30.643 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:30 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/1755591949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:30.643 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:30 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:30.643 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:30 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"}]': finished 2024-09-10T22:41:30.643 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:30 smithi121 ceph-mon[35119]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:41:30.644 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:30 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/1755591949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:30 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:30 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"}]': finished 2024-09-10T22:41:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:30 smithi117 ceph-mon[35081]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:41:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:31.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:30 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/1755591949' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:31.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:30 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"} : dispatch 2024-09-10T22:41:31.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:30 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5520a789-19f0-4775-a38f-0d7707878cdf"}]': finished 2024-09-10T22:41:31.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:30 smithi096 ceph-mon[30869]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T22:41:31.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:31 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/3124267099' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:31 smithi121 ceph-mon[35119]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:31 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/3124267099' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:31 smithi117 ceph-mon[35081]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:32.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:31 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/3124267099' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:32.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:31 smithi096 ceph-mon[30869]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:33 smithi096 ceph-mon[30869]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:33 smithi121 ceph-mon[35119]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:33 smithi117 ceph-mon[35081]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:35.585 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:35 smithi121 ceph-mon[35119]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:35 smithi096 ceph-mon[30869]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:35 smithi117 ceph-mon[35081]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:37.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:37 smithi121 ceph-mon[35119]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:37.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:37 smithi096 ceph-mon[30869]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:37 smithi117 ceph-mon[35081]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:41:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:41:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:38.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T22:41:38.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:39.684 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:39 smithi121 ceph-mon[35119]: Deploying daemon osd.5 on smithi121 2024-09-10T22:41:39.684 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:39 smithi121 ceph-mon[35119]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:39 smithi096 ceph-mon[30869]: Deploying daemon osd.5 on smithi121 2024-09-10T22:41:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:39 smithi096 ceph-mon[30869]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:39 smithi117 ceph-mon[35081]: Deploying daemon osd.5 on smithi121 2024-09-10T22:41:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:39 smithi117 ceph-mon[35081]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:41.538 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:41 smithi121 ceph-mon[35119]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:41 smithi096 ceph-mon[30869]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:41 smithi117 ceph-mon[35081]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:43.391 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:43 smithi121 ceph-mon[35119]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:43 smithi096 ceph-mon[30869]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:43 smithi117 ceph-mon[35081]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:45.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:45 smithi121 ceph-mon[35119]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:45 smithi096 ceph-mon[30869]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:46.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:45 smithi117 ceph-mon[35081]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:47.483 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:47 smithi121 ceph-mon[35119]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:47 smithi096 ceph-mon[30869]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:47 smithi117 ceph-mon[35081]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:48.930 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:48.930 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:48.930 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:48.930 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:48.930 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:48.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.968+0000 7f7623fff640 1 -- 172.21.15.121:0/335105062 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f763c063590 con 0x7f76240b52e0 2024-09-10T22:41:48.970 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 5 on host 'smithi121' 2024-09-10T22:41:48.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 msgr2=0x7f76240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 0x7f76240b77a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f762c005fd0 tx=0x7f762c009870 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 msgr2=0x7f763c19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7634009fd0 tx=0x7f763400b8e0 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 shutdown_connections 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76240b52e0 0x7f76240b77a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f763c108750 0x7f763c19a140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f763c105c00 0x7f763c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 --2- 172.21.15.121:0/335105062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f763c06c080 0x7f763c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 >> 172.21.15.121:0/335105062 conn(0x7f763c0fd210 msgr2=0x7f763c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 shutdown_connections 2024-09-10T22:41:48.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:41:48.971+0000 7f76412df640 1 -- 172.21.15.121:0/335105062 wait complete. 2024-09-10T22:41:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.601 DEBUG:teuthology.orchestra.run.smithi121:osd.5> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T22:41:49.604 INFO:tasks.cephadm:Deploying osd.6 on smithi096 with /dev/vg_nvme/lv_2... 2024-09-10T22:41:49.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:41:49.826 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:49.826 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.826 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:49 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.826 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:49 smithi121 ceph-mon[35119]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:49.856 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:41:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:49 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:49 smithi096 ceph-mon[30869]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:41:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:49 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:49 smithi117 ceph-mon[35081]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:50.513 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 22:41:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T22:41:50.212+0000 7f89c07ce640 -1 osd.5 0 log_to_monitors true 2024-09-10T22:41:51.097 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:50 smithi121 ceph-mon[35119]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:51.097 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:50 smithi121 ceph-mon[35119]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:51.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:50 smithi096 ceph-mon[30869]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:51.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:50 smithi096 ceph-mon[30869]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:51.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:50 smithi117 ceph-mon[35081]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:51.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:50 smithi117 ceph-mon[35081]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T22:41:52.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:41:52.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:41:52.217 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:52.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:51 smithi096 ceph-mon[30869]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:51 smithi121 ceph-mon[35119]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T22:41:52.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T22:41:52.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:52.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: from='osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:41:52.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:51 smithi117 ceph-mon[35081]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-10T22:41:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:41:52.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-09-10T22:41:52.581 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T22:41:52.415+0000 7f89bc751640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T22:41:52.581 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T22:41:52.421+0000 7f89b4570640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:41:52.784 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='osd.5 ' entity='osd.5' 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.155 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.156 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.156 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.156 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:53.156 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:53.156 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:52 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/596428778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec106520 msgr2=0x7fe1ec1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/596428778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec106520 0x7fe1ec1069d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009ca0 tx=0x7fe1e402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/596428778 shutdown_connections 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/596428778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec106520 0x7fe1ec1069d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/596428778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec103970 0x7fe1ec105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.237+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/596428778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 0x7fe1ec1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/596428778 >> 172.21.15.96:0/596428778 conn(0x7fe1ec0fd1e0 msgr2=0x7fe1ec0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/596428778 shutdown_connections 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/596428778 wait complete. 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 Processor -- start 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 -- start start 2024-09-10T22:41:53.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.238+0000 7fe1f3dd2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 0x7fe1ec19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec106520 0x7fe1ec1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec114c70 con 0x7fe1ec103970 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec114af0 con 0x7fe1ec06b6e0 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec114df0 con 0x7fe1ec106520 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43906/0 (socket says 172.21.15.96:43906) 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 -- 172.21.15.96:0/2982007471 learned_addr learned my addr 172.21.15.96:0/2982007471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f236f640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec106520 0x7fe1ec1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:53.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f1b6e640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 0x7fe1ec19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 -- 172.21.15.96:0/2982007471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec106520 msgr2=0x7fe1ec1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec106520 0x7fe1ec1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 -- 172.21.15.96:0/2982007471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 msgr2=0x7fe1ec19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 0x7fe1ec19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 -- 172.21.15.96:0/2982007471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec1a0ec0 con 0x7fe1ec103970 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f136d640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc017020 con 0x7fe1ec103970 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1dc009c30 con 0x7fe1ec103970 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec1a11b0 con 0x7fe1ec103970 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00f6b0 con 0x7fe1ec103970 2024-09-10T22:41:53.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.239+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1a7890 con 0x7fe1ec103970 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.240+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec0fecd0 con 0x7fe1ec103970 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.242+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fe1dc024060 con 0x7fe1ec103970 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.243+0000 7fe1e2ffd640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 0x7fe1d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.243+0000 7fe1f1b6e640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 0x7fe1d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.243+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4533+0+0 (secure 0 0 0) 0x7fe1dc0e4c90 con 0x7fe1ec103970 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.243+0000 7fe1f1b6e640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 0x7fe1d80b77a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800bbe0 tx=0x7fe1e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:53.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.246+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1dc009de0 con 0x7fe1ec103970 2024-09-10T22:41:53.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:41:53.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='osd.5 ' entity='osd.5' 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:53.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:53.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:52 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='osd.5 ' entity='osd.5' 2024-09-10T22:41:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:41:53.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:52 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:41:53.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:41:53.439+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe1ec063590 con 0x7fe1d80b52e0 2024-09-10T22:41:54.149 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:41:54.149 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:41:54.149 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi121 2024-09-10T22:41:54.149 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] boot 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:54.150 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:54.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi121 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] boot 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:54.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:54.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:41:54.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:41:54.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi121 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: osd.5 [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] boot 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:41:54.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:41:55.123 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:55 smithi096 ceph-mon[30869]: from='client.14481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:55.123 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:55 smithi096 ceph-mon[30869]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:41:55.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:55 smithi121 ceph-mon[35119]: from='client.14481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:55.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:55 smithi121 ceph-mon[35119]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:41:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:55 smithi117 ceph-mon[35081]: from='client.14481 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:41:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:55 smithi117 ceph-mon[35081]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T22:41:56.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:56 smithi096 ceph-mon[30869]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:56.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:56 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"} : dispatch 2024-09-10T22:41:56.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:56 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"}]': finished 2024-09-10T22:41:56.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:56 smithi096 ceph-mon[30869]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:41:56.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:41:56.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:56 smithi121 ceph-mon[35119]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:56.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:56 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"} : dispatch 2024-09-10T22:41:56.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:56 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"}]': finished 2024-09-10T22:41:56.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:56 smithi121 ceph-mon[35119]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:41:56.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:41:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:56 smithi117 ceph-mon[35081]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:56 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"} : dispatch 2024-09-10T22:41:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:56 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2350624850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372e13d3-06be-488a-aac3-1bae70af8eb9"}]': finished 2024-09-10T22:41:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:56 smithi117 ceph-mon[35081]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T22:41:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:41:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:57 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/914350609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:57.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:57 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/914350609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:57 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/914350609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:41:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:58 smithi096 ceph-mon[30869]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:58.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:58 smithi121 ceph-mon[35119]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:58.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:58 smithi117 ceph-mon[35081]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:41:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:41:59 smithi096 ceph-mon[30869]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:41:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:41:59 smithi121 ceph-mon[35119]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:41:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:41:59 smithi117 ceph-mon[35081]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:42:01.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:01 smithi096 ceph-mon[30869]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:42:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:01 smithi121 ceph-mon[35119]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:42:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:01 smithi117 ceph-mon[35081]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:42:03.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:03 smithi096 ceph-mon[30869]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:42:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:03 smithi121 ceph-mon[35119]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:42:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:03 smithi117 ceph-mon[35081]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:42:04.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:42:04.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:04 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:42:04.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:04 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T22:42:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:04 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:05.638 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:05 smithi096 ceph-mon[30869]: Deploying daemon osd.6 on smithi096 2024-09-10T22:42:05.638 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:05 smithi096 ceph-mon[30869]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:42:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:05 smithi121 ceph-mon[35119]: Deploying daemon osd.6 on smithi096 2024-09-10T22:42:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:05 smithi121 ceph-mon[35119]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:42:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:05 smithi117 ceph-mon[35081]: Deploying daemon osd.6 on smithi096 2024-09-10T22:42:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:05 smithi117 ceph-mon[35081]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:42:07.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:07 smithi096 ceph-mon[30869]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T22:42:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:07 smithi121 ceph-mon[35119]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T22:42:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:07 smithi117 ceph-mon[35081]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T22:42:09.495 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:09 smithi096 ceph-mon[30869]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:42:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:09 smithi121 ceph-mon[35119]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:42:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:09 smithi117 ceph-mon[35081]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:42:10.733 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:10.733 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:10.733 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:11 smithi096 ceph-mon[30869]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:11 smithi121 ceph-mon[35119]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:11 smithi117 ceph-mon[35081]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:13.742 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:13 smithi096 ceph-mon[30869]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:13 smithi121 ceph-mon[35119]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:13 smithi117 ceph-mon[35081]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:13.779 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 6 on host 'smithi096' 2024-09-10T22:42:13.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.777+0000 7fe1e2ffd640 1 -- 172.21.15.96:0/2982007471 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1ec063590 con 0x7fe1d80b52e0 2024-09-10T22:42:13.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 msgr2=0x7fe1d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:13.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 0x7fe1d80b77a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800bbe0 tx=0x7fe1e800b390 comp rx=0 tx=0).stop 2024-09-10T22:42:13.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 msgr2=0x7fe1ec19f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:13.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).stop 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 shutdown_connections 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1d80b52e0 0x7fe1d80b77a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1ec106520 0x7fe1ec1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1ec103970 0x7fe1ec19f9e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 --2- 172.21.15.96:0/2982007471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1ec06b6e0 0x7fe1ec19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 >> 172.21.15.96:0/2982007471 conn(0x7fe1ec0fd1e0 msgr2=0x7fe1ec101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 shutdown_connections 2024-09-10T22:42:13.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:42:13.780+0000 7fe1f3dd2640 1 -- 172.21.15.96:0/2982007471 wait complete. 2024-09-10T22:42:14.492 DEBUG:teuthology.orchestra.run.smithi096:osd.6> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T22:42:14.495 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/vg_nvme/lv_2... 2024-09-10T22:42:14.495 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:42:14.753 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:42:15.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.036 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.036 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.036 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:15.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:16.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:15 smithi117 ceph-mon[35081]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:16.058 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:15 smithi096 ceph-mon[30869]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:15 smithi121 ceph-mon[35119]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:17.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:16 smithi096 ceph-mon[30869]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:42:17.001 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 22:42:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T22:42:16.760+0000 7f426ff5c640 -1 osd.6 0 log_to_monitors true 2024-09-10T22:42:17.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:16 smithi121 ceph-mon[35119]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:42:17.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:16 smithi117 ceph-mon[35081]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T22:42:17.573 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:42:17.605 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-09-10T22:42:17.861 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:42:17.880 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:17 smithi117 ceph-mon[35081]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:42:17.880 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:17 smithi117 ceph-mon[35081]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:42:17.880 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:17 smithi117 ceph-mon[35081]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:42:17.880 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:17 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:17.880 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:17 smithi117 ceph-mon[35081]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:18.250 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T22:42:17.790+0000 7f426bedf640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T22:42:18.250 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T22:42:17.797+0000 7f4267d06640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:42:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-mon[30869]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:42:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-mon[30869]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:42:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-mon[30869]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:42:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:18.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:17 smithi096 ceph-mon[30869]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:18.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:17 smithi121 ceph-mon[35119]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T22:42:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:17 smithi121 ceph-mon[35119]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T22:42:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:17 smithi121 ceph-mon[35119]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:42:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:17 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:17 smithi121 ceph-mon[35119]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:18.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 -- 172.21.15.117:0/1448856867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50101910 msgr2=0x7f1a50101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:18.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 --2- 172.21.15.117:0/1448856867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50101910 0x7f1a50101d10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009930 tx=0x7f1a4002a420 comp rx=0 tx=0).stop 2024-09-10T22:42:18.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 -- 172.21.15.117:0/1448856867 shutdown_connections 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 --2- 172.21.15.117:0/1448856867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50103430 0x7f1a50109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 --2- 172.21.15.117:0/1448856867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50102a70 0x7f1a50102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 --2- 172.21.15.117:0/1448856867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50101910 0x7f1a50101d10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 -- 172.21.15.117:0/1448856867 >> 172.21.15.117:0/1448856867 conn(0x7f1a500fd1e0 msgr2=0x7f1a500ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 -- 172.21.15.117:0/1448856867 shutdown_connections 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.340+0000 7f1a56732640 1 -- 172.21.15.117:0/1448856867 wait complete. 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 Processor -- start 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 -- start start 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50101910 0x7f1a50198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 0x7f1a50199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:18.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d590 con 0x7f1a50102a70 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d410 con 0x7f1a50103430 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.341+0000 7f1a56732640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d710 con 0x7f1a50101910 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a54ccf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a54ccf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40652/0 (socket says 172.21.15.117:40652) 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a54ccf640 1 -- 172.21.15.117:0/2203307055 learned_addr learned my addr 172.21.15.117:0/2203307055 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 0x7f1a50199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4ffff640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50101910 0x7f1a50198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 -- 172.21.15.117:0/2203307055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50101910 msgr2=0x7f1a50198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50101910 0x7f1a50198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 -- 172.21.15.117:0/2203307055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 msgr2=0x7f1a5019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:18.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 -- 172.21.15.117:0/2203307055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5019a960 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a54ccf640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4f7fe640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 0x7f1a50199480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400d720 tx=0x7f1a4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44017070 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4400de30 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5019ac50 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a440047a0 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.342+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a50103f50 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.343+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a50101d10 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.345+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f1a440099d0 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.345+0000 7f1a4d7fa640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 0x7f1a380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.346+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f1a440e4da0 con 0x7f1a50102a70 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.346+0000 7f1a4ffff640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 0x7f1a380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:18.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.346+0000 7f1a4ffff640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 0x7f1a380b7820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009900 tx=0x7f1a40031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:42:18.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.349+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1a44020020 con 0x7f1a50102a70 2024-09-10T22:42:18.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:18.541+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1a500ff3a0 con 0x7f1a380b5360 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi096 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.123 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:19.124 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:18 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi096 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:19.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:18 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi096 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:19.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:19.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:18 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:20.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] boot 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:20.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:19 smithi117 ceph-mon[35081]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] boot 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:42:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:20.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:19 smithi096 ceph-mon[30869]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:20.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: osd.6 [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] boot 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T22:42:20.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:19 smithi121 ceph-mon[35119]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T22:42:21.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:20 smithi117 ceph-mon[35081]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:42:21.201 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:20 smithi121 ceph-mon[35119]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:42:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:20 smithi096 ceph-mon[30869]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T22:42:22.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/1843403683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"}]': finished 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:22.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:21 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/257409108' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:22.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/1843403683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"}]': finished 2024-09-10T22:42:22.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:42:22.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:22.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:22.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:21 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/257409108' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/1843403683' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"} : dispatch 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "783ef350-a8c5-4e79-b761-13da9d17c03b"}]': finished 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:22.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:21 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/257409108' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:23 smithi096 ceph-mon[30869]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:23 smithi121 ceph-mon[35119]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:23 smithi117 ceph-mon[35081]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:25 smithi096 ceph-mon[30869]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:25 smithi121 ceph-mon[35119]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:25 smithi117 ceph-mon[35081]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:27.592 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:27 smithi117 ceph-mon[35081]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:27 smithi096 ceph-mon[30869]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:27 smithi121 ceph-mon[35119]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:29.679 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:29 smithi117 ceph-mon[35081]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:29 smithi096 ceph-mon[30869]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:29 smithi121 ceph-mon[35119]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:30.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:42:30.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:30.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:30 smithi117 ceph-mon[35081]: Deploying daemon osd.7 on smithi117 2024-09-10T22:42:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:42:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:30 smithi096 ceph-mon[30869]: Deploying daemon osd.7 on smithi117 2024-09-10T22:42:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T22:42:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:30 smithi121 ceph-mon[35119]: Deploying daemon osd.7 on smithi117 2024-09-10T22:42:31.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:31 smithi117 ceph-mon[35081]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:31 smithi096 ceph-mon[30869]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:31 smithi121 ceph-mon[35119]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:33 smithi117 ceph-mon[35081]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:33 smithi096 ceph-mon[30869]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:33 smithi121 ceph-mon[35119]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:35.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:35 smithi117 ceph-mon[35081]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:35 smithi096 ceph-mon[30869]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:35 smithi121 ceph-mon[35119]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:36.672 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:36.672 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:36.673 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:37.512 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:37 smithi117 ceph-mon[35081]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:37 smithi096 ceph-mon[30869]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:37 smithi121 ceph-mon[35119]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:39.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:39.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:39.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:39.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:39.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:39.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:39 smithi117 ceph-mon[35081]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:39.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.962+0000 7f1a4d7fa640 1 -- 172.21.15.117:0/2203307055 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a500ff3a0 con 0x7f1a380b5360 2024-09-10T22:42:39.964 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2024-09-10T22:42:39.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 msgr2=0x7f1a380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:39.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 0x7f1a380b7820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009900 tx=0x7f1a40031430 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 msgr2=0x7f1a50199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 0x7f1a50199480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400d720 tx=0x7f1a4400dbe0 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 shutdown_connections 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a50103430 0x7f1a5019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1a380b5360 0x7f1a380b7820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a50102a70 0x7f1a50199480 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 --2- 172.21.15.117:0/2203307055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a50101910 0x7f1a50198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 >> 172.21.15.117:0/2203307055 conn(0x7f1a500fd1e0 msgr2=0x7f1a500fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 shutdown_connections 2024-09-10T22:42:39.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:42:39.965+0000 7f1a56732640 1 -- 172.21.15.117:0/2203307055 wait complete. 2024-09-10T22:42:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:39 smithi096 ceph-mon[30869]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:39 smithi121 ceph-mon[35119]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:40.658 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T22:42:40.660 INFO:tasks.cephadm:Deploying osd.8 on smithi121 with /dev/vg_nvme/lv_2... 2024-09-10T22:42:40.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T22:42:40.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:40.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:40.920 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:42:41.190 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:41.191 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:41.191 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:41.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:42:41.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:41.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:42.137 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:41 smithi121 ceph-mon[35119]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:42.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:41 smithi117 ceph-mon[35081]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:42.138 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 22:42:41 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T22:42:41.877+0000 7f5929c80640 -1 osd.7 0 log_to_monitors true 2024-09-10T22:42:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:41 smithi096 ceph-mon[30869]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:43.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:42 smithi096 ceph-mon[30869]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:42 smithi096 ceph-mon[30869]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:42 smithi121 ceph-mon[35119]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:42 smithi121 ceph-mon[35119]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.270 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:42 smithi117 ceph-mon[35081]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.270 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:42 smithi117 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T22:42:43.545 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:42:43.571 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_2 2024-09-10T22:42:43.816 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:42:43.874 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T22:42:43.864+0000 7f5925c03640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T22:42:43.874 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T22:42:43.872+0000 7f5921a2a640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:42:43.874 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:42:43.874 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:42:43.874 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:43.874 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:43.875 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.196 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:42:44.196 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:44.197 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T22:42:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T22:42:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:42:44.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 -- 172.21.15.121:0/3209821336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4102ab0 msgr2=0x7f91d4102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 --2- 172.21.15.121:0/3209821336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4102ab0 0x7f91d4102f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009960 tx=0x7f91c802a460 comp rx=0 tx=0).stop 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 -- 172.21.15.121:0/3209821336 shutdown_connections 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 --2- 172.21.15.121:0/3209821336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4103470 0x7f91d4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 --2- 172.21.15.121:0/3209821336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4102ab0 0x7f91d4102f30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 --2- 172.21.15.121:0/3209821336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4101950 0x7f91d4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 -- 172.21.15.121:0/3209821336 >> 172.21.15.121:0/3209821336 conn(0x7f91d40fd200 msgr2=0x7f91d40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.277+0000 7f91da231640 1 -- 172.21.15.121:0/3209821336 shutdown_connections 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 -- 172.21.15.121:0/3209821336 wait complete. 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 Processor -- start 2024-09-10T22:42:44.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 -- start start 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4102ab0 0x7f91d419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4103470 0x7f91d419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410e740 con 0x7f91d4103470 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410e5c0 con 0x7f91d4102ab0 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91da231640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91d410e8c0 con 0x7f91d4101950 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.278+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:49022/0 (socket says 172.21.15.121:49022) 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 -- 172.21.15.121:0/950674205 learned_addr learned my addr 172.21.15.121:0/950674205 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-10T22:42:44.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d2ffd640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4102ab0 0x7f91d419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d3fff640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4103470 0x7f91d419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 -- 172.21.15.121:0/950674205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4102ab0 msgr2=0x7f91d419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4102ab0 0x7f91d419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 -- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4103470 msgr2=0x7f91d419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4103470 0x7f91d419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 -- 172.21.15.121:0/950674205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d419cc20 con 0x7f91d4101950 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d37fe640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b4a0 tx=0x7f91c400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4014070 con 0x7f91d4101950 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c4010ac0 con 0x7f91d4101950 2024-09-10T22:42:44.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4017c30 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d419cf10 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.279+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d41a35f0 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.280+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d41a37c0 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.282+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f91c4010610 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.282+0000 7f91d0ff9640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 0x7f91bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.282+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7f91c40e4820 con 0x7f91d4101950 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.283+0000 7f91d2ffd640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 0x7f91bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.283+0000 7f91d2ffd640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 0x7f91bc0b77a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91c8000c00 tx=0x7f91c8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:42:44.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.285+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f91c40ac360 con 0x7f91d4101950 2024-09-10T22:42:44.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:42:44.471+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f91d410bfa0 con 0x7f91bc0b52e0 2024-09-10T22:42:45.174 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi117 2024-09-10T22:42:45.174 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-10T22:42:45.174 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:45.175 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi117 2024-09-10T22:42:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-10T22:42:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:42:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:42:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi117 2024-09-10T22:42:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-10T22:42:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:42:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T22:42:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:42:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:42:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:45.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:42:45.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:42:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] boot 2024-09-10T22:42:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:42:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:45 smithi121 ceph-mon[35119]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:42:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:42:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] boot 2024-09-10T22:42:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:42:46.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:46.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:45 smithi096 ceph-mon[30869]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: osd.7 [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] boot 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T22:42:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T22:42:46.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:45 smithi117 ceph-mon[35081]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:47.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:46 smithi121 ceph-mon[35119]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:42:47.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:46 smithi096 ceph-mon[30869]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:42:47.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:46 smithi117 ceph-mon[35081]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/1891701906' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"}]': finished 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:42:48.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:42:48.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:48.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:47 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/561791029' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:48.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/1891701906' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"}]': finished 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:48.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:47 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/561791029' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T22:42:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/1891701906' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"} : dispatch 2024-09-10T22:42:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55e87abf-88f6-47fe-9297-9c71f8997270"}]': finished 2024-09-10T22:42:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T22:42:48.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:42:48.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:48.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:47 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/561791029' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:42:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:49 smithi096 ceph-mon[30869]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:49 smithi121 ceph-mon[35119]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:49.773 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:49 smithi117 ceph-mon[35081]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:42:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:51 smithi096 ceph-mon[30869]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:42:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:51 smithi121 ceph-mon[35119]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:42:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:51 smithi117 ceph-mon[35081]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T22:42:53.618 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:53 smithi121 ceph-mon[35119]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T22:42:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:53 smithi096 ceph-mon[30869]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T22:42:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:53 smithi117 ceph-mon[35081]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T22:42:55.711 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:55 smithi121 ceph-mon[35119]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T22:42:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:55 smithi096 ceph-mon[30869]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T22:42:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:55 smithi117 ceph-mon[35081]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T22:42:56.506 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:42:56.506 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:56.506 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:56 smithi121 ceph-mon[35119]: Deploying daemon osd.8 on smithi121 2024-09-10T22:42:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:42:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:56 smithi096 ceph-mon[30869]: Deploying daemon osd.8 on smithi121 2024-09-10T22:42:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T22:42:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:42:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:56 smithi117 ceph-mon[35081]: Deploying daemon osd.8 on smithi121 2024-09-10T22:42:57.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:57 smithi121 ceph-mon[35119]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:42:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:57 smithi096 ceph-mon[30869]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:42:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:57 smithi117 ceph-mon[35081]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T22:42:59.444 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:42:59 smithi121 ceph-mon[35119]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:42:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:42:59 smithi096 ceph-mon[30869]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:42:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:42:59 smithi117 ceph-mon[35081]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:01.692 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:01 smithi121 ceph-mon[35119]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:01 smithi096 ceph-mon[30869]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:01 smithi117 ceph-mon[35081]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:02.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:02.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:03.589 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:03 smithi121 ceph-mon[35119]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:03 smithi096 ceph-mon[30869]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:03 smithi117 ceph-mon[35081]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:05.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.951+0000 7f91d0ff9640 1 -- 172.21.15.121:0/950674205 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91d410bfa0 con 0x7f91bc0b52e0 2024-09-10T22:43:05.954 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 8 on host 'smithi121' 2024-09-10T22:43:05.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 msgr2=0x7f91bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 0x7f91bc0b77a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91c8000c00 tx=0x7f91c8005d50 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 msgr2=0x7f91d419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f91c400b4a0 tx=0x7f91c400b960 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 shutdown_connections 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91d4102ab0 0x7f91d419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91bc0b52e0 0x7f91bc0b77a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d4103470 0x7f91d419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 --2- 172.21.15.121:0/950674205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91d4101950 0x7f91d419b0f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 >> 172.21.15.121:0/950674205 conn(0x7f91d40fd200 msgr2=0x7f91d410a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 shutdown_connections 2024-09-10T22:43:05.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:43:05.954+0000 7f91da231640 1 -- 172.21.15.121:0/950674205 wait complete. 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:05 smithi096 ceph-mon[30869]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:06.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:06.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:06.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:05 smithi121 ceph-mon[35119]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:06.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:05 smithi117 ceph-mon[35081]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:06.653 DEBUG:teuthology.orchestra.run.smithi121:osd.8> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T22:43:06.655 INFO:tasks.cephadm:Deploying osd.9 on smithi096 with /dev/vg_nvme/lv_1... 2024-09-10T22:43:06.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:43:06.917 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:07.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:06 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:07.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:06 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:07.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:06 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:07.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:06 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:07.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:06 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:07.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:06 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:07.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:06 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:07.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:06 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:07.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:06 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:08.117 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:07 smithi121 ceph-mon[35119]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:08.118 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:07 smithi121 ceph-mon[35119]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:08.118 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:07 smithi121 ceph-mon[35119]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:08.118 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 22:43:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T22:43:07.766+0000 7fc62ce2a640 -1 osd.8 0 log_to_monitors true 2024-09-10T22:43:08.196 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:07 smithi096 ceph-mon[30869]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:08.196 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:07 smithi096 ceph-mon[30869]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:08.196 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:07 smithi096 ceph-mon[30869]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:08.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:07 smithi117 ceph-mon[35081]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:08.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:07 smithi117 ceph-mon[35081]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:08.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:07 smithi117 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T22:43:09.015 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:08 smithi096 ceph-mon[30869]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:43:09.015 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:08 smithi096 ceph-mon[30869]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:08 smithi096 ceph-mon[30869]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:43:09.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:08 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:09.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:08 smithi096 ceph-mon[30869]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:08 smithi121 ceph-mon[35119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:43:09.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:08 smithi121 ceph-mon[35119]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:08 smithi121 ceph-mon[35119]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:43:09.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:08 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:09.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:08 smithi121 ceph-mon[35119]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:08 smithi117 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T22:43:09.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:08 smithi117 ceph-mon[35081]: from='osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:08 smithi117 ceph-mon[35081]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T22:43:09.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:08 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:09.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:08 smithi117 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:43:09.613 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T22:43:09.339+0000 7fc628dad640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T22:43:09.613 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T22:43:09.349+0000 7fc620bcc640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:43:09.833 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:43:09.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-09-10T22:43:10.124 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:43:10.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:43:10.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='osd.8 ' entity='osd.8' 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.246 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:10.247 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:09 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:10.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='osd.8 ' entity='osd.8' 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:10.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:09 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:43:10.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T22:43:10.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='osd.8 ' entity='osd.8' 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:10.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:09 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 -- 172.21.15.96:0/3464904047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08108750 msgr2=0x7fdf0810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/3464904047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08108750 0x7fdf0810ac10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009ca0 tx=0x7fdf0002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 -- 172.21.15.96:0/3464904047 shutdown_connections 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/3464904047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08108750 0x7fdf0810ac10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/3464904047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf08105ba0 0x7fdf08108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/3464904047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf0806c0e0 0x7fdf08105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.573+0000 7fdf0dd21640 1 -- 172.21.15.96:0/3464904047 >> 172.21.15.96:0/3464904047 conn(0x7fdf080fd220 msgr2=0x7fdf080ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:43:10.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- 172.21.15.96:0/3464904047 shutdown_connections 2024-09-10T22:43:10.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- 172.21.15.96:0/3464904047 wait complete. 2024-09-10T22:43:10.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 Processor -- start 2024-09-10T22:43:10.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- start start 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0806c0e0 0x7fdf0819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08108750 0x7fdf0819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e6d0 con 0x7fdf08105ba0 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e550 con 0x7fdf08108750 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.574+0000 7fdf0dd21640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810e850 con 0x7fdf0806c0e0 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:10.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41672/0 (socket says 172.21.15.96:41672) 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 -- 172.21.15.96:0/2205307610 learned_addr learned my addr 172.21.15.96:0/2205307610 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf07fff640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08108750 0x7fdf0819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf077fe640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0806c0e0 0x7fdf0819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 -- 172.21.15.96:0/2205307610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0806c0e0 msgr2=0x7fdf0819b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0806c0e0 0x7fdf0819b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 -- 172.21.15.96:0/2205307610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08108750 msgr2=0x7fdf0819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08108750 0x7fdf0819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 -- 172.21.15.96:0/2205307610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0819cb30 con 0x7fdf08105ba0 2024-09-10T22:43:10.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf06ffd640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c970 tx=0x7fdefc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc007bb0 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdefc007d50 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0819ce20 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc018730 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.575+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a3500 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.576+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf081a36d0 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.578+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fdefc020030 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.578+0000 7fdf04ff9640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 0x7fdef00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.578+0000 7fdf077fe640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 0x7fdef00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.578+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5862+0+0 (secure 0 0 0) 0x7fdefc0e5240 con 0x7fdf08105ba0 2024-09-10T22:43:10.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.579+0000 7fdf077fe640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 0x7fdef00b7820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdef8005d00 tx=0x7fdef8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:43:10.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.581+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdefc014030 con 0x7fdf08105ba0 2024-09-10T22:43:10.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:10.766+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdf0810a420 con 0x7fdef00b5360 2024-09-10T22:43:11.174 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:43:11.174 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi121 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] boot 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:11.175 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:11.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi121 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] boot 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi121 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: osd.8 [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] boot 2024-09-10T22:43:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T22:43:11.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T22:43:11.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:11.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:11.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:11 smithi096 ceph-mon[30869]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:11 smithi096 ceph-mon[30869]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:43:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:11 smithi096 ceph-mon[30869]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:11 smithi121 ceph-mon[35119]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:11 smithi121 ceph-mon[35119]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:43:12.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:11 smithi121 ceph-mon[35119]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:11 smithi117 ceph-mon[35081]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:11 smithi117 ceph-mon[35081]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T22:43:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:11 smithi117 ceph-mon[35081]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:12 smithi096 ceph-mon[30869]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:43:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:12 smithi121 ceph-mon[35119]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:43:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:12 smithi117 ceph-mon[35081]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T22:43:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"} : dispatch 2024-09-10T22:43:14.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"}]': finished 2024-09-10T22:43:14.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T22:43:14.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:14.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2090106028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:14.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:14.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"} : dispatch 2024-09-10T22:43:14.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"}]': finished 2024-09-10T22:43:14.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T22:43:14.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:14.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2090106028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"} : dispatch 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2523465350' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b757f430-ca41-4632-8891-d74c23873b69"}]': finished 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:14.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2090106028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:15 smithi096 ceph-mon[30869]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-10T22:43:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:15 smithi121 ceph-mon[35119]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-10T22:43:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:15 smithi117 ceph-mon[35081]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-10T22:43:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:17 smithi096 ceph-mon[30869]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T22:43:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:17 smithi121 ceph-mon[35119]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T22:43:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:17 smithi117 ceph-mon[35081]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T22:43:19.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:19 smithi096 ceph-mon[30869]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:43:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:19 smithi121 ceph-mon[35119]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:43:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:19 smithi117 ceph-mon[35081]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:43:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:21 smithi096 ceph-mon[30869]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:43:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:21 smithi121 ceph-mon[35119]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:43:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:21 smithi117 ceph-mon[35081]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:43:22.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:43:22.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:22 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:22.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:22 smithi096 ceph-mon[30869]: Deploying daemon osd.9 on smithi096 2024-09-10T22:43:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:43:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:22 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:22 smithi121 ceph-mon[35119]: Deploying daemon osd.9 on smithi096 2024-09-10T22:43:22.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T22:43:22.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:22 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:22.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:22 smithi117 ceph-mon[35081]: Deploying daemon osd.9 on smithi096 2024-09-10T22:43:23.594 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:23 smithi096 ceph-mon[30869]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:43:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:23 smithi121 ceph-mon[35119]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:43:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:23 smithi117 ceph-mon[35081]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T22:43:25.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:25 smithi096 ceph-mon[30869]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:25 smithi121 ceph-mon[35119]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:25 smithi117 ceph-mon[35081]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T22:43:27.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:27 smithi096 ceph-mon[30869]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:27 smithi121 ceph-mon[35119]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:27 smithi117 ceph-mon[35081]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:28.657 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:28.657 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:28.657 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:29.578 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:29 smithi096 ceph-mon[30869]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:29.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:29 smithi121 ceph-mon[35119]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:29 smithi117 ceph-mon[35081]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:31.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.337+0000 7fdf04ff9640 1 -- 172.21.15.96:0/2205307610 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf0810a420 con 0x7fdef00b5360 2024-09-10T22:43:31.339 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 9 on host 'smithi096' 2024-09-10T22:43:31.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 msgr2=0x7fdef00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:31.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 0x7fdef00b7820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdef8005d00 tx=0x7fdef8005c50 comp rx=0 tx=0).stop 2024-09-10T22:43:31.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 msgr2=0x7fdf0819b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:31.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c970 tx=0x7fdefc00ce30 comp rx=0 tx=0).stop 2024-09-10T22:43:31.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 shutdown_connections 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08108750 0x7fdf0819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdef00b5360 0x7fdef00b7820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08105ba0 0x7fdf0819b650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 --2- 172.21.15.96:0/2205307610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0806c0e0 0x7fdf0819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 >> 172.21.15.96:0/2205307610 conn(0x7fdf080fd220 msgr2=0x7fdf080ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 shutdown_connections 2024-09-10T22:43:31.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:43:31.340+0000 7fdf0dd21640 1 -- 172.21.15.96:0/2205307610 wait complete. 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.953 DEBUG:teuthology.orchestra.run.smithi096:osd.9> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T22:43:31.956 INFO:tasks.cephadm:Deploying osd.10 on smithi117 with /dev/vg_nvme/lv_1... 2024-09-10T22:43:31.956 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:43:31.984 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:31.985 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:32.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:32.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:32.212 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:43:33.495 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:33 smithi096 ceph-mon[30869]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:33.622 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:33 smithi117 ceph-mon[35081]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:33 smithi121 ceph-mon[35119]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:34 smithi096 ceph-mon[30869]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:43:34.500 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 22:43:34 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T22:43:34.144+0000 7f5d06df6640 -1 osd.9 0 log_to_monitors true 2024-09-10T22:43:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:34 smithi117 ceph-mon[35081]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:43:34.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:34 smithi121 ceph-mon[35119]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T22:43:34.876 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T22:43:34.902 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-09-10T22:43:35.202 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:43:35.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:43:35.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:43:35.659 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi096 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:35.660 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:35.661 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.661 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:35 smithi117 ceph-mon[35081]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:35.739 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi096 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-10T22:43:35.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:35.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:35.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-mon[30869]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]} : dispatch 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi096 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:35.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:35 smithi121 ceph-mon[35119]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:35.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- 172.21.15.117:0/958562249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8103ba0 msgr2=0x7fb8e8104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:35.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/958562249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8104020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009930 tx=0x7fb8d802a420 comp rx=0 tx=0).stop 2024-09-10T22:43:35.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- 172.21.15.117:0/958562249 shutdown_connections 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/958562249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8104020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/958562249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8104560 0x7fb8e810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/958562249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 0x7fb8e8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- 172.21.15.117:0/958562249 >> 172.21.15.117:0/958562249 conn(0x7fb8e80fe2f0 msgr2=0x7fb8e8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- 172.21.15.117:0/958562249 shutdown_connections 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- 172.21.15.117:0/958562249 wait complete. 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 Processor -- start 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.742+0000 7fb8eedb2640 1 -- start start 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 0x7fb8e8076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e6f0 con 0x7fb8e8103ba0 2024-09-10T22:43:35.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e570 con 0x7fb8e8104560 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e870 con 0x7fb8e8102a40 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51882/0 (socket says 172.21.15.117:51882) 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 -- 172.21.15.117:0/3647496994 learned_addr learned my addr 172.21.15.117:0/3647496994 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8dffff640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ecb4e640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 0x7fb8e8076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 -- 172.21.15.117:0/3647496994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 msgr2=0x7fb8e8076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 0x7fb8e8076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 -- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8103ba0 msgr2=0x7fb8e8075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 -- 172.21.15.117:0/3647496994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e8076340 con 0x7fb8e8104560 2024-09-10T22:43:35.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ed34f640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000eb10 tx=0x7fb8e000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004b20 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e0004cc0 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81a2f00 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.743+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0010670 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.744+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a3360 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.745+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e8109d60 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.746+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb8e0005020 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.746+0000 7fb8ddffb640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 0x7fb8d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.747+0000 7fb8ecb4e640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 0x7fb8d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.747+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7fb8e00e57e0 con 0x7fb8e8104560 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.747+0000 7fb8ecb4e640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 0x7fb8d00b77a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005d00 tx=0x7fb8e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:43:35.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.750+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8e00acfc0 con 0x7fb8e8104560 2024-09-10T22:43:35.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:35.938+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb8e8063590 con 0x7fb8d00b52e0 2024-09-10T22:43:36.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T22:43:35.737+0000 7f5d02d79640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T22:43:36.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 22:43:35 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T22:43:35.748+0000 7f5cfab98640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:43:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:43:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:43:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' 2024-09-10T22:43:36.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:36.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:36.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:36.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:36.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-10T22:43:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T22:43:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888]' entity='osd.9' 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='client.24434 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:43:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] boot 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T22:43:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:37 smithi096 ceph-mon[30869]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:37.753 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:43:37.753 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:43:37.753 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.754 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] boot 2024-09-10T22:43:37.754 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T22:43:37.754 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.754 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:37 smithi117 ceph-mon[35081]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:43:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:43:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: osd.9 [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] boot 2024-09-10T22:43:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T22:43:37.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T22:43:37.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:37 smithi121 ceph-mon[35119]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-10T22:43:39.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:43:39.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/1232645416' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"}]': finished 2024-09-10T22:43:39.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T22:43:39.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/1232645416' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"}]': finished 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T22:43:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/1232645416' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"} : dispatch 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e49921d8-dcf9-427a-8a29-57f0450576f4"}]': finished 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T22:43:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:43:40.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.117:0/1802307107' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:40.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:39 smithi117 ceph-mon[35081]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.117:0/1802307107' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:39 smithi096 ceph-mon[30869]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.117:0/1802307107' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:43:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:39 smithi121 ceph-mon[35119]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:41 smithi096 ceph-mon[30869]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:41 smithi121 ceph-mon[35119]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:41 smithi117 ceph-mon[35081]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:43.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:43 smithi117 ceph-mon[35081]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:43 smithi096 ceph-mon[30869]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:43 smithi121 ceph-mon[35119]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:45 smithi096 ceph-mon[30869]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:45 smithi121 ceph-mon[35119]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:45 smithi117 ceph-mon[35081]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:43:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:47 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:43:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:47 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:47.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T22:43:47.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:47 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:43:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:48 smithi096 ceph-mon[30869]: Deploying daemon osd.10 on smithi117 2024-09-10T22:43:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:48 smithi096 ceph-mon[30869]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:48.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:48 smithi121 ceph-mon[35119]: Deploying daemon osd.10 on smithi117 2024-09-10T22:43:48.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:48 smithi121 ceph-mon[35119]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:48 smithi117 ceph-mon[35081]: Deploying daemon osd.10 on smithi117 2024-09-10T22:43:48.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:48 smithi117 ceph-mon[35081]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:49.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:49 smithi117 ceph-mon[35081]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:49 smithi096 ceph-mon[30869]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:49 smithi121 ceph-mon[35119]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:51.667 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:51 smithi117 ceph-mon[35081]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:51 smithi096 ceph-mon[30869]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:51 smithi121 ceph-mon[35119]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:53 smithi117 ceph-mon[35081]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:53 smithi096 ceph-mon[30869]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:53 smithi121 ceph-mon[35119]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:43:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:55 smithi117 ceph-mon[35081]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:55.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:55 smithi096 ceph-mon[30869]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:55 smithi121 ceph-mon[35119]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:57.210 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 10 on host 'smithi117' 2024-09-10T22:43:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.208+0000 7fb8ddffb640 1 -- 172.21.15.117:0/3647496994 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb8e8063590 con 0x7fb8d00b52e0 2024-09-10T22:43:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 msgr2=0x7fb8d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 0x7fb8d00b77a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005d00 tx=0x7fb8e400b410 comp rx=0 tx=0).stop 2024-09-10T22:43:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 msgr2=0x7fb8e8075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:43:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000eb10 tx=0x7fb8e000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 shutdown_connections 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e8104560 0x7fb8e8075a80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8d00b52e0 0x7fb8d00b77a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8e8103ba0 0x7fb8e8075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 --2- 172.21.15.117:0/3647496994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8e8102a40 0x7fb8e8076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.211+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 >> 172.21.15.117:0/3647496994 conn(0x7fb8e80fe2f0 msgr2=0x7fb8e8106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.212+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 shutdown_connections 2024-09-10T22:43:57.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-10T22:43:57.212+0000 7fb8eedb2640 1 -- 172.21.15.117:0/3647496994 wait complete. 2024-09-10T22:43:57.908 DEBUG:teuthology.orchestra.run.smithi117:osd.10> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T22:43:57.911 INFO:tasks.cephadm:Deploying osd.11 on smithi121 with /dev/vg_nvme/lv_1... 2024-09-10T22:43:57.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T22:43:58.166 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:43:58.455 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.455 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.455 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:58 smithi121 ceph-mon[35119]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.456 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:58 smithi117 ceph-mon[35081]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:43:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:58 smithi096 ceph-mon[30869]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:59.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:43:59 smithi117 ceph-mon[35081]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:59.570 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:43:59 smithi121 ceph-mon[35119]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:43:59.697 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 22:43:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T22:43:59.454+0000 7f4b4e41f640 -1 osd.10 0 log_to_monitors true 2024-09-10T22:43:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:43:59 smithi096 ceph-mon[30869]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:00.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:00 smithi121 ceph-mon[35119]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:00 smithi121 ceph-mon[35119]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:00 smithi117 ceph-mon[35081]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:00 smithi117 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:00 smithi096 ceph-mon[30869]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:00 smithi096 ceph-mon[30869]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T22:44:00.912 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T22:44:00.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_1 2024-09-10T22:44:01.200 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi117 2024-09-10T22:44:01.604 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.605 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.606 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.606 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:01 smithi121 ceph-mon[35119]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- 172.21.15.121:0/1671483500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 msgr2=0x7fe894102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/1671483500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe894102f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe884009960 tx=0x7fe88402a460 comp rx=0 tx=0).stop 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- 172.21.15.121:0/1671483500 shutdown_connections 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/1671483500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe894103460 0x7fe894109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/1671483500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe894102f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/1671483500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894101940 0x7fe894101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:01.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- 172.21.15.121:0/1671483500 >> 172.21.15.121:0/1671483500 conn(0x7fe8940fd210 msgr2=0x7fe8940ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- 172.21.15.121:0/1671483500 shutdown_connections 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- 172.21.15.121:0/1671483500 wait complete. 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 Processor -- start 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- start start 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894101940 0x7fe89406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe894103460 0x7fe8941a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410d6f0 con 0x7fe894101940 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410d570 con 0x7fe894103460 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe89c0fb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe89410d870 con 0x7fe894102aa0 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.686+0000 7fe899696640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:01.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43336/0 (socket says 172.21.15.121:43336) 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 -- 172.21.15.121:0/3781147314 learned_addr learned my addr 172.21.15.121:0/3781147314 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 -- 172.21.15.121:0/3781147314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe894103460 msgr2=0x7fe8941a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe894103460 0x7fe8941a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 -- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894101940 msgr2=0x7fe89406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894101940 0x7fe89406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 -- 172.21.15.121:0/3781147314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8941a2e10 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe899696640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe884000c00 tx=0x7fe884003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88403f070 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe884003f30 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8941a30a0 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884035030 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.687+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a3600 con 0x7fe894102aa0 2024-09-10T22:44:01.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.688+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe894108d60 con 0x7fe894102aa0 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.689+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fe88403b4f0 con 0x7fe894102aa0 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.690+0000 7fe88affd640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 0x7fe8800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.691+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7fe8841061c0 con 0x7fe894102aa0 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.691+0000 7fe899e97640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 0x7fe8800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.691+0000 7fe899e97640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 0x7fe8800b77a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe890009710 tx=0x7fe890006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:01.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.695+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe8840cd7f0 con 0x7fe894102aa0 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi117 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:01.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:01 smithi096 ceph-mon[30869]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:01.776 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T22:44:01.381+0000 7f4b4a3a2640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T22:44:01.776 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T22:44:01.391+0000 7f4b461c9640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:44:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi117 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:01.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:01.778 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:01 smithi117 ceph-mon[35081]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:01.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:01.880+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe894063590 con 0x7fe8800b52e0 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:02.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:02.566 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:02.566 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:02 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:02.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:02 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T22:44:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:02 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:03.674 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:44:03.674 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:44:03.675 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] boot 2024-09-10T22:44:03.675 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T22:44:03.675 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:03.675 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:44:03.675 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:03 smithi121 ceph-mon[35119]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:44:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:44:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] boot 2024-09-10T22:44:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T22:44:03.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:03.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:44:03.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:03 smithi096 ceph-mon[30869]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:44:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:44:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: osd.10 [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] boot 2024-09-10T22:44:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T22:44:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T22:44:03.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T22:44:03.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:03 smithi117 ceph-mon[35081]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-10T22:44:05.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/224745896' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"}]': finished 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:05.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:05 smithi121 ceph-mon[35119]: from='client.? 172.21.15.121:0/4290431765' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:44:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/224745896' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"}]': finished 2024-09-10T22:44:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:44:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:05 smithi096 ceph-mon[30869]: from='client.? 172.21.15.121:0/4290431765' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T22:44:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/224745896' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"} : dispatch 2024-09-10T22:44:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5084606-c0d3-4731-8e88-17f0ef95d91d"}]': finished 2024-09-10T22:44:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T22:44:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:05 smithi117 ceph-mon[35081]: from='client.? 172.21.15.121:0/4290431765' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T22:44:06.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:06 smithi096 ceph-mon[30869]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:44:06.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:06 smithi121 ceph-mon[35119]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:44:06.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:06 smithi117 ceph-mon[35081]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-10T22:44:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:07 smithi096 ceph-mon[30869]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T22:44:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:07 smithi121 ceph-mon[35119]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T22:44:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:07 smithi117 ceph-mon[35081]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T22:44:09.720 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:09 smithi121 ceph-mon[35119]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:44:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:09 smithi096 ceph-mon[30869]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:44:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:09 smithi117 ceph-mon[35081]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T22:44:11.648 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:11 smithi121 ceph-mon[35119]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:44:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:11 smithi096 ceph-mon[30869]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:44:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:11 smithi117 ceph-mon[35081]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T22:44:13.470 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:44:13.471 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:44:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T22:44:13.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:14 smithi096 ceph-mon[30869]: Deploying daemon osd.11 on smithi121 2024-09-10T22:44:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:14 smithi096 ceph-mon[30869]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:44:14.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:14 smithi121 ceph-mon[35119]: Deploying daemon osd.11 on smithi121 2024-09-10T22:44:14.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:14 smithi121 ceph-mon[35119]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:44:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:14 smithi117 ceph-mon[35081]: Deploying daemon osd.11 on smithi121 2024-09-10T22:44:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:14 smithi117 ceph-mon[35081]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T22:44:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:15 smithi096 ceph-mon[30869]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T22:44:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:15 smithi121 ceph-mon[35119]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T22:44:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:15 smithi117 ceph-mon[35081]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T22:44:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:17 smithi096 ceph-mon[30869]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:17 smithi121 ceph-mon[35119]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:17 smithi117 ceph-mon[35081]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:19 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:19 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:19.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:19 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:19 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:19 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:19 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:19 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:19 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:19 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:20.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:20 smithi117 ceph-mon[35081]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:20.706 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:20 smithi121 ceph-mon[35119]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:20.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:20 smithi096 ceph-mon[30869]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:21.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:21 smithi121 ceph-mon[35119]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:21 smithi096 ceph-mon[30869]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:21 smithi117 ceph-mon[35081]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:22.696 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 11 on host 'smithi121' 2024-09-10T22:44:22.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.694+0000 7fe88affd640 1 -- 172.21.15.121:0/3781147314 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe894063590 con 0x7fe8800b52e0 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 msgr2=0x7fe8800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 0x7fe8800b77a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe890009710 tx=0x7fe890006c90 comp rx=0 tx=0).stop 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 msgr2=0x7fe8941a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe884000c00 tx=0x7fe884003c80 comp rx=0 tx=0).stop 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 shutdown_connections 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe894103460 0x7fe8941a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8800b52e0 0x7fe8800b77a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe894102aa0 0x7fe8941a0140 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 --2- 172.21.15.121:0/3781147314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894101940 0x7fe89406bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 >> 172.21.15.121:0/3781147314 conn(0x7fe8940fd210 msgr2=0x7fe894105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 shutdown_connections 2024-09-10T22:44:22.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:22.696+0000 7fe89c0fb640 1 -- 172.21.15.121:0/3781147314 wait complete. 2024-09-10T22:44:23.348 DEBUG:teuthology.orchestra.run.smithi121:osd.11> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T22:44:23.350 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T22:44:23.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:44:23.610 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:23.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:23.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:23.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:23 smithi096 ceph-mon[30869]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:23.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:23.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:23 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:23 smithi121 ceph-mon[35119]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:23 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:23 smithi117 ceph-mon[35081]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:23 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 -- 172.21.15.96:0/3851098082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4108730 msgr2=0x7f67c410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 --2- 172.21.15.96:0/3851098082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4108730 0x7f67c410abf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009ca0 tx=0x7f67bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 -- 172.21.15.96:0/3851098082 shutdown_connections 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 --2- 172.21.15.96:0/3851098082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4105b80 0x7f67c4107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 --2- 172.21.15.96:0/3851098082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4108730 0x7f67c410abf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 --2- 172.21.15.96:0/3851098082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c41055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 -- 172.21.15.96:0/3851098082 >> 172.21.15.96:0/3851098082 conn(0x7f67c40fd1e0 msgr2=0x7f67c40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 -- 172.21.15.96:0/3851098082 shutdown_connections 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 -- 172.21.15.96:0/3851098082 wait complete. 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.070+0000 7f67c942e640 1 Processor -- start 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 -- start start 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c4196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4108730 0x7f67c41a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410e680 con 0x7f67c4105b80 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410e500 con 0x7f67c4108730 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c942e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410e800 con 0x7f67c406b6e0 2024-09-10T22:44:24.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35688/0 (socket says 172.21.15.96:35688) 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c4196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c27fc640 1 -- 172.21.15.96:0/1485419185 learned_addr learned my addr 172.21.15.96:0/1485419185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c37fe640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4108730 0x7f67c41a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 -- 172.21.15.96:0/1485419185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4108730 msgr2=0x7f67c41a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4108730 0x7f67c41a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 -- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 msgr2=0x7f67c4196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c2ffd640 1 -- 172.21.15.96:0/1485419185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a6f40 con 0x7f67c406b6e0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.071+0000 7f67c27fc640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67c2ffd640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c4196900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d6d0 tx=0x7f67b400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4017070 con 0x7f67c406b6e0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a7230 con 0x7f67c406b6e0 2024-09-10T22:44:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4009820 con 0x7f67c406b6e0 2024-09-10T22:44:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4004c10 con 0x7f67c406b6e0 2024-09-10T22:44:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.072+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a7770 con 0x7f67c406b6e0 2024-09-10T22:44:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.073+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c410a220 con 0x7f67c406b6e0 2024-09-10T22:44:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.074+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f67b4005220 con 0x7f67c406b6e0 2024-09-10T22:44:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.075+0000 7f67abfff640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 0x7f67ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.075+0000 7f67c27fc640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 0x7f67ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.075+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7f67b40e5100 con 0x7f67c406b6e0 2024-09-10T22:44:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.075+0000 7f67c27fc640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 0x7f67ac0b77a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67c4198140 tx=0x7f67b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:24.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.079+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f67b40ac580 con 0x7f67c406b6e0 2024-09-10T22:44:24.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.261+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f67c402e760 con 0x7f67c406b6e0 2024-09-10T22:44:24.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.262+0000 7f67abfff640 1 -- 172.21.15.96:0/1485419185 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f67b40abe80 con 0x7f67c406b6e0 2024-09-10T22:44:24.263 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:24.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 msgr2=0x7f67ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 0x7f67ac0b77a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67c4198140 tx=0x7f67b800b410 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 msgr2=0x7f67c4196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c4196900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67b400d6d0 tx=0x7f67b400db90 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 shutdown_connections 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67c4108730 0x7f67c41a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67ac0b52e0 0x7f67ac0b77a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67c4105b80 0x7f67c4196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 --2- 172.21.15.96:0/1485419185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67c406b6e0 0x7f67c4196900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 >> 172.21.15.96:0/1485419185 conn(0x7f67c40fd1e0 msgr2=0x7f67c4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 shutdown_connections 2024-09-10T22:44:24.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:24.264+0000 7f67c942e640 1 -- 172.21.15.96:0/1485419185 wait complete. 2024-09-10T22:44:25.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:24 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1485419185' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:25.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:24 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1485419185' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:25.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:24 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1485419185' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:25.072 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726008242,"num_in_osds":12,"osd_in_since":1726008244,"num_remapped_pgs":0} 2024-09-10T22:44:25.704 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 22:44:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T22:44:25.536+0000 7f3f0a33d640 -1 osd.11 0 log_to_monitors true 2024-09-10T22:44:25.975 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:25 smithi121 ceph-mon[35119]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:25.975 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:25 smithi121 ceph-mon[35119]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:25.975 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:25 smithi121 ceph-mon[35119]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:25 smithi096 ceph-mon[30869]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:25 smithi096 ceph-mon[30869]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:25 smithi096 ceph-mon[30869]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:26.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:25 smithi117 ceph-mon[35081]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:26.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:25 smithi117 ceph-mon[35081]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:26.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:25 smithi117 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T22:44:26.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:44:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:26.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1201547435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 msgr2=0x7fbfd0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1201547435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd0102ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009930 tx=0x7fbfb802a420 comp rx=0 tx=0).stop 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1201547435 shutdown_connections 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1201547435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 0x7fbfd0109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1201547435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd0102ef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1201547435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 0x7fbfd0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1201547435 >> 172.21.15.96:0/1201547435 conn(0x7fbfd00fd1e0 msgr2=0x7fbfd00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1201547435 shutdown_connections 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1201547435 wait complete. 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.856+0000 7fbfd5aff640 1 Processor -- start 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 -- start start 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 0x7fbfd0198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 0x7fbfd019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010e6c0 con 0x7fbfd0102a70 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010e540 con 0x7fbfd0103430 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfd5aff640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010e840 con 0x7fbfd0101910 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50012/0 (socket says 172.21.15.96:50012) 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 -- 172.21.15.96:0/1001916001 learned_addr learned my addr 172.21.15.96:0/1001916001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfcf7fe640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 0x7fbfd0198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfcffff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 0x7fbfd019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 -- 172.21.15.96:0/1001916001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 msgr2=0x7fbfd0198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 0x7fbfd0198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 -- 172.21.15.96:0/1001916001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 msgr2=0x7fbfd019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 0x7fbfd019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 -- 172.21.15.96:0/1001916001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019a980 con 0x7fbfd0102a70 2024-09-10T22:44:26.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfceffd640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005ab0 tx=0x7fbfb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.857+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb803f030 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.858+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8003cb0 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.858+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8035070 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.858+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019ac10 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.858+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01a1180 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.859+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd01a13b0 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.860+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fbfb803bc20 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.861+0000 7fbfccff9640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 0x7fbfbc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.861+0000 7fbfcf7fe640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 0x7fbfbc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.861+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6847+0+0 (secure 0 0 0) 0x7fbfb8106230 con 0x7fbfd0102a70 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.861+0000 7fbfcf7fe640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 0x7fbfbc0b7820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005fd0 tx=0x7fbfc8009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:26.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:26.865+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbfb80ccfc0 con 0x7fbfd0102a70 2024-09-10T22:44:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:26 smithi096 ceph-mon[30869]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:44:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:26 smithi096 ceph-mon[30869]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:26 smithi096 ceph-mon[30869]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:44:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:26 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:26 smithi096 ceph-mon[30869]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-mon[35119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:44:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-mon[35119]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-mon[35119]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:44:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:27.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-mon[35119]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.014 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T22:44:26.713+0000 7f3f062c0640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T22:44:27.014 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 22:44:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T22:44:26.727+0000 7f3f020e7640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:44:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:26 smithi117 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T22:44:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:26 smithi117 ceph-mon[35081]: from='osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:26 smithi117 ceph-mon[35081]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T22:44:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:26 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:26 smithi117 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]} : dispatch 2024-09-10T22:44:27.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.050+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbfd01a1ae0 con 0x7fbfd0102a70 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.051+0000 7fbfccff9640 1 -- 172.21.15.96:0/1001916001 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7fbfb80d19f0 con 0x7fbfd0102a70 2024-09-10T22:44:27.052 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:27.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.052+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 msgr2=0x7fbfbc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.052+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 0x7fbfbc0b7820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8005fd0 tx=0x7fbfc8009680 comp rx=0 tx=0).stop 2024-09-10T22:44:27.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 msgr2=0x7fbfd01994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:27.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005ab0 tx=0x7fbfb80042f0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 shutdown_connections 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd0103430 0x7fbfd019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbfbc0b5360 0x7fbfbc0b7820 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd0102a70 0x7fbfd01994a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 --2- 172.21.15.96:0/1001916001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfd0101910 0x7fbfd0198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 >> 172.21.15.96:0/1001916001 conn(0x7fbfd00fd1e0 msgr2=0x7fbfd0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 shutdown_connections 2024-09-10T22:44:27.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:27.053+0000 7fbfd5aff640 1 -- 172.21.15.96:0/1001916001 wait complete. 2024-09-10T22:44:27.724 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726008242,"num_in_osds":12,"osd_in_since":1726008244,"num_remapped_pgs":0} 2024-09-10T22:44:28.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:44:28.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:44:28.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: Detected new or changed devices on smithi121 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1001916001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:28.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:27 smithi096 ceph-mon[30869]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:28.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: Detected new or changed devices on smithi121 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1001916001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:28.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:27 smithi121 ceph-mon[35119]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: Detected new or changed devices on smithi121 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:28.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:28.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1001916001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:28.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:27 smithi117 ceph-mon[35081]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T22:44:28.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T22:44:28.981 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:28 smithi096 ceph-mon[30869]: purged_snaps scrub starts 2024-09-10T22:44:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:28 smithi096 ceph-mon[30869]: purged_snaps scrub ok 2024-09-10T22:44:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:28 smithi096 ceph-mon[30869]: osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] boot 2024-09-10T22:44:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:28 smithi096 ceph-mon[30869]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:44:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:28 smithi121 ceph-mon[35119]: purged_snaps scrub starts 2024-09-10T22:44:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:28 smithi121 ceph-mon[35119]: purged_snaps scrub ok 2024-09-10T22:44:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:28 smithi121 ceph-mon[35119]: osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] boot 2024-09-10T22:44:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:28 smithi121 ceph-mon[35119]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:44:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:28 smithi117 ceph-mon[35081]: purged_snaps scrub starts 2024-09-10T22:44:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:28 smithi117 ceph-mon[35081]: purged_snaps scrub ok 2024-09-10T22:44:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:28 smithi117 ceph-mon[35081]: osd.11 [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] boot 2024-09-10T22:44:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:28 smithi117 ceph-mon[35081]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T22:44:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 -- 172.21.15.96:0/3142133327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0102ab0 msgr2=0x7f6bd0102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 --2- 172.21.15.96:0/3142133327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0102f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009cd0 tx=0x7f6bc402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 -- 172.21.15.96:0/3142133327 shutdown_connections 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 --2- 172.21.15.96:0/3142133327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0103470 0x7f6bd0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 --2- 172.21.15.96:0/3142133327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0102f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 --2- 172.21.15.96:0/3142133327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 0x7f6bd0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 -- 172.21.15.96:0/3142133327 >> 172.21.15.96:0/3142133327 conn(0x7f6bd00fd200 msgr2=0x7f6bd00ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 -- 172.21.15.96:0/3142133327 shutdown_connections 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.450+0000 7f6bd7870640 1 -- 172.21.15.96:0/3142133327 wait complete. 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 Processor -- start 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 -- start start 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 0x7f6bd0199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010d6b0 con 0x7f6bd0103470 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010d530 con 0x7f6bd0102ab0 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd7870640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010d830 con 0x7f6bd0101950 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50040/0 (socket says 172.21.15.96:50040) 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 -- 172.21.15.96:0/1274546291 learned_addr learned my addr 172.21.15.96:0/1274546291 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd560c640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 0x7f6bd0199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd4e0b640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 -- 172.21.15.96:0/1274546291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 msgr2=0x7f6bd0199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 0x7f6bd0199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 -- 172.21.15.96:0/1274546291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 msgr2=0x7f6bd0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.451+0000 7f6bd5e0d640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bd5e0d640 1 -- 172.21.15.96:0/1274546291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd019ab70 con 0x7f6bd0103470 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bd4e0b640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bd5e0d640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009fd0 tx=0x7f6bc000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0015020 con 0x7f6bd0103470 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc0011ce0 con 0x7f6bd0103470 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0010dc0 con 0x7f6bd0103470 2024-09-10T22:44:29.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd019ae60 con 0x7f6bd0103470 2024-09-10T22:44:29.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.452+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a1540 con 0x7f6bd0103470 2024-09-10T22:44:29.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.453+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd0101dd0 con 0x7f6bd0103470 2024-09-10T22:44:29.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.454+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f6bc0011830 con 0x7f6bd0103470 2024-09-10T22:44:29.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.455+0000 7f6bce7fc640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 0x7f6bb80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:29.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.455+0000 7f6bd560c640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 0x7f6bb80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:29.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.455+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f6bc00e5730 con 0x7f6bd0103470 2024-09-10T22:44:29.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.455+0000 7f6bd560c640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 0x7f6bb80b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009990 tx=0x7f6bc8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:29.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.458+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6bc001f020 con 0x7f6bd0103470 2024-09-10T22:44:29.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.633+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6bd01a1a40 con 0x7f6bd0103470 2024-09-10T22:44:29.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.633+0000 7f6bce7fc640 1 -- 172.21.15.96:0/1274546291 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f6bc00aca90 con 0x7f6bd0103470 2024-09-10T22:44:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:29.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 msgr2=0x7f6bb80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 0x7f6bb80b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009990 tx=0x7f6bc8009110 comp rx=0 tx=0).stop 2024-09-10T22:44:29.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 msgr2=0x7f6bd019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:29.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009fd0 tx=0x7f6bc000b820 comp rx=0 tx=0).stop 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 shutdown_connections 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bd0102ab0 0x7f6bd0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6bb80b52e0 0x7f6bb80b77a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bd0103470 0x7f6bd019a2d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 --2- 172.21.15.96:0/1274546291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bd0101950 0x7f6bd0199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 >> 172.21.15.96:0/1274546291 conn(0x7f6bd00fd200 msgr2=0x7f6bd00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 shutdown_connections 2024-09-10T22:44:29.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:29.636+0000 7f6bd7870640 1 -- 172.21.15.96:0/1274546291 wait complete. 2024-09-10T22:44:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:29 smithi096 ceph-mon[30869]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:44:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:29 smithi096 ceph-mon[30869]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:29 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1274546291' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:30.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:29 smithi121 ceph-mon[35119]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:44:30.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:29 smithi121 ceph-mon[35119]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:29 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1274546291' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:29 smithi117 ceph-mon[35081]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T22:44:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:29 smithi117 ceph-mon[35081]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:29 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1274546291' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T22:44:30.489 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726008267,"num_in_osds":12,"osd_in_since":1726008244,"num_remapped_pgs":0} 2024-09-10T22:44:30.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:44:30.733 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:31.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 -- 172.21.15.96:0/3480755207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 msgr2=0x7f36fc107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 --2- 172.21.15.96:0/3480755207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc107b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009900 tx=0x7f36f002a410 comp rx=0 tx=0).stop 2024-09-10T22:44:31.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 -- 172.21.15.96:0/3480755207 shutdown_connections 2024-09-10T22:44:31.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 --2- 172.21.15.96:0/3480755207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc108040 0x7f36fc10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 --2- 172.21.15.96:0/3480755207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc107b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 --2- 172.21.15.96:0/3480755207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc101cb0 0x7f36fc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 -- 172.21.15.96:0/3480755207 >> 172.21.15.96:0/3480755207 conn(0x7f36fc0fd740 msgr2=0x7f36fc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 -- 172.21.15.96:0/3480755207 shutdown_connections 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 -- 172.21.15.96:0/3480755207 wait complete. 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.168+0000 7f3703b90640 1 Processor -- start 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 -- start start 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc101cb0 0x7f36fc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 0x7f36fc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc111d00 con 0x7f36fc1076c0 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc111b80 con 0x7f36fc108040 2024-09-10T22:44:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f3703b90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc111e80 con 0x7f36fc101cb0 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50066/0 (socket says 172.21.15.96:50066) 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 -- 172.21.15.96:0/715530271 learned_addr learned my addr 172.21.15.96:0/715530271 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370192c640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc101cb0 0x7f36fc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370212d640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 0x7f36fc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 -- 172.21.15.96:0/715530271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc101cb0 msgr2=0x7f36fc197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc101cb0 0x7f36fc197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 -- 172.21.15.96:0/715530271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 msgr2=0x7f36fc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 0x7f36fc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 -- 172.21.15.96:0/715530271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc1a76a0 con 0x7f36fc1076c0 2024-09-10T22:44:31.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370212d640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 0x7f36fc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.169+0000 7f370112b640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36f0006010 tx=0x7f36f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.170+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f003f030 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.170+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f0003800 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.170+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc1a7930 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.170+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0035070 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.170+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc1a7ee0 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.171+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc102090 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.172+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f36f0003cb0 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.173+0000 7f36faffd640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 0x7f36e40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.173+0000 7f370192c640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 0x7f36e40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.173+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f36f0106210 con 0x7f36fc1076c0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.174+0000 7f370192c640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 0x7f36e40b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009990 tx=0x7f36f4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:31.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.176+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f36f00cd550 con 0x7f36fc1076c0 2024-09-10T22:44:31.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.353+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36fc02e5c0 con 0x7f36fc1076c0 2024-09-10T22:44:31.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.354+0000 7f36faffd640 1 -- 172.21.15.96:0/715530271 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18480 (secure 0 0 0) 0x7f36f00cce50 con 0x7f36fc1076c0 2024-09-10T22:44:31.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:31.355 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":70,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:44:28.928145+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:44:31.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 msgr2=0x7f36e40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 0x7f36e40b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009990 tx=0x7f36f4009110 comp rx=0 tx=0).stop 2024-09-10T22:44:31.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 msgr2=0x7f36fc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:31.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36f0006010 tx=0x7f36f00042f0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 shutdown_connections 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36fc108040 0x7f36fc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f36e40b5360 0x7f36e40b7820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36fc1076c0 0x7f36fc198400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 --2- 172.21.15.96:0/715530271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36fc101cb0 0x7f36fc197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 >> 172.21.15.96:0/715530271 conn(0x7f36fc0fd740 msgr2=0x7f36fc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 shutdown_connections 2024-09-10T22:44:31.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:31.356+0000 7f3703b90640 1 -- 172.21.15.96:0/715530271 wait complete. 2024-09-10T22:44:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:31 smithi096 ceph-mon[30869]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:31 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/715530271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:44:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:31 smithi121 ceph-mon[35119]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:31.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:31 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/715530271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:44:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:31 smithi117 ceph-mon[35081]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:31 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/715530271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:44:32.160 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:40:37.540323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:44:32.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T22:44:32.405 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:32.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1769924800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 msgr2=0x7f62e410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:32.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1769924800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e410f640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009ca0 tx=0x7f62dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1769924800 shutdown_connections 2024-09-10T22:44:32.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1769924800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e410f640 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1769924800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e410a920 0x7f62e410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1769924800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e4101810 0x7f62e4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1769924800 >> 172.21.15.96:0/1769924800 conn(0x7f62e4076f50 msgr2=0x7f62e4077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1769924800 shutdown_connections 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1769924800 wait complete. 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.868+0000 7f62eb3bf640 1 Processor -- start 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 -- start start 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4101810 0x7f62e4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 0x7f62e41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112e30 con 0x7f62e410d250 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112cb0 con 0x7f62e410a920 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112fb0 con 0x7f62e4101810 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:32.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50088/0 (socket says 172.21.15.96:50088) 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 -- 172.21.15.96:0/1826245621 learned_addr learned my addr 172.21.15.96:0/1826245621 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e895a640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 0x7f62e41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e915b640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4101810 0x7f62e4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 -- 172.21.15.96:0/1826245621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4101810 msgr2=0x7f62e4199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4101810 0x7f62e4199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 -- 172.21.15.96:0/1826245621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 msgr2=0x7f62e41998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 0x7f62e41998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 -- 172.21.15.96:0/1826245621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e419f430 con 0x7f62e410d250 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e895a640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 0x7f62e41998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62e995c640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005cc0 tx=0x7f62dc02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc003cf0 con 0x7f62e410d250 2024-09-10T22:44:32.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62dc0044f0 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02fd10 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.869+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e419f6c0 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.870+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a5e60 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.870+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4101c70 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.873+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f62dc004040 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.873+0000 7f62da7fc640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 0x7f62d00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.873+0000 7f62e915b640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 0x7f62d00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.874+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f62dc106ee0 con 0x7f62e410d250 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.874+0000 7f62e915b640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 0x7f62d00b7820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009890 tx=0x7f62e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:32.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:32.876+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f62dc0ce240 con 0x7f62e410d250 2024-09-10T22:44:33.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.050+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62e41a6160 con 0x7f62e410d250 2024-09-10T22:44:33.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.051+0000 7f62da7fc640 1 -- 172.21.15.96:0/1826245621 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f62dc0cdb40 con 0x7f62e410d250 2024-09-10T22:44:33.053 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-10T22:44:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 msgr2=0x7f62d00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 0x7f62d00b7820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009890 tx=0x7f62e0008040 comp rx=0 tx=0).stop 2024-09-10T22:44:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 msgr2=0x7f62e419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005cc0 tx=0x7f62dc02f470 comp rx=0 tx=0).stop 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 shutdown_connections 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e410a920 0x7f62e41998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62d00b5360 0x7f62d00b7820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62e410d250 0x7f62e419ec60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 --2- 172.21.15.96:0/1826245621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e4101810 0x7f62e4199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 >> 172.21.15.96:0/1826245621 conn(0x7f62e4076f50 msgr2=0x7f62e41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 shutdown_connections 2024-09-10T22:44:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:33.053+0000 7f62eb3bf640 1 -- 172.21.15.96:0/1826245621 wait complete. 2024-09-10T22:44:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:33 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1826245621' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:44:33.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:33 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1826245621' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:44:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:33 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1826245621' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.a on smithi096 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.d on smithi096 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.g on smithi096 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.j on smithi096 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.b on smithi117 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.e on smithi117 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.h on smithi117 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.k on smithi117 2024-09-10T22:44:33.824 INFO:tasks.cephadm:Adding mds.c on smithi121 2024-09-10T22:44:33.825 INFO:tasks.cephadm:Adding mds.f on smithi121 2024-09-10T22:44:33.825 INFO:tasks.cephadm:Adding mds.i on smithi121 2024-09-10T22:44:33.825 INFO:tasks.cephadm:Adding mds.l on smithi121 2024-09-10T22:44:33.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l' 2024-09-10T22:44:34.084 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:44:34.368 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:34 smithi121 ceph-mon[35119]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:34 smithi096 ceph-mon[30869]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:34 smithi117 ceph-mon[35081]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:34.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 -- 172.21.15.121:0/1749273176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84103b80 msgr2=0x7fdb84104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1749273176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84103b80 0x7fdb84104000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009960 tx=0x7fdb7802a460 comp rx=0 tx=0).stop 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 -- 172.21.15.121:0/1749273176 shutdown_connections 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1749273176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84104540 0x7fdb8410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1749273176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84103b80 0x7fdb84104000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1749273176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 0x7fdb84102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 -- 172.21.15.121:0/1749273176 >> 172.21.15.121:0/1749273176 conn(0x7fdb840fe2f0 msgr2=0x7fdb84100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 -- 172.21.15.121:0/1749273176 shutdown_connections 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.572+0000 7fdb8a023640 1 -- 172.21.15.121:0/1749273176 wait complete. 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.573+0000 7fdb8a023640 1 Processor -- start 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.573+0000 7fdb8a023640 1 -- start start 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 0x7fdb84198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84103b80 0x7fdb84199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:34.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410e680 con 0x7fdb84102a20 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410e500 con 0x7fdb84103b80 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb8a023640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410e800 con 0x7fdb84104540 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59678/0 (socket says 172.21.15.121:59678) 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 -- 172.21.15.121:0/1007968300 learned_addr learned my addr 172.21.15.121:0/1007968300 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb837fe640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 0x7fdb84198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb82ffd640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84103b80 0x7fdb84199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 -- 172.21.15.121:0/1007968300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84103b80 msgr2=0x7fdb84199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84103b80 0x7fdb84199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 -- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 msgr2=0x7fdb84198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:34.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 0x7fdb84198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 -- 172.21.15.121:0/1007968300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8419a8e0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb83fff640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00cad0 tx=0x7fdb7c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00ae20 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.574+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c005020 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.575+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00f6b0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.575+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8419abd0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.575+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a12b0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.576+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84109da0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.577+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fdb7c0054f0 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.578+0000 7fdb80ff9640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 0x7fdb6c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.578+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fdb7c0e5b90 con 0x7fdb84104540 2024-09-10T22:44:34.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.578+0000 7fdb837fe640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 0x7fdb6c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:34.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.578+0000 7fdb837fe640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 0x7fdb6c0b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005e10 tx=0x7fdb74005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:34.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.581+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb7c014030 con 0x7fdb84104540 2024-09-10T22:44:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.767+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}) -- 0x7fdb84063690 con 0x7fdb6c0b5360 2024-09-10T22:44:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.776+0000 7fdb80ff9640 1 -- 172.21.15.121:0/1007968300 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdb84063690 con 0x7fdb6c0b5360 2024-09-10T22:44:34.777 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mds.all update... 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 msgr2=0x7fdb6c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 0x7fdb6c0b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005e10 tx=0x7fdb74005da0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 msgr2=0x7fdb8419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00cad0 tx=0x7fdb7c00cf90 comp rx=0 tx=0).stop 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 shutdown_connections 2024-09-10T22:44:34.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb84103b80 0x7fdb84199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb6c0b5360 0x7fdb6c0b7820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb84104540 0x7fdb8419a040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 --2- 172.21.15.121:0/1007968300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb84102a20 0x7fdb84198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 >> 172.21.15.121:0/1007968300 conn(0x7fdb840fe2f0 msgr2=0x7fdb84106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 shutdown_connections 2024-09-10T22:44:34.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T22:44:34.778+0000 7fdb8a023640 1 -- 172.21.15.121:0/1007968300 wait complete. 2024-09-10T22:44:35.515 DEBUG:teuthology.orchestra.run.smithi096:mds.a> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T22:44:35.518 DEBUG:teuthology.orchestra.run.smithi096:mds.d> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T22:44:35.561 DEBUG:teuthology.orchestra.run.smithi096:mds.g> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T22:44:35.604 DEBUG:teuthology.orchestra.run.smithi096:mds.j> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T22:44:35.647 DEBUG:teuthology.orchestra.run.smithi117:mds.b> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T22:44:35.650 DEBUG:teuthology.orchestra.run.smithi117:mds.e> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T22:44:35.693 DEBUG:teuthology.orchestra.run.smithi117:mds.h> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T22:44:35.736 DEBUG:teuthology.orchestra.run.smithi117:mds.k> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T22:44:35.780 DEBUG:teuthology.orchestra.run.smithi121:mds.c> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T22:44:35.782 DEBUG:teuthology.orchestra.run.smithi121:mds.f> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T22:44:35.825 DEBUG:teuthology.orchestra.run.smithi121:mds.i> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T22:44:35.868 DEBUG:teuthology.orchestra.run.smithi121:mds.l> sudo journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T22:44:35.911 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T22:44:35.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:35.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: Deploying daemon mds.l on smithi121 2024-09-10T22:44:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:35 smithi096 ceph-mon[30869]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2024-09-10T22:44:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: Deploying daemon mds.l on smithi121 2024-09-10T22:44:36.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:35 smithi117 ceph-mon[35081]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:36.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:44:36.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: Deploying daemon mds.l on smithi121 2024-09-10T22:44:36.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:35 smithi121 ceph-mon[35119]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:36.171 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:36.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 -- 172.21.15.96:0/3957700740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 msgr2=0x7f6a20108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:36.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 --2- 172.21.15.96:0/3957700740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20108d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009930 tx=0x7f6a1402a420 comp rx=0 tx=0).stop 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 -- 172.21.15.96:0/3957700740 shutdown_connections 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 --2- 172.21.15.96:0/3957700740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 0x7f6a2010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 --2- 172.21.15.96:0/3957700740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20108d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 --2- 172.21.15.96:0/3957700740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 0x7f6a20101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 -- 172.21.15.96:0/3957700740 >> 172.21.15.96:0/3957700740 conn(0x7f6a20076f20 msgr2=0x7f6a20077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 -- 172.21.15.96:0/3957700740 shutdown_connections 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.617+0000 7f6a259b8640 1 -- 172.21.15.96:0/3957700740 wait complete. 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 Processor -- start 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 -- start start 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 0x7f6a20199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:36.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 0x7f6a2019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20112dd0 con 0x7f6a20108910 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20112c50 con 0x7f6a20109290 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a259b8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20112f50 con 0x7f6a20101810 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38392/0 (socket says 172.21.15.96:38392) 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 -- 172.21.15.96:0/1095182677 learned_addr learned my addr 172.21.15.96:0/1095182677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1effd640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 0x7f6a20199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1f7fe640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 0x7f6a2019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 -- 172.21.15.96:0/1095182677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 msgr2=0x7f6a20199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 0x7f6a20199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 -- 172.21.15.96:0/1095182677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 msgr2=0x7f6a2019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.618+0000 7f6a1e7fc640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 0x7f6a2019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a1e7fc640 1 -- 172.21.15.96:0/1095182677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2019f3b0 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a1e7fc640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a14004770 tx=0x7f6a140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1403f070 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a1403b460 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2019f640 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14035070 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.619+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a5d60 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.620+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20101bf0 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.621+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f6a1404b060 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.622+0000 7f6a249b6640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 0x7f6a080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.622+0000 7f6a1effd640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 0x7f6a080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.622+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f6a14105af0 con 0x7f6a20108910 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.622+0000 7f6a1effd640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 0x7f6a080b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005fd0 tx=0x7f6a1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:36.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.625+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6a140cce50 con 0x7f6a20108910 2024-09-10T22:44:36.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.833+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2019a430 con 0x7f6a20108910 2024-09-10T22:44:36.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.834+0000 7f6a249b6640 1 -- 172.21.15.96:0/1095182677 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f6a140cc750 con 0x7f6a20108910 2024-09-10T22:44:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:36.837 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"btime":"2024-09-10T22:37:12:788415+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T22:44:36.837 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 1 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 msgr2=0x7f6a080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 0x7f6a080b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005fd0 tx=0x7f6a1000b390 comp rx=0 tx=0).stop 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 msgr2=0x7f6a20199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a14004770 tx=0x7f6a140042f0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 shutdown_connections 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a20109290 0x7f6a2019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6a080b52e0 0x7f6a080b77a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a20108910 0x7f6a20199840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 --2- 172.21.15.96:0/1095182677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a20101810 0x7f6a20199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:36.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.836+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 >> 172.21.15.96:0/1095182677 conn(0x7f6a20076f20 msgr2=0x7f6a20100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:36.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.837+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 shutdown_connections 2024-09-10T22:44:36.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:36.837+0000 7f6a259b8640 1 -- 172.21.15.96:0/1095182677 wait complete. 2024-09-10T22:44:37.418 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:37 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1095182677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:37.419 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 systemd[1]: Starting Ceph mds.l for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:37.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:37 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1095182677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:37.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:37 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1095182677' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:37.675 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 podman[64947]: 2024-09-10 22:44:37.319718116 +0000 UTC m=+0.015884308 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:37.675 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 podman[64947]: 2024-09-10 22:44:37.438551081 +0000 UTC m=+0.134717274 container create 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T22:44:37.675 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 podman[64947]: 2024-09-10 22:44:37.674586529 +0000 UTC m=+0.370752722 container init 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T22:44:38.013 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 podman[64947]: 2024-09-10 22:44:37.677430272 +0000 UTC m=+0.373596465 container start 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T22:44:38.013 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 ceph-mds[64966]: -- 172.21.15.121:0/284789884 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591b67a1860 con 0x5591b677f400 2024-09-10T22:44:38.014 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: starting mds.l at 2024-09-10T22:44:38.014 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 bash[64947]: 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 2024-09-10T22:44:38.014 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 22:44:37 smithi121 systemd[1]: Started Ceph mds.l for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:38.475 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6832/2699941517,v1:172.21.15.121:6833/2699941517] up:boot 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: fsmap 1 up:standby 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:38.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:38 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6832/2699941517,v1:172.21.15.121:6833/2699941517] up:boot 2024-09-10T22:44:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: fsmap 1 up:standby 2024-09-10T22:44:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:38 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:38.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:38.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6832/2699941517,v1:172.21.15.121:6833/2699941517] up:boot 2024-09-10T22:44:38.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: fsmap 1 up:standby 2024-09-10T22:44:38.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:38.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:38 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:39.464 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:39 smithi117 ceph-mon[35081]: Deploying daemon mds.k on smithi117 2024-09-10T22:44:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:39 smithi096 ceph-mon[30869]: Deploying daemon mds.k on smithi117 2024-09-10T22:44:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:39 smithi121 ceph-mon[35119]: Deploying daemon mds.k on smithi117 2024-09-10T22:44:39.702 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:39.959 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:40.100 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 podman[64053]: 2024-09-10 22:44:39.937882962 +0000 UTC m=+0.015995132 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:40.100 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 podman[64053]: 2024-09-10 22:44:40.098750162 +0000 UTC m=+0.176862334 container create 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T22:44:40.352 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:40 smithi117 ceph-mon[35081]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:40.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:40 smithi096 ceph-mon[30869]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 -- 172.21.15.96:0/785380276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198109150 msgr2=0x7fd19810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 --2- 172.21.15.96:0/785380276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198109150 0x7fd19810f950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd190009ca0 tx=0x7fd19002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 -- 172.21.15.96:0/785380276 shutdown_connections 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 --2- 172.21.15.96:0/785380276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198109150 0x7fd19810f950 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 --2- 172.21.15.96:0/785380276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1981087d0 0x7fd198108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 --2- 172.21.15.96:0/785380276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd198102dc0 0x7fd1981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 -- 172.21.15.96:0/785380276 >> 172.21.15.96:0/785380276 conn(0x7fd1980fe830 msgr2=0x7fd198100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 -- 172.21.15.96:0/785380276 shutdown_connections 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.443+0000 7fd19d40a640 1 -- 172.21.15.96:0/785380276 wait complete. 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 Processor -- start 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 -- start start 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1981087d0 0x7fd198199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd198109150 0x7fd19819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112e10 con 0x7fd198102dc0 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112c90 con 0x7fd1981087d0 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd19d40a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112f90 con 0x7fd198109150 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd196ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd196ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38410/0 (socket says 172.21.15.96:38410) 2024-09-10T22:44:40.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd196ffd640 1 -- 172.21.15.96:0/1363339907 learned_addr learned my addr 172.21.15.96:0/1363339907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.444+0000 7fd1967fc640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1981087d0 0x7fd198199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd1977fe640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd198109150 0x7fd19819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 -- 172.21.15.96:0/1363339907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd198109150 msgr2=0x7fd19819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd198109150 0x7fd19819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 -- 172.21.15.96:0/1363339907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1981087d0 msgr2=0x7fd198199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1981087d0 0x7fd198199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 -- 172.21.15.96:0/1363339907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19819f370 con 0x7fd198102dc0 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd196ffd640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d950 tx=0x7fd18800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188017070 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19819f660 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18800de80 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18800f620 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.445+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a5da0 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.446+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1981031a0 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.447+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fd188005030 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.448+0000 7fd17ffff640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 0x7fd1800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.448+0000 7fd1967fc640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 0x7fd1800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.448+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fd1880e5340 con 0x7fd198102dc0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.449+0000 7fd1967fc640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 0x7fd1800b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd198107230 tx=0x7fd18c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.451+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd1880ac6a0 con 0x7fd198102dc0 2024-09-10T22:44:40.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:40 smithi121 ceph-mon[35119]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:40.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.673+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd19802e5c0 con 0x7fd198102dc0 2024-09-10T22:44:40.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.674+0000 7fd17ffff640 1 -- 172.21.15.96:0/1363339907 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fd1880abfa0 con 0x7fd198102dc0 2024-09-10T22:44:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:40.676 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"btime":"2024-09-10T22:44:37:705169+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:44:40.676 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-09-10T22:44:40.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.676+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 msgr2=0x7fd1800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:40.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.676+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 0x7fd1800b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd198107230 tx=0x7fd18c009110 comp rx=0 tx=0).stop 2024-09-10T22:44:40.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.676+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 msgr2=0x7fd1981992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.676+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d950 tx=0x7fd18800dd50 comp rx=0 tx=0).stop 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 shutdown_connections 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1981087d0 0x7fd198199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1800b52e0 0x7fd1800b77a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd198109150 0x7fd19819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 --2- 172.21.15.96:0/1363339907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd198102dc0 0x7fd1981992c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 >> 172.21.15.96:0/1363339907 conn(0x7fd1980fe830 msgr2=0x7fd198100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 shutdown_connections 2024-09-10T22:44:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:40.677+0000 7fd19d40a640 1 -- 172.21.15.96:0/1363339907 wait complete. 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 podman[64053]: 2024-09-10 22:44:40.351425353 +0000 UTC m=+0.429537525 container init 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 podman[64053]: 2024-09-10 22:44:40.355714074 +0000 UTC m=+0.433826251 container start 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 ceph-mds[64074]: -- 172.21.15.117:0/1431389785 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d320c47860 con 0x55d320c24800 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k[64069]: starting mds.k at 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 bash[64053]: 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 2024-09-10T22:44:40.686 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 22:44:40 smithi117 systemd[1]: Started Ceph mds.k for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:41.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:41.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: Deploying daemon mds.j on smithi096 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1363339907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6832/317768324,v1:172.21.15.117:6833/317768324] up:boot 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: fsmap 2 up:standby 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:44:41.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:41 smithi117 ceph-mon[35081]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: Deploying daemon mds.j on smithi096 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1363339907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6832/317768324,v1:172.21.15.117:6833/317768324] up:boot 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: fsmap 2 up:standby 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:44:41.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:41 smithi096 ceph-mon[30869]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:42.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:42.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:42.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:42.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:42.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: Deploying daemon mds.j on smithi096 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1363339907' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6832/317768324,v1:172.21.15.117:6833/317768324] up:boot 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: fsmap 2 up:standby 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:44:42.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:41 smithi121 ceph-mon[35119]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:43.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:43.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:43 smithi096 ceph-mon[30869]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:43 smithi121 ceph-mon[35119]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:43 smithi117 ceph-mon[35081]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:44.047 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 22:44:43 smithi096 systemd[1]: Started Ceph mds.j for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 -- 172.21.15.96:0/3516251462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 msgr2=0x7f735810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 --2- 172.21.15.96:0/3516251462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735810f950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7350009ca0 tx=0x7f735002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 -- 172.21.15.96:0/3516251462 shutdown_connections 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 --2- 172.21.15.96:0/3516251462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73581087d0 0x7f7358108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 --2- 172.21.15.96:0/3516251462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735810f950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 --2- 172.21.15.96:0/3516251462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7358102dc0 0x7f73581031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 -- 172.21.15.96:0/3516251462 >> 172.21.15.96:0/3516251462 conn(0x7f73580fe830 msgr2=0x7f7358100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 -- 172.21.15.96:0/3516251462 shutdown_connections 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.421+0000 7f73604be640 1 -- 172.21.15.96:0/3516251462 wait complete. 2024-09-10T22:44:44.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 Processor -- start 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 -- start start 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358102dc0 0x7f73581991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73581087d0 0x7f7358199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112e10 con 0x7f7358109150 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112c90 con 0x7f7358102dc0 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f73604be640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7358112f90 con 0x7f73581087d0 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f735ea5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f735ea5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38452/0 (socket says 172.21.15.96:38452) 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f735ea5b640 1 -- 172.21.15.96:0/1805701639 learned_addr learned my addr 172.21.15.96:0/1805701639 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.422+0000 7f735da59640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73581087d0 0x7f7358199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735e25a640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358102dc0 0x7f73581991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 -- 172.21.15.96:0/1805701639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73581087d0 msgr2=0x7f7358199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:44.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73581087d0 0x7f7358199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 -- 172.21.15.96:0/1805701639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358102dc0 msgr2=0x7f73581991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358102dc0 0x7f73581991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 -- 172.21.15.96:0/1805701639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735819f290 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f735ea5b640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7350005cc0 tx=0x7f735002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350003bf0 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735819f520 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7350003d90 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735002fb20 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.423+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73581a5c40 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.424+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73581031a0 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.426+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f7350004580 con 0x7f7358109150 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.426+0000 7f734f7fe640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 0x7f73440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:44.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.426+0000 7f735e25a640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 0x7f73440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:44.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.426+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f7350106c20 con 0x7f7358109150 2024-09-10T22:44:44.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.427+0000 7f735e25a640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 0x7f73440b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7354005fd0 tx=0x7f7354005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:44.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.429+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f73500cdf80 con 0x7f7358109150 2024-09-10T22:44:44.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.648+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f735819a310 con 0x7f7358109150 2024-09-10T22:44:44.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.648+0000 7f734f7fe640 1 -- 172.21.15.96:0/1805701639 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f73500098b0 con 0x7f7358109150 2024-09-10T22:44:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:44.650 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"btime":"2024-09-10T22:44:44:374122+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:44:44.651 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 4 2024-09-10T22:44:44.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 msgr2=0x7f73440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:44.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 0x7f73440b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7354005fd0 tx=0x7f7354005850 comp rx=0 tx=0).stop 2024-09-10T22:44:44.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 msgr2=0x7f735819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:44.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7350005cc0 tx=0x7f735002f380 comp rx=0 tx=0).stop 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 shutdown_connections 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73440b52e0 0x7f73440b77a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7358109150 0x7f735819eac0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73581087d0 0x7f7358199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 --2- 172.21.15.96:0/1805701639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7358102dc0 0x7f73581991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 >> 172.21.15.96:0/1805701639 conn(0x7f73580fe830 msgr2=0x7f7358101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 shutdown_connections 2024-09-10T22:44:44.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:44.651+0000 7f73604be640 1 -- 172.21.15.96:0/1805701639 wait complete. 2024-09-10T22:44:45.163 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.163 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.163 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.163 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: Deploying daemon mds.d on smithi096 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6834/3071875945,v1:172.21.15.96:6835/3071875945] up:boot 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: fsmap 3 up:standby 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:44:45.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:44 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1805701639' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:45.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: Deploying daemon mds.d on smithi096 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6834/3071875945,v1:172.21.15.96:6835/3071875945] up:boot 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: fsmap 3 up:standby 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:44:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:44 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1805701639' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: Deploying daemon mds.d on smithi096 2024-09-10T22:44:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6834/3071875945,v1:172.21.15.96:6835/3071875945] up:boot 2024-09-10T22:44:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: fsmap 3 up:standby 2024-09-10T22:44:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T22:44:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:44 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1805701639' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:46.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:45 smithi096 ceph-mon[30869]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:45 smithi121 ceph-mon[35119]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:45 smithi117 ceph-mon[35081]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:47.000 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:46 smithi096 systemd[1]: Starting Ceph mds.d for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:47.356 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 podman[72282]: 2024-09-10 22:44:46.978559405 +0000 UTC m=+0.015944751 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:47.356 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 podman[72282]: 2024-09-10 22:44:47.110749637 +0000 UTC m=+0.148134974 container create 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:44:47.356 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 podman[72282]: 2024-09-10 22:44:47.355114218 +0000 UTC m=+0.392499558 container init 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T22:44:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:47 smithi096 ceph-mon[30869]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:47.750 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 podman[72282]: 2024-09-10 22:44:47.358060552 +0000 UTC m=+0.395445899 container start 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T22:44:47.750 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 ceph-mds[72302]: -- 172.21.15.96:0/1395664270 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564049c3f860 con 0x564049c1c800 2024-09-10T22:44:47.750 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: starting mds.d at 2024-09-10T22:44:47.751 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 bash[72282]: 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f 2024-09-10T22:44:47.751 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 22:44:47 smithi096 systemd[1]: Started Ceph mds.d for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:47 smithi121 ceph-mon[35119]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:47 smithi117 ceph-mon[35081]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:47.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:48.099 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 -- 172.21.15.96:0/3376371349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c102dc0 msgr2=0x7f7e3c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:48.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/3376371349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c1031a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009900 tx=0x7f7e2c02a410 comp rx=0 tx=0).stop 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 -- 172.21.15.96:0/3376371349 shutdown_connections 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/3376371349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 0x7f7e3c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/3376371349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c109150 0x7f7e3c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/3376371349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c1031a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.565+0000 7f7e41f0c640 1 -- 172.21.15.96:0/3376371349 >> 172.21.15.96:0/3376371349 conn(0x7f7e3c0fe830 msgr2=0x7f7e3c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- 172.21.15.96:0/3376371349 shutdown_connections 2024-09-10T22:44:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- 172.21.15.96:0/3376371349 wait complete. 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 Processor -- start 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- start start 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 0x7f7e3c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112e10 con 0x7f7e3c109150 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112c90 con 0x7f7e3c1087d0 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.566+0000 7f7e41f0c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112f90 con 0x7f7e3c102dc0 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51078/0 (socket says 172.21.15.96:51078) 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 -- 172.21.15.96:0/2075288265 learned_addr learned my addr 172.21.15.96:0/2075288265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e33fff640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 0x7f7e3c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3b7fe640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 -- 172.21.15.96:0/2075288265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c102dc0 msgr2=0x7f7e3c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 -- 172.21.15.96:0/2075288265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 msgr2=0x7f7e3c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 0x7f7e3c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 -- 172.21.15.96:0/2075288265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c1a15d0 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e3bfff640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b970 tx=0x7f7e3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34019070 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e34014440 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c1a18c0 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340075d0 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.567+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a8000 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.568+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c1031a0 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.569+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f7e3400cb40 con 0x7f7e3c109150 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.570+0000 7f7e397fa640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 0x7f7e280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:48.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.570+0000 7f7e3b7fe640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 0x7f7e280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.570+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f7e340e4eb0 con 0x7f7e3c109150 2024-09-10T22:44:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.571+0000 7f7e3b7fe640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 0x7f7e280b77a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c031440 tx=0x7f7e2c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.574+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7e340abbc0 con 0x7f7e3c109150 2024-09-10T22:44:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: Deploying daemon mds.e on smithi117 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6836/3503446146,v1:172.21.15.96:6837/3503446146] up:boot 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: fsmap 4 up:standby 2024-09-10T22:44:48.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:48 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:44:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: Deploying daemon mds.e on smithi117 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6836/3503446146,v1:172.21.15.96:6837/3503446146] up:boot 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: fsmap 4 up:standby 2024-09-10T22:44:48.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:48 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:44:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: Deploying daemon mds.e on smithi117 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6836/3503446146,v1:172.21.15.96:6837/3503446146] up:boot 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: fsmap 4 up:standby 2024-09-10T22:44:48.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:48 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T22:44:48.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.786+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e3c1a82f0 con 0x7f7e3c109150 2024-09-10T22:44:48.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.787+0000 7f7e397fa640 1 -- 172.21.15.96:0/2075288265 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f7e340b05f0 con 0x7f7e3c109150 2024-09-10T22:44:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:48.788 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"btime":"2024-09-10T22:44:48:374780+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:44:48.789 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 5 2024-09-10T22:44:48.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 msgr2=0x7f7e280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:48.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 0x7f7e280b77a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c031440 tx=0x7f7e2c005b50 comp rx=0 tx=0).stop 2024-09-10T22:44:48.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 msgr2=0x7f7e3c1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b970 tx=0x7f7e3400be30 comp rx=0 tx=0).stop 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 shutdown_connections 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e3c1087d0 0x7f7e3c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7e280b52e0 0x7f7e280b77a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e3c109150 0x7f7e3c1a0e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 --2- 172.21.15.96:0/2075288265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e3c102dc0 0x7f7e3c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 >> 172.21.15.96:0/2075288265 conn(0x7f7e3c0fe830 msgr2=0x7f7e3c0753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 shutdown_connections 2024-09-10T22:44:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:48.789+0000 7f7e41f0c640 1 -- 172.21.15.96:0/2075288265 wait complete. 2024-09-10T22:44:49.743 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:49 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2075288265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:49.743 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:49 smithi117 ceph-mon[35081]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:49.744 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:49 smithi117 systemd[1]: Starting Ceph mds.e for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:49 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2075288265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:49 smithi121 ceph-mon[35119]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:50.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2075288265' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:50.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:49 smithi096 ceph-mon[30869]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:50.015 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:49 smithi117 podman[64365]: 2024-09-10 22:44:49.644564022 +0000 UTC m=+0.015897301 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:50.016 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:49 smithi117 podman[64365]: 2024-09-10 22:44:49.762165402 +0000 UTC m=+0.133498670 container create ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T22:44:50.016 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 podman[64365]: 2024-09-10 22:44:50.015078379 +0000 UTC m=+0.386411647 container init ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:44:50.276 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 podman[64365]: 2024-09-10 22:44:50.01796905 +0000 UTC m=+0.389302325 container start ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:44:50.276 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 ceph-mds[64384]: -- 172.21.15.117:0/2145076730 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a0b93d860 con 0x557a0b91b400 2024-09-10T22:44:50.276 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: starting mds.e at 2024-09-10T22:44:50.276 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 bash[64365]: ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd 2024-09-10T22:44:50.277 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 22:44:50 smithi117 systemd[1]: Started Ceph mds.e for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:51.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: Deploying daemon mds.f on smithi121 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:boot 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: fsmap 5 up:standby 2024-09-10T22:44:51.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:51 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:44:51.469 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:51.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: Deploying daemon mds.f on smithi121 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:boot 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: fsmap 5 up:standby 2024-09-10T22:44:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:51 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: Deploying daemon mds.f on smithi121 2024-09-10T22:44:51.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:boot 2024-09-10T22:44:51.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: fsmap 5 up:standby 2024-09-10T22:44:51.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:51 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:44:51.731 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:52.171 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 systemd[1]: Starting Ceph mds.f for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:52.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 -- 172.21.15.96:0/489015154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad8102dc0 msgr2=0x7f4ad81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:52.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/489015154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad8102dc0 0x7f4ad81031a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad402a410 comp rx=0 tx=0).stop 2024-09-10T22:44:52.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 -- 172.21.15.96:0/489015154 shutdown_connections 2024-09-10T22:44:52.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/489015154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 0x7f4ad810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/489015154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad81087d0 0x7f4ad8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/489015154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad8102dc0 0x7f4ad81031a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 -- 172.21.15.96:0/489015154 >> 172.21.15.96:0/489015154 conn(0x7f4ad80fe830 msgr2=0x7f4ad8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 -- 172.21.15.96:0/489015154 shutdown_connections 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 -- 172.21.15.96:0/489015154 wait complete. 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.188+0000 7f4adfdd7640 1 Processor -- start 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 -- start start 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad8102dc0 0x7f4ad8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 0x7f4ad819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8112e10 con 0x7f4ad81087d0 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8112c90 con 0x7f4ad8109150 2024-09-10T22:44:52.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8112f90 con 0x7f4ad8102dc0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51106/0 (socket says 172.21.15.96:51106) 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 -- 172.21.15.96:0/897539252 learned_addr learned my addr 172.21.15.96:0/897539252 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4addb73640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad8102dc0 0x7f4ad8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4ade374640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 0x7f4ad819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 -- 172.21.15.96:0/897539252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad8102dc0 msgr2=0x7f4ad8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad8102dc0 0x7f4ad8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 -- 172.21.15.96:0/897539252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 msgr2=0x7f4ad819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 0x7f4ad819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 -- 172.21.15.96:0/897539252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad819f3e0 con 0x7f4ad81087d0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4add372640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d770 tx=0x7f4ac800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4ad81087d0 2024-09-10T22:44:52.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.189+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad819f6d0 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.190+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac80054d0 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.190+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004c30 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.190+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad81a5e10 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.191+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad81031a0 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.192+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f4ac8004e10 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.192+0000 7f4aceffd640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 0x7f4ac40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.192+0000 7f4addb73640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 0x7f4ac40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.193+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4ac80e52d0 con 0x7f4ad81087d0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.193+0000 7f4addb73640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 0x7f4ac40b77a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4031440 tx=0x7f4ad4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:52.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.197+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4ac80ac590 con 0x7f4ad81087d0 2024-09-10T22:44:52.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.409+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad810af30 con 0x7f4ad81087d0 2024-09-10T22:44:52.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:52 smithi096 ceph-mon[30869]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:52.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.410+0000 7f4aceffd640 1 -- 172.21.15.96:0/897539252 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f4ac80abe90 con 0x7f4ad81087d0 2024-09-10T22:44:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:52.413 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"btime":"2024-09-10T22:44:50:509076+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T22:44:52.413 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 msgr2=0x7f4ac40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 0x7f4ac40b77a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4031440 tx=0x7f4ad4005b50 comp rx=0 tx=0).stop 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 msgr2=0x7f4ad8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d770 tx=0x7f4ac800dc30 comp rx=0 tx=0).stop 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 shutdown_connections 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ad8109150 0x7f4ad819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ac40b52e0 0x7f4ac40b77a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ad81087d0 0x7f4ad8199870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 --2- 172.21.15.96:0/897539252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ad8102dc0 0x7f4ad8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:52.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.412+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 >> 172.21.15.96:0/897539252 conn(0x7f4ad80fe830 msgr2=0x7f4ad8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:52.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.413+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 shutdown_connections 2024-09-10T22:44:52.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:52.413+0000 7f4adfdd7640 1 -- 172.21.15.96:0/897539252 wait complete. 2024-09-10T22:44:52.424 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:52 smithi121 ceph-mon[35119]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:52.425 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 podman[65249]: 2024-09-10 22:44:52.18400814 +0000 UTC m=+0.016393763 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:52.425 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 podman[65249]: 2024-09-10 22:44:52.301199725 +0000 UTC m=+0.133585333 container create 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:44:52.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:52 smithi117 ceph-mon[35081]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 podman[65249]: 2024-09-10 22:44:52.554329617 +0000 UTC m=+0.386715225 container init 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 podman[65249]: 2024-09-10 22:44:52.557045154 +0000 UTC m=+0.389430775 container start 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 ceph-mds[65268]: -- 172.21.15.121:0/1760497286 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8dbcd5860 con 0x55b8dbcb2800 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: starting mds.f at 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 bash[65249]: 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d 2024-09-10T22:44:53.014 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 22:44:52 smithi121 systemd[1]: Started Ceph mds.f for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:53.425 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/897539252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:53.425 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.426 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.426 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.426 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:53.426 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:53.426 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:53 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/897539252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:53.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:53 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/897539252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:53.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:53 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:54 smithi096 ceph-mon[30869]: Deploying daemon mds.c on smithi121 2024-09-10T22:44:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:54 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6834/1599221610,v1:172.21.15.121:6835/1599221610] up:boot 2024-09-10T22:44:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:54 smithi096 ceph-mon[30869]: fsmap 6 up:standby 2024-09-10T22:44:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:54 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:44:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:54 smithi096 ceph-mon[30869]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:54.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:54 smithi121 ceph-mon[35119]: Deploying daemon mds.c on smithi121 2024-09-10T22:44:54.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:54 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6834/1599221610,v1:172.21.15.121:6835/1599221610] up:boot 2024-09-10T22:44:54.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:54 smithi121 ceph-mon[35119]: fsmap 6 up:standby 2024-09-10T22:44:54.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:54 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:44:54.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:54 smithi121 ceph-mon[35119]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:54 smithi117 ceph-mon[35081]: Deploying daemon mds.c on smithi121 2024-09-10T22:44:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:54 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6834/1599221610,v1:172.21.15.121:6835/1599221610] up:boot 2024-09-10T22:44:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:54 smithi117 ceph-mon[35081]: fsmap 6 up:standby 2024-09-10T22:44:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:54 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:44:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:54 smithi117 ceph-mon[35081]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:54.923 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:54 smithi121 systemd[1]: Starting Ceph mds.c for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:55.140 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:55.212 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:54 smithi121 podman[65550]: 2024-09-10 22:44:54.824693783 +0000 UTC m=+0.016170060 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:55.212 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:54 smithi121 podman[65550]: 2024-09-10 22:44:54.956144534 +0000 UTC m=+0.147620812 container create 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:44:55.212 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 podman[65550]: 2024-09-10 22:44:55.211494456 +0000 UTC m=+0.402970733 container init 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:44:55.410 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:55.513 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 podman[65550]: 2024-09-10 22:44:55.21449298 +0000 UTC m=+0.405969267 container start 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T22:44:55.513 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 ceph-mds[65569]: -- 172.21.15.121:0/4228501437 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bb6b97860 con 0x555bb6b75400 2024-09-10T22:44:55.514 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: starting mds.c at 2024-09-10T22:44:55.514 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 bash[65550]: 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 2024-09-10T22:44:55.514 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 22:44:55 smithi121 systemd[1]: Started Ceph mds.c for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 -- 172.21.15.96:0/4273585861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 msgr2=0x7f583410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 --2- 172.21.15.96:0/4273585861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f583410cb30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f581c009ca0 tx=0x7f581c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 -- 172.21.15.96:0/4273585861 shutdown_connections 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 --2- 172.21.15.96:0/4273585861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 0x7f583410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 --2- 172.21.15.96:0/4273585861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f583410cb30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 --2- 172.21.15.96:0/4273585861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 0x7f5834103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 -- 172.21.15.96:0/4273585861 >> 172.21.15.96:0/4273585861 conn(0x7f58340fd700 msgr2=0x7f58340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 -- 172.21.15.96:0/4273585861 shutdown_connections 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.874+0000 7f583896a640 1 -- 172.21.15.96:0/4273585861 wait complete. 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 Processor -- start 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 -- start start 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 0x7f5834199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 0x7f583419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112d60 con 0x7f5834104550 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112be0 con 0x7f5834103b50 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583896a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112ee0 con 0x7f583410d100 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f5831d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f5831d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51124/0 (socket says 172.21.15.96:51124) 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f5831d9b640 1 -- 172.21.15.96:0/2351416541 learned_addr learned my addr 172.21.15.96:0/2351416541 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f583259c640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 0x7f5834199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.875+0000 7f5832d9d640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 0x7f583419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 -- 172.21.15.96:0/2351416541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 msgr2=0x7f583419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 0x7f583419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 -- 172.21.15.96:0/2351416541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 msgr2=0x7f5834199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 0x7f5834199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 -- 172.21.15.96:0/2351416541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419f2e0 con 0x7f5834104550 2024-09-10T22:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f5831d9b640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f581c006010 tx=0x7f581c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c003bf0 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419f570 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581c003d90 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c02fc20 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.876+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a5c90 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.877+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834103f30 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.879+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f581c0046a0 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.880+0000 7f582b7fe640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 0x7f58200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.880+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f581c106b30 con 0x7f5834104550 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.880+0000 7f583259c640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 0x7f58200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.880+0000 7f583259c640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 0x7f58200b77a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f582c00abe0 tx=0x7f582c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:55.886+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f581c0cde90 con 0x7f5834104550 2024-09-10T22:44:56.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.106+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f583419a360 con 0x7f5834104550 2024-09-10T22:44:56.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.106+0000 7f582b7fe640 1 -- 172.21.15.96:0/2351416541 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f583419a360 con 0x7f5834104550 2024-09-10T22:44:56.108 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:56.108 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"btime":"2024-09-10T22:44:55:292625+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T22:44:56.109 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-09-10T22:44:56.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 msgr2=0x7f58200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:56.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 0x7f58200b77a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f582c00abe0 tx=0x7f582c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 msgr2=0x7f5834199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f581c006010 tx=0x7f581c0319f0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 shutdown_connections 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58200b52e0 0x7f58200b77a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f583410d100 0x7f583419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5834104550 0x7f5834199770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 --2- 172.21.15.96:0/2351416541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5834103b50 0x7f5834199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 >> 172.21.15.96:0/2351416541 conn(0x7f58340fd700 msgr2=0x7f58340fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 shutdown_connections 2024-09-10T22:44:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:56.109+0000 7f583896a640 1 -- 172.21.15.96:0/2351416541 wait complete. 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:boot 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: fsmap 7 up:standby 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:56.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:56.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:56 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2351416541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:boot 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: fsmap 7 up:standby 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:56.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:56.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:56.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:56 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2351416541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:boot 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: fsmap 7 up:standby 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:56.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:56 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2351416541' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:44:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:57 smithi117 ceph-mon[35081]: Deploying daemon mds.h on smithi117 2024-09-10T22:44:57.527 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 systemd[1]: Starting Ceph mds.h for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:44:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:57 smithi096 ceph-mon[30869]: Deploying daemon mds.h on smithi117 2024-09-10T22:44:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:57 smithi121 ceph-mon[35119]: Deploying daemon mds.h on smithi117 2024-09-10T22:44:57.921 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 podman[64664]: 2024-09-10 22:44:57.546961529 +0000 UTC m=+0.015805778 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:44:57.921 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 podman[64664]: 2024-09-10 22:44:57.675720413 +0000 UTC m=+0.144564663 container create 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:44:57.921 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 podman[64664]: 2024-09-10 22:44:57.92008793 +0000 UTC m=+0.388932180 container init 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T22:44:58.276 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 podman[64664]: 2024-09-10 22:44:57.922951702 +0000 UTC m=+0.391795953 container start 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T22:44:58.276 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 ceph-mds[64683]: -- 172.21.15.117:0/3685619124 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef894a1c20 con 0x55ef8947f000 2024-09-10T22:44:58.276 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: starting mds.h at 2024-09-10T22:44:58.276 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:58 smithi117 bash[64664]: 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f 2024-09-10T22:44:58.276 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 22:44:58 smithi117 systemd[1]: Started Ceph mds.h for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:44:58.661 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:58.661 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.662 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.662 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.662 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:58.662 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:58.662 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:58.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:44:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:44:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:44:58.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:44:58.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:44:58.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:44:59.054 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:44:59.471 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:59 smithi096 ceph-mon[30869]: Deploying daemon mds.i on smithi121 2024-09-10T22:44:59.472 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:59 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:boot 2024-09-10T22:44:59.472 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:59 smithi096 ceph-mon[30869]: fsmap 8 up:standby 2024-09-10T22:44:59.472 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:44:59 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:44:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.469+0000 7fe0bd04d640 1 -- 172.21.15.96:0/612145789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b8102190 msgr2=0x7fe0b810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.469+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/612145789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b8102190 0x7fe0b810eee0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009900 tx=0x7fe0ac02a410 comp rx=0 tx=0).stop 2024-09-10T22:44:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 -- 172.21.15.96:0/612145789 shutdown_connections 2024-09-10T22:44:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/612145789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b810f420 0x7fe0b8111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/612145789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b8102190 0x7fe0b810eee0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/612145789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8101870 0x7fe0b8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 -- 172.21.15.96:0/612145789 >> 172.21.15.96:0/612145789 conn(0x7fe0b80fd700 msgr2=0x7fe0b80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 -- 172.21.15.96:0/612145789 shutdown_connections 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 -- 172.21.15.96:0/612145789 wait complete. 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.470+0000 7fe0bd04d640 1 Processor -- start 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 -- start start 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b8101870 0x7fe0b810ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8102190 0x7fe0b810be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:59.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8114f90 con 0x7fe0b810f420 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8114e10 con 0x7fe0b8102190 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0bd04d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8115110 con 0x7fe0b8101870 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47608/0 (socket says 172.21.15.96:47608) 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 -- 172.21.15.96:0/2934011158 learned_addr learned my addr 172.21.15.96:0/2934011158 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b659c640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8102190 0x7fe0b810be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b6d9d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b8101870 0x7fe0b810ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 -- 172.21.15.96:0/2934011158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b8101870 msgr2=0x7fe0b810ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b8101870 0x7fe0b810ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 -- 172.21.15.96:0/2934011158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8102190 msgr2=0x7fe0b810be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:59.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8102190 0x7fe0b810be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 -- 172.21.15.96:0/2934011158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b810cb90 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.471+0000 7fe0b759e640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d720 tx=0x7fe0b000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.472+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0017070 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.472+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b000de30 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.472+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b81abd30 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.472+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004d00 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.472+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b81ac190 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.473+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b8101c50 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.474+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fe0b0004ea0 con 0x7fe0b810f420 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.475+0000 7fe09ffff640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 0x7fe0a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:44:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.475+0000 7fe0b6d9d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 0x7fe0a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:44:59.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.475+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fe0b00e5370 con 0x7fe0b810f420 2024-09-10T22:44:59.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.475+0000 7fe0b6d9d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 0x7fe0a00b77a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8007ec0 tx=0x7fe0a80075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:44:59.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.477+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0b00ac080 con 0x7fe0b810f420 2024-09-10T22:44:59.620 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:59 smithi121 ceph-mon[35119]: Deploying daemon mds.i on smithi121 2024-09-10T22:44:59.621 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:59 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:boot 2024-09-10T22:44:59.621 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:59 smithi121 ceph-mon[35119]: fsmap 8 up:standby 2024-09-10T22:44:59.621 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:44:59 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:44:59.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.689+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b81ac640 con 0x7fe0b810f420 2024-09-10T22:44:59.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.689+0000 7fe09ffff640 1 -- 172.21.15.96:0/2934011158 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7fe0b00b0ab0 con 0x7fe0b810f420 2024-09-10T22:44:59.691 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:44:59.691 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"btime":"2024-09-10T22:44:58:300737+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:44:59.691 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-09-10T22:44:59.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 msgr2=0x7fe0a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:59.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 0x7fe0a00b77a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8007ec0 tx=0x7fe0a80075b0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 msgr2=0x7fe0b810c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:44:59.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d720 tx=0x7fe0b000dbe0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 shutdown_connections 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0b8102190 0x7fe0b810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0a00b52e0 0x7fe0a00b77a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0b810f420 0x7fe0b810c360 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 --2- 172.21.15.96:0/2934011158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0b8101870 0x7fe0b810ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 >> 172.21.15.96:0/2934011158 conn(0x7fe0b80fd700 msgr2=0x7fe0b81115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 shutdown_connections 2024-09-10T22:44:59.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:44:59.692+0000 7fe0bd04d640 1 -- 172.21.15.96:0/2934011158 wait complete. 2024-09-10T22:44:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:59 smithi117 ceph-mon[35081]: Deploying daemon mds.i on smithi121 2024-09-10T22:44:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:59 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:boot 2024-09-10T22:44:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:59 smithi117 ceph-mon[35081]: fsmap 8 up:standby 2024-09-10T22:44:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:44:59 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:45:00.264 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 podman[65850]: 2024-09-10 22:45:00.239773079 +0000 UTC m=+0.129633021 container create 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T22:45:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:00 smithi096 ceph-mon[30869]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:00 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2934011158' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:00.652 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:00 smithi121 ceph-mon[35119]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:00.652 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:00 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2934011158' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:00.652 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 podman[65850]: 2024-09-10 22:45:00.550967858 +0000 UTC m=+0.440827803 container init 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T22:45:00.652 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 podman[65850]: 2024-09-10 22:45:00.553763291 +0000 UTC m=+0.443623232 container start 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:45:00.652 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 ceph-mds[65870]: -- 172.21.15.121:0/1706394121 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc6c8a5860 con 0x55fc6c883000 2024-09-10T22:45:00.653 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: starting mds.i at 2024-09-10T22:45:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:00 smithi117 ceph-mon[35081]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:00 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2934011158' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:01.013 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 bash[65850]: 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c 2024-09-10T22:45:01.013 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 22:45:00 smithi121 systemd[1]: Started Ceph mds.i for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:45:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: Deploying daemon mds.g on smithi096 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:boot 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: fsmap 9 up:standby 2024-09-10T22:45:01.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:01 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: Deploying daemon mds.g on smithi096 2024-09-10T22:45:02.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:02.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:boot 2024-09-10T22:45:02.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: fsmap 9 up:standby 2024-09-10T22:45:02.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:01 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:45:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: Deploying daemon mds.g on smithi096 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:boot 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: fsmap 9 up:standby 2024-09-10T22:45:02.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:01 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:45:02.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:45:02.671 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:02.751 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:02 smithi096 systemd[1]: Starting Ceph mds.g for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/3103507411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 msgr2=0x7fdf04107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/3103507411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf04107b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009900 tx=0x7fdef402a410 comp rx=0 tx=0).stop 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/3103507411 shutdown_connections 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/3103507411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04108060 0x7fdf0410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/3103507411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf04107b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/3103507411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04101cd0 0x7fdf041020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/3103507411 >> 172.21.15.96:0/3103507411 conn(0x7fdf040fd740 msgr2=0x7fdf040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/3103507411 shutdown_connections 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.113+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/3103507411 wait complete. 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 Processor -- start 2024-09-10T22:45:03.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 -- start start 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04101cd0 0x7fdf0419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04108060 0x7fdf041a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04111d30 con 0x7fdf041076e0 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04111bb0 con 0x7fdf04108060 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf0b1e7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04111eb0 con 0x7fdf04101cd0 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdefbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdefbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47624/0 (socket says 172.21.15.96:47624) 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdefbfff640 1 -- 172.21.15.96:0/1432962552 learned_addr learned my addr 172.21.15.96:0/1432962552 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.114+0000 7fdf08f83640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04101cd0 0x7fdf0419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdf09784640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04108060 0x7fdf041a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 -- 172.21.15.96:0/1432962552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04101cd0 msgr2=0x7fdf0419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04101cd0 0x7fdf0419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 -- 172.21.15.96:0/1432962552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04108060 msgr2=0x7fdf041a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04108060 0x7fdf041a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 -- 172.21.15.96:0/1432962552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf041a1500 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdefbfff640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdef4005ab0 tx=0x7fdef4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef403f070 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef4031d60 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf041a1790 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4035030 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.115+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf041a7eb0 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.116+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf041020b0 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.118+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fdef403b580 con 0x7fdf041076e0 2024-09-10T22:45:03.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.118+0000 7fdef9ffb640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 0x7fdeec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:03.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.118+0000 7fdf08f83640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 0x7fdeec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:03.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.119+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fdef41068d0 con 0x7fdf041076e0 2024-09-10T22:45:03.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.119+0000 7fdf08f83640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 0x7fdeec0b77a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf00007ec0 tx=0x7fdf000075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:03.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.121+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdef40cdc30 con 0x7fdf041076e0 2024-09-10T22:45:03.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.335+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf041a8150 con 0x7fdf041076e0 2024-09-10T22:45:03.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.335+0000 7fdef9ffb640 1 -- 172.21.15.96:0/1432962552 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7fdef40cd530 con 0x7fdf041076e0 2024-09-10T22:45:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:03.338 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"btime":"2024-09-10T22:45:01:348582+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T22:45:03.338 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 10 2024-09-10T22:45:03.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 msgr2=0x7fdeec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 0x7fdeec0b77a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf00007ec0 tx=0x7fdf000075b0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 msgr2=0x7fdf0419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdef4005ab0 tx=0x7fdef4003cd0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 shutdown_connections 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04108060 0x7fdf041a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdeec0b52e0 0x7fdeec0b77a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf041076e0 0x7fdf0419b990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 --2- 172.21.15.96:0/1432962552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04101cd0 0x7fdf0419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:03.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 >> 172.21.15.96:0/1432962552 conn(0x7fdf040fd740 msgr2=0x7fdf04100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:03.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 shutdown_connections 2024-09-10T22:45:03.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:03.338+0000 7fdf0b1e7640 1 -- 172.21.15.96:0/1432962552 wait complete. 2024-09-10T22:45:03.378 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:03 smithi096 podman[73130]: 2024-09-10 22:45:03.164618215 +0000 UTC m=+0.119986565 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:45:03.378 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:03 smithi096 podman[73130]: 2024-09-10 22:45:03.283332689 +0000 UTC m=+0.238701028 container create 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:45:03.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:03 smithi096 ceph-mon[30869]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:03.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:03 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1432962552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:03 smithi121 ceph-mon[35119]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:03 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1432962552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:03 smithi117 ceph-mon[35081]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:03 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1432962552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 podman[73130]: 2024-09-10 22:45:04.057401149 +0000 UTC m=+1.012769487 container init 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 podman[73130]: 2024-09-10 22:45:04.060670595 +0000 UTC m=+1.016038934 container start 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 ceph-mds[73185]: -- 172.21.15.96:0/881590631 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e203597860 con 0x55e203575000 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: starting mds.g at 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 bash[73130]: 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b 2024-09-10T22:45:04.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 22:45:04 smithi096 systemd[1]: Started Ceph mds.g for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:45:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: Deploying daemon mds.a on smithi096 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6838/2726589142,v1:172.21.15.96:6839/2726589142] up:boot 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: fsmap 10 up:standby 2024-09-10T22:45:05.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:05 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:45:05.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: Deploying daemon mds.a on smithi096 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6838/2726589142,v1:172.21.15.96:6839/2726589142] up:boot 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: fsmap 10 up:standby 2024-09-10T22:45:05.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:05 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:45:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:05.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: Deploying daemon mds.a on smithi096 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6838/2726589142,v1:172.21.15.96:6839/2726589142] up:boot 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: fsmap 10 up:standby 2024-09-10T22:45:05.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:05 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:45:05.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:45:06.214 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 systemd[1]: Starting Ceph mds.a for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:45:06.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:06 smithi096 ceph-mon[30869]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:06.501 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 podman[73487]: 2024-09-10 22:45:06.172035627 +0000 UTC m=+0.022443671 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:45:06.501 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 podman[73487]: 2024-09-10 22:45:06.29239612 +0000 UTC m=+0.142804150 container create 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T22:45:06.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:06 smithi121 ceph-mon[35119]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:06.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:06 smithi117 ceph-mon[35081]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 podman[73487]: 2024-09-10 22:45:06.528824761 +0000 UTC m=+0.379232793 container init 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 podman[73487]: 2024-09-10 22:45:06.532111127 +0000 UTC m=+0.382519166 container start 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 ceph-mds[73532]: -- 172.21.15.96:0/4049586445 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645f5f5f860 con 0x5645f5f3d000 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: starting mds.a at 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 bash[73487]: 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 2024-09-10T22:45:06.814 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 22:45:06 smithi096 systemd[1]: Started Ceph mds.a for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- 172.21.15.96:0/4294280453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac1045e0 msgr2=0x7fb1ac10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 --2- 172.21.15.96:0/4294280453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac10cad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009900 tx=0x7fb1a002a410 comp rx=0 tx=0).stop 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- 172.21.15.96:0/4294280453 shutdown_connections 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 --2- 172.21.15.96:0/4294280453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac10d120 0x7fb1ac10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 --2- 172.21.15.96:0/4294280453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac10cad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 --2- 172.21.15.96:0/4294280453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- 172.21.15.96:0/4294280453 >> 172.21.15.96:0/4294280453 conn(0x7fb1ac0fd700 msgr2=0x7fb1ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- 172.21.15.96:0/4294280453 shutdown_connections 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- 172.21.15.96:0/4294280453 wait complete. 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 Processor -- start 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.188+0000 7fb1b2361640 1 -- start start 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac10d120 0x7fb1ac19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac112d90 con 0x7fb1ac103cc0 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac112c10 con 0x7fb1ac1045e0 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1b2361640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac112f10 con 0x7fb1ac10d120 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58186/0 (socket says 172.21.15.96:58186) 2024-09-10T22:45:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 -- 172.21.15.96:0/2711148617 learned_addr learned my addr 172.21.15.96:0/2711148617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1ab7fe640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 -- 172.21.15.96:0/2711148617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac10d120 msgr2=0x7fb1ac19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac10d120 0x7fb1ac19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 -- 172.21.15.96:0/2711148617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1045e0 msgr2=0x7fb1ac199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 -- 172.21.15.96:0/2711148617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac19f370 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.189+0000 7fb1abfff640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c970 tx=0x7fb19c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.190+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c007bb0 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.190+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac19f660 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.190+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19c007d50 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.190+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00f450 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.190+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac1a5d20 con 0x7fb1ac103cc0 2024-09-10T22:45:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.191+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac1040a0 con 0x7fb1ac103cc0 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.192+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb19c004f00 con 0x7fb1ac103cc0 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.193+0000 7fb1a97fa640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 0x7fb1940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.193+0000 7fb1ab7fe640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 0x7fb1940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.193+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb19c0e5820 con 0x7fb1ac103cc0 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.193+0000 7fb1ab7fe640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 0x7fb1940b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000c00 tx=0x7fb1a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.196+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb19c014030 con 0x7fb1ac103cc0 2024-09-10T22:45:07.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.410+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1ac19a3f0 con 0x7fb1ac103cc0 2024-09-10T22:45:07.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.410+0000 7fb1a97fa640 1 -- 172.21.15.96:0/2711148617 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7fb19c0acb80 con 0x7fb1ac103cc0 2024-09-10T22:45:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:07.412 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"btime":"2024-09-10T22:45:07:213374+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T22:45:07.412 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-09-10T22:45:07.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.412+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 msgr2=0x7fb1940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:07.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.412+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 0x7fb1940b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000c00 tx=0x7fb1a0031400 comp rx=0 tx=0).stop 2024-09-10T22:45:07.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.412+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 msgr2=0x7fb1ac1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:07.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.412+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c970 tx=0x7fb19c00ce30 comp rx=0 tx=0).stop 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 shutdown_connections 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1045e0 0x7fb1ac199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1940b52e0 0x7fb1940b77a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac10d120 0x7fb1ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 --2- 172.21.15.96:0/2711148617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ac103cc0 0x7fb1ac1992c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 >> 172.21.15.96:0/2711148617 conn(0x7fb1ac0fd700 msgr2=0x7fb1ac0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 shutdown_connections 2024-09-10T22:45:07.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:07.413+0000 7fb1b2361640 1 -- 172.21.15.96:0/2711148617 wait complete. 2024-09-10T22:45:08.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: Deploying daemon mds.b on smithi117 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6840/2618224226,v1:172.21.15.96:6841/2618224226] up:boot 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: fsmap 11 up:standby 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:08.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:07 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2711148617' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: Deploying daemon mds.b on smithi117 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6840/2618224226,v1:172.21.15.96:6841/2618224226] up:boot 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: fsmap 11 up:standby 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:08.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:07 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2711148617' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: Deploying daemon mds.b on smithi117 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6840/2618224226,v1:172.21.15.96:6841/2618224226] up:boot 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: fsmap 11 up:standby 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:45:08.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:08.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:07 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2711148617' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:08.934 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:08 smithi117 systemd[1]: Starting Ceph mds.b for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T22:45:09.215 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:08 smithi117 podman[64964]: 2024-09-10 22:45:08.835503573 +0000 UTC m=+0.016066993 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:45:09.215 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:08 smithi117 podman[64964]: 2024-09-10 22:45:08.964597401 +0000 UTC m=+0.145160823 container create 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:45:09.215 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 podman[64964]: 2024-09-10 22:45:09.214754557 +0000 UTC m=+0.395317986 container init 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T22:45:09.526 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 podman[64964]: 2024-09-10 22:45:09.21763345 +0000 UTC m=+0.398196870 container start 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:45:09.526 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 ceph-mds[64983]: -- 172.21.15.117:0/685683318 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e12b4d860 con 0x562e12b2a800 2024-09-10T22:45:09.526 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: starting mds.b at 2024-09-10T22:45:09.526 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 bash[64964]: 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 2024-09-10T22:45:09.526 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:09 smithi117 systemd[1]: Started Ceph mds.b for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T22:45:10.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:boot 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: fsmap 12 up:standby 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.405 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.406 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.406 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.406 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:10 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:boot 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: fsmap 12 up:standby 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:10 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:boot 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: fsmap 12 up:standby 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:10.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:10 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:11.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 -- 172.21.15.96:0/2105656899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 msgr2=0x7f248c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:11.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 --2- 172.21.15.96:0/2105656899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c10f930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2484009ca0 tx=0x7f248402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 -- 172.21.15.96:0/2105656899 shutdown_connections 2024-09-10T22:45:11.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 --2- 172.21.15.96:0/2105656899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 0x7f248c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 --2- 172.21.15.96:0/2105656899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c10f930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 --2- 172.21.15.96:0/2105656899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 0x7f248c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 -- 172.21.15.96:0/2105656899 >> 172.21.15.96:0/2105656899 conn(0x7f248c0fe810 msgr2=0x7f248c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 -- 172.21.15.96:0/2105656899 shutdown_connections 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.042+0000 7f2492683640 1 -- 172.21.15.96:0/2105656899 wait complete. 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 Processor -- start 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 -- start start 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 0x7f248c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 0x7f248c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:11.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112e00 con 0x7f248c109130 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112c80 con 0x7f248c1087b0 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2492683640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112f80 con 0x7f248c102da0 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58206/0 (socket says 172.21.15.96:58206) 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 -- 172.21.15.96:0/1201224409 learned_addr learned my addr 172.21.15.96:0/1201224409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f248bfff640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 0x7f248c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:11.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2483fff640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 0x7f248c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 -- 172.21.15.96:0/1201224409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 msgr2=0x7f248c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 0x7f248c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 -- 172.21.15.96:0/1201224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 msgr2=0x7f248c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 0x7f248c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.043+0000 7f2490c20640 1 -- 172.21.15.96:0/1201224409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c19f2e0 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2490c20640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f248402f540 tx=0x7f24840316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484003b70 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484003d10 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c19f570 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248402f990 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.044+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1a5c90 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.045+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c103180 con 0x7f248c109130 2024-09-10T22:45:11.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.046+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f248402fb30 con 0x7f248c109130 2024-09-10T22:45:11.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.047+0000 7f2489ffb640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:11.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.047+0000 7f248bfff640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:11.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.047+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f2484106a80 con 0x7f248c109130 2024-09-10T22:45:11.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.047+0000 7f248bfff640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f247c009bd0 tx=0x7f247c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:11.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.050+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f24840cdde0 con 0x7f248c109130 2024-09-10T22:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.293+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248c19a360 con 0x7f248c109130 2024-09-10T22:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.294+0000 7f2489ffb640 1 -- 172.21.15.96:0/1201224409 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7f248c19a360 con 0x7f248c109130 2024-09-10T22:45:11.296 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:11.296 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"btime":"2024-09-10T22:45:09:295984+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14700,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24481,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24527,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T22:45:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 13 2024-09-10T22:45:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 msgr2=0x7f24780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f247c009bd0 tx=0x7f247c005f90 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 msgr2=0x7f248c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f248402f540 tx=0x7f24840316f0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 shutdown_connections 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f248c1087b0 0x7f248c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248c109130 0x7f248c19eb10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 --2- 172.21.15.96:0/1201224409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f248c102da0 0x7f248c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.296+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 >> 172.21.15.96:0/1201224409 conn(0x7f248c0fe810 msgr2=0x7f248c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.297+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 shutdown_connections 2024-09-10T22:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:11.297+0000 7f2492683640 1 -- 172.21.15.96:0/1201224409 wait complete. 2024-09-10T22:45:12.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1201224409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:12.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:12 smithi096 ceph-mon[30869]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1201224409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:12.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:12 smithi121 ceph-mon[35119]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:12.273 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1201224409' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:12.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:12 smithi117 ceph-mon[35081]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.785+0000 7efe7a472640 1 -- 172.21.15.96:0/463365960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe741076a0 msgr2=0x7efe74107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.785+0000 7efe7a472640 1 --2- 172.21.15.96:0/463365960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe741076a0 0x7efe74107ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 -- 172.21.15.96:0/463365960 shutdown_connections 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 --2- 172.21.15.96:0/463365960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 0x7efe7410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 --2- 172.21.15.96:0/463365960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe741076a0 0x7efe74107ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 --2- 172.21.15.96:0/463365960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe74101c90 0x7efe74102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 -- 172.21.15.96:0/463365960 >> 172.21.15.96:0/463365960 conn(0x7efe740fd700 msgr2=0x7efe740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 -- 172.21.15.96:0/463365960 shutdown_connections 2024-09-10T22:45:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 -- 172.21.15.96:0/463365960 wait complete. 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 Processor -- start 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.786+0000 7efe7a472640 1 -- start start 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741076a0 0x7efe7419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 0x7efe7419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111ce0 con 0x7efe74101c90 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111b60 con 0x7efe741076a0 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111e60 con 0x7efe74108020 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58226/0 (socket says 172.21.15.96:58226) 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 -- 172.21.15.96:0/4052882481 learned_addr learned my addr 172.21.15.96:0/4052882481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe737fe640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741076a0 0x7efe7419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe78a0f640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 0x7efe7419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 -- 172.21.15.96:0/4052882481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 msgr2=0x7efe7419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 0x7efe7419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 -- 172.21.15.96:0/4052882481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741076a0 msgr2=0x7efe7419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741076a0 0x7efe7419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 -- 172.21.15.96:0/4052882481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741a54d0 con 0x7efe74101c90 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe73fff640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe6400d950 tx=0x7efe6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64017070 con 0x7efe74101c90 2024-09-10T22:45:12.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe64009c30 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741a57c0 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400f620 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.787+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741a5cd0 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.789+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe74102070 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.790+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7efe64010460 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.791+0000 7efe717fa640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 0x7efe5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.791+0000 7efe737fe640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 0x7efe5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.791+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7efe640e52c0 con 0x7efe74101c90 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.791+0000 7efe737fe640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 0x7efe5c0b77a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe68031440 tx=0x7efe68005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:12.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.794+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efe64009de0 con 0x7efe74101c90 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.986+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efe7402e5c0 con 0x7efe74101c90 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.986+0000 7efe717fa640 1 -- 172.21.15.96:0/4052882481 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7efe640abfd0 con 0x7efe74101c90 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T22:45:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-10T22:45:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-10T22:45:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:45:12.989 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-10T22:45:12.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.988+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 msgr2=0x7efe5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:12.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 0x7efe5c0b77a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe68031440 tx=0x7efe68005b50 comp rx=0 tx=0).stop 2024-09-10T22:45:12.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 msgr2=0x7efe7419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:12.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe6400d950 tx=0x7efe6400de10 comp rx=0 tx=0).stop 2024-09-10T22:45:12.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 shutdown_connections 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe741076a0 0x7efe7419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efe5c0b52e0 0x7efe5c0b77a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe74108020 0x7efe7419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 --2- 172.21.15.96:0/4052882481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74101c90 0x7efe7419a0f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 >> 172.21.15.96:0/4052882481 conn(0x7efe740fd700 msgr2=0x7efe740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 shutdown_connections 2024-09-10T22:45:12.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:12.989+0000 7efe7a472640 1 -- 172.21.15.96:0/4052882481 wait complete. 2024-09-10T22:45:13.215 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4052882481' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4052882481' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:45:13.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:13.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4052882481' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:45:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:14 smithi096 ceph-mon[30869]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:14 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.285 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:45:14.286 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:45:14.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:14 smithi121 ceph-mon[35119]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:14.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:14 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:14 smithi117 ceph-mon[35081]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:14 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:14.558 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- 172.21.15.96:0/2649220953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 msgr2=0x7fbf44103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 --2- 172.21.15.96:0/2649220953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44103f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009900 tx=0x7fbf3402a410 comp rx=0 tx=0).stop 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- 172.21.15.96:0/2649220953 shutdown_connections 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 --2- 172.21.15.96:0/2649220953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4410d140 0x7fbf4410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 --2- 172.21.15.96:0/2649220953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf44104590 0x7fbf4410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 --2- 172.21.15.96:0/2649220953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44103f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- 172.21.15.96:0/2649220953 >> 172.21.15.96:0/2649220953 conn(0x7fbf440fd700 msgr2=0x7fbf440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- 172.21.15.96:0/2649220953 shutdown_connections 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- 172.21.15.96:0/2649220953 wait complete. 2024-09-10T22:45:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 Processor -- start 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.053+0000 7fbf496fa640 1 -- start start 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf44104590 0x7fbf441997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf4410d140 0x7fbf4419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44112d60 con 0x7fbf44103b90 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44112be0 con 0x7fbf44104590 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf496fa640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44112ee0 con 0x7fbf4410d140 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58238/0 (socket says 172.21.15.96:58238) 2024-09-10T22:45:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 -- 172.21.15.96:0/3152851159 learned_addr learned my addr 172.21.15.96:0/3152851159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf437fe640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf4410d140 0x7fbf4419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf427fc640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf44104590 0x7fbf441997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 -- 172.21.15.96:0/3152851159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf4410d140 msgr2=0x7fbf4419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf4410d140 0x7fbf4419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 -- 172.21.15.96:0/3152851159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf44104590 msgr2=0x7fbf441997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf44104590 0x7fbf441997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 -- 172.21.15.96:0/3152851159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4419f330 con 0x7fbf44103b90 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf42ffd640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005940 tx=0x7fbf340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.054+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3403f030 con 0x7fbf44103b90 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.055+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4419f5c0 con 0x7fbf44103b90 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.055+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf34003cb0 con 0x7fbf44103b90 2024-09-10T22:45:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.055+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34035070 con 0x7fbf44103b90 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.055+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441a5ce0 con 0x7fbf44103b90 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.056+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf44103f70 con 0x7fbf44103b90 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.058+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fbf3403bbf0 con 0x7fbf44103b90 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.058+0000 7fbf2bfff640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 0x7fbf2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.059+0000 7fbf427fc640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 0x7fbf2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.059+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fbf34106480 con 0x7fbf44103b90 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.059+0000 7fbf427fc640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 0x7fbf2c0b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf4410f8d0 tx=0x7fbf3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.061+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbf340cd190 con 0x7fbf44103b90 2024-09-10T22:45:15.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.238+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbf4419a3b0 con 0x7fbf44103b90 2024-09-10T22:45:15.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.918+0000 7fbf2bfff640 1 -- 172.21.15.96:0/3152851159 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fbf4419a3b0 con 0x7fbf44103b90 2024-09-10T22:45:15.920 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 msgr2=0x7fbf2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 0x7fbf2c0b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf4410f8d0 tx=0x7fbf3800b390 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 msgr2=0x7fbf44199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005940 tx=0x7fbf340042f0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 shutdown_connections 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf44104590 0x7fbf441997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf2c0b52e0 0x7fbf2c0b77a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf4410d140 0x7fbf4419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 --2- 172.21.15.96:0/3152851159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf44103b90 0x7fbf44199280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.920+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 >> 172.21.15.96:0/3152851159 conn(0x7fbf440fd700 msgr2=0x7fbf440fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.921+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 shutdown_connections 2024-09-10T22:45:15.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:15.921+0000 7fbf496fa640 1 -- 172.21.15.96:0/3152851159 wait complete. 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:45:16.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:15 smithi096 ceph-mon[30869]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:16.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:16.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:45:16.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:15 smithi121 ceph-mon[35119]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:45:16.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:15 smithi117 ceph-mon[35081]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:16.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:45:16.905 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:16.930 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:16 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:45:16.930 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:16 smithi096 ceph-mon[30869]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:45:17.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:16 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:45:17.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:16 smithi121 ceph-mon[35119]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:45:17.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:16 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3152851159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:45:17.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:16 smithi117 ceph-mon[35081]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T22:45:17.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 -- 172.21.15.96:0/3965495008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e4101c90 msgr2=0x7fb2e4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:17.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 --2- 172.21.15.96:0/3965495008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e4101c90 0x7fb2e4102070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009900 tx=0x7fb2d402a410 comp rx=0 tx=0).stop 2024-09-10T22:45:17.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 -- 172.21.15.96:0/3965495008 shutdown_connections 2024-09-10T22:45:17.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 --2- 172.21.15.96:0/3965495008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 0x7fb2e410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 --2- 172.21.15.96:0/3965495008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e41076a0 0x7fb2e4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 --2- 172.21.15.96:0/3965495008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e4101c90 0x7fb2e4102070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.396+0000 7fb2e921f640 1 -- 172.21.15.96:0/3965495008 >> 172.21.15.96:0/3965495008 conn(0x7fb2e40fd700 msgr2=0x7fb2e40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.397+0000 7fb2e921f640 1 -- 172.21.15.96:0/3965495008 shutdown_connections 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.397+0000 7fb2e921f640 1 -- 172.21.15.96:0/3965495008 wait complete. 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.397+0000 7fb2e921f640 1 Processor -- start 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.397+0000 7fb2e921f640 1 -- start start 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e4101c90 0x7fb2e419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 0x7fb2e419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111d20 con 0x7fb2e41076a0 2024-09-10T22:45:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111ba0 con 0x7fb2e4108020 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111ea0 con 0x7fb2e4101c90 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32934/0 (socket says 172.21.15.96:32934) 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 -- 172.21.15.96:0/4114191905 learned_addr learned my addr 172.21.15.96:0/4114191905 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e2d9d640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e4101c90 0x7fb2e419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e359e640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 0x7fb2e419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 -- 172.21.15.96:0/4114191905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e4101c90 msgr2=0x7fb2e419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e4101c90 0x7fb2e419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 -- 172.21.15.96:0/4114191905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 msgr2=0x7fb2e419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:17.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 0x7fb2e419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 -- 172.21.15.96:0/4114191905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e41a55d0 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e259c640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b4a0 tx=0x7fb2d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8014070 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41a58c0 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.398+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8010ac0 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.399+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a5e80 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.399+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8017c30 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.399+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e41020f0 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.401+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb2d8010610 con 0x7fb2e41076a0 2024-09-10T22:45:17.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.402+0000 7fb2cbfff640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 0x7fb2cc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:17.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.402+0000 7fb2e2d9d640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 0x7fb2cc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:17.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.402+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7fb2d80e5230 con 0x7fb2e41076a0 2024-09-10T22:45:17.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.402+0000 7fb2e2d9d640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 0x7fb2cc0b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4031440 tx=0x7fb2d4005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:17.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.405+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb2d80abd30 con 0x7fb2e41076a0 2024-09-10T22:45:17.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.582+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2e4109870 con 0x7fb2e41076a0 2024-09-10T22:45:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.920+0000 7fb2cbfff640 1 -- 172.21.15.96:0/4114191905 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fb2d80b0760 con 0x7fb2e41076a0 2024-09-10T22:45:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-10T22:45:17.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.923+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 msgr2=0x7fb2cc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:17.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.923+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 0x7fb2cc0b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4031440 tx=0x7fb2d4005e80 comp rx=0 tx=0).stop 2024-09-10T22:45:17.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.923+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 msgr2=0x7fb2e419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:17.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.923+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b4a0 tx=0x7fb2d800b960 comp rx=0 tx=0).stop 2024-09-10T22:45:17.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 shutdown_connections 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2e4108020 0x7fb2e419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2cc0b52e0 0x7fb2cc0b77a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e41076a0 0x7fb2e419a730 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 --2- 172.21.15.96:0/4114191905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e4101c90 0x7fb2e419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 >> 172.21.15.96:0/4114191905 conn(0x7fb2e40fd700 msgr2=0x7fb2e40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 shutdown_connections 2024-09-10T22:45:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:17.924+0000 7fb2e921f640 1 -- 172.21.15.96:0/4114191905 wait complete. 2024-09-10T22:45:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:17 smithi096 ceph-mon[30869]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:45:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:17 smithi096 ceph-mon[30869]: pgmap v282: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:17 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:45:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:17 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:45:18.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:17 smithi096 ceph-mon[30869]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:45:18.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:17 smithi121 ceph-mon[35119]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:45:18.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:17 smithi121 ceph-mon[35119]: pgmap v282: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:17 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:45:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:17 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:45:18.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:17 smithi121 ceph-mon[35119]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:45:18.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:17 smithi117 ceph-mon[35081]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T22:45:18.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:17 smithi117 ceph-mon[35081]: pgmap v282: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:18.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:17 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:45:18.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:17 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4114191905' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:45:18.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:17 smithi117 ceph-mon[35081]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:45:18.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:45:18.833 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:19.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:18 smithi096 ceph-mon[30869]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:45:19.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:18 smithi096 ceph-mon[30869]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:45:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:18 smithi121 ceph-mon[35119]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:45:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:18 smithi121 ceph-mon[35119]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:45:19.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:18 smithi117 ceph-mon[35081]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:45:19.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:18 smithi117 ceph-mon[35081]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 -- 172.21.15.96:0/1445755171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c1087b0 msgr2=0x7ff95c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 --2- 172.21.15.96:0/1445755171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c1087b0 0x7ff95c108bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009900 tx=0x7ff94c02a410 comp rx=0 tx=0).stop 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 -- 172.21.15.96:0/1445755171 shutdown_connections 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 --2- 172.21.15.96:0/1445755171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c109130 0x7ff95c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 --2- 172.21.15.96:0/1445755171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c1087b0 0x7ff95c108bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 --2- 172.21.15.96:0/1445755171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c102da0 0x7ff95c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 -- 172.21.15.96:0/1445755171 >> 172.21.15.96:0/1445755171 conn(0x7ff95c0fe810 msgr2=0x7ff95c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 -- 172.21.15.96:0/1445755171 shutdown_connections 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.325+0000 7ff964427640 1 -- 172.21.15.96:0/1445755171 wait complete. 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 Processor -- start 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 -- start start 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c1087b0 0x7ff95c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 0x7ff95c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112df0 con 0x7ff95c102da0 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112c70 con 0x7ff95c109130 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff964427640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112f70 con 0x7ff95c1087b0 2024-09-10T22:45:19.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32952/0 (socket says 172.21.15.96:32952) 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 -- 172.21.15.96:0/2644464812 learned_addr learned my addr 172.21.15.96:0/2644464812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9619c2640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c1087b0 0x7ff95c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9629c4640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 0x7ff95c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 -- 172.21.15.96:0/2644464812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c1087b0 msgr2=0x7ff95c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c1087b0 0x7ff95c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 -- 172.21.15.96:0/2644464812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 msgr2=0x7ff95c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 0x7ff95c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.326+0000 7ff9621c3640 1 -- 172.21.15.96:0/2644464812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c19f2d0 con 0x7ff95c102da0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff9629c4640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 0x7ff95c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff9621c3640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff95800d950 tx=0x7ff95800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958017070 con 0x7ff95c102da0 2024-09-10T22:45:19.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff958009c30 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c19f5c0 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95800f620 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.327+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1a5e50 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.328+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95c104050 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.329+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff958005030 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.330+0000 7ff9537fe640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 0x7ff9480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.330+0000 7ff9619c2640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 0x7ff9480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.330+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7ff9580e5710 con 0x7ff95c102da0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.331+0000 7ff9619c2640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 0x7ff9480b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff94c031440 tx=0x7ff94c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:19.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.334+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff9580ac760 con 0x7ff95c102da0 2024-09-10T22:45:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.549+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff95c1a6080 con 0x7ff95c102da0 2024-09-10T22:45:19.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.953+0000 7ff9537fe640 1 -- 172.21.15.96:0/2644464812 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff958009de0 con 0x7ff95c102da0 2024-09-10T22:45:19.956 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:45:19.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 msgr2=0x7ff9480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 0x7ff9480b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff94c031440 tx=0x7ff94c0095f0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 msgr2=0x7ff95c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff95800d950 tx=0x7ff95800de10 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 shutdown_connections 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff95c109130 0x7ff95c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff9480b5360 0x7ff9480b7820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff95c1087b0 0x7ff95c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 --2- 172.21.15.96:0/2644464812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c102da0 0x7ff95c199220 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff95800d950 tx=0x7ff95800de10 comp rx=0 tx=0).stop 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 >> 172.21.15.96:0/2644464812 conn(0x7ff95c0fe810 msgr2=0x7ff95c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 shutdown_connections 2024-09-10T22:45:19.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:19.956+0000 7ff964427640 1 -- 172.21.15.96:0/2644464812 wait complete. 2024-09-10T22:45:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:19 smithi096 ceph-mon[30869]: pgmap v285: 97 pgs: 42 creating+peering, 34 unknown, 21 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:19 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:45:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:19 smithi096 ceph-mon[30869]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:45:20.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:19 smithi121 ceph-mon[35119]: pgmap v285: 97 pgs: 42 creating+peering, 34 unknown, 21 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:20.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:19 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:45:20.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:19 smithi121 ceph-mon[35119]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:45:20.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:19 smithi117 ceph-mon[35081]: pgmap v285: 97 pgs: 42 creating+peering, 34 unknown, 21 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:45:20.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:19 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:45:20.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:19 smithi117 ceph-mon[35081]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:45:20.707 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:45:20.958 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:45:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:20 smithi096 ceph-mon[30869]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:45:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:20 smithi096 ceph-mon[30869]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:45:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:20 smithi096 ceph-mon[30869]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:45:21.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:20 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:45:21.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:20 smithi121 ceph-mon[35119]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:45:21.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:20 smithi121 ceph-mon[35119]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:45:21.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:20 smithi121 ceph-mon[35119]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:45:21.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:20 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2644464812' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:45:21.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:20 smithi117 ceph-mon[35081]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:45:21.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:20 smithi117 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:45:21.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:20 smithi117 ceph-mon[35081]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.496+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/3328066356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 msgr2=0x7f5d581020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.496+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/3328066356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d581020b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009900 tx=0x7f5d4802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/3328066356 shutdown_connections 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/3328066356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d58107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/3328066356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 0x7f5d5810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/3328066356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d581020b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/3328066356 >> 172.21.15.96:0/3328066356 conn(0x7f5d580fd700 msgr2=0x7f5d580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/3328066356 shutdown_connections 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/3328066356 wait complete. 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 Processor -- start 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- start start 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d581997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 0x7f5d5819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58111ce0 con 0x7f5d58101cd0 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58111b60 con 0x7f5d581076e0 2024-09-10T22:45:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5ef6a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58111e60 con 0x7f5d58108060 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.497+0000 7f5d5cd06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32974/0 (socket says 172.21.15.96:32974) 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 -- 172.21.15.96:0/1981253172 learned_addr learned my addr 172.21.15.96:0/1981253172 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5d507640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 0x7f5d5819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d57fff640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d581997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 -- 172.21.15.96:0/1981253172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 msgr2=0x7f5d5819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 0x7f5d5819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 -- 172.21.15.96:0/1981253172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 msgr2=0x7f5d581997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d581997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 -- 172.21.15.96:0/1981253172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5819f340 con 0x7f5d58101cd0 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d57fff640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d581997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:21.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5cd06640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005940 tx=0x7f5d480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4803f030 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5819f5d0 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d48003cb0 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48035070 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.498+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581a5cf0 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.499+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d581020b0 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.501+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f5d4803bbf0 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.501+0000 7f5d55ffb640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 0x7f5d400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.502+0000 7f5d57fff640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 0x7f5d400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.502+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5d48106640 con 0x7f5d58101cd0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.502+0000 7f5d57fff640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 0x7f5d400b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d58107390 tx=0x7f5d4c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:21.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.506+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5d480cd650 con 0x7f5d58101cd0 2024-09-10T22:45:21.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.681+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d5810ae50 con 0x7f5d58101cd0 2024-09-10T22:45:21.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.683+0000 7f5d55ffb640 1 -- 172.21.15.96:0/1981253172 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22291 (secure 0 0 0) 0x7f5d480ccf50 con 0x7f5d58101cd0 2024-09-10T22:45:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:21.685 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:19.551740+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:45:21.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 msgr2=0x7f5d400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:21.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 0x7f5d400b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5d58107390 tx=0x7f5d4c005e50 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 msgr2=0x7f5d58199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005940 tx=0x7f5d480042f0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 shutdown_connections 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d581076e0 0x7f5d581997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d400b52e0 0x7f5d400b77a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d58108060 0x7f5d5819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 --2- 172.21.15.96:0/1981253172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d58101cd0 0x7f5d58199290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 >> 172.21.15.96:0/1981253172 conn(0x7f5d580fd700 msgr2=0x7f5d5806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:21.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 shutdown_connections 2024-09-10T22:45:21.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:21.685+0000 7f5d5ef6a640 1 -- 172.21.15.96:0/1981253172 wait complete. 2024-09-10T22:45:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:21 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:active 2024-09-10T22:45:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:21 smithi096 ceph-mon[30869]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:21 smithi096 ceph-mon[30869]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:21 smithi096 ceph-mon[30869]: pgmap v287: 97 pgs: 36 creating+peering, 10 unknown, 51 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 233 B/s wr, 4 op/s 2024-09-10T22:45:22.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:21 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1981253172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:22.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:21 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:active 2024-09-10T22:45:22.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:21 smithi121 ceph-mon[35119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:21 smithi121 ceph-mon[35119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:21 smithi121 ceph-mon[35119]: pgmap v287: 97 pgs: 36 creating+peering, 10 unknown, 51 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 233 B/s wr, 4 op/s 2024-09-10T22:45:22.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:21 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1981253172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:22.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:21 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] up:active 2024-09-10T22:45:22.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:21 smithi117 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:21 smithi117 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:22.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:21 smithi117 ceph-mon[35081]: pgmap v287: 97 pgs: 36 creating+peering, 10 unknown, 51 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 233 B/s wr, 4 op/s 2024-09-10T22:45:22.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:21 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1981253172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:22.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:45:22.658 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:23.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:22 smithi096 ceph-mon[30869]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:45:23.036 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:22 smithi096 ceph-mon[30869]: Cluster is now healthy 2024-09-10T22:45:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.111+0000 7ff82d637640 1 -- 172.21.15.96:0/4085488403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8281076a0 msgr2=0x7ff828107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.111+0000 7ff82d637640 1 --2- 172.21.15.96:0/4085488403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8281076a0 0x7ff828107ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009ca0 tx=0x7ff81c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 -- 172.21.15.96:0/4085488403 shutdown_connections 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 --2- 172.21.15.96:0/4085488403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff828108020 0x7ff82810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 --2- 172.21.15.96:0/4085488403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8281076a0 0x7ff828107ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 --2- 172.21.15.96:0/4085488403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff828102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 -- 172.21.15.96:0/4085488403 >> 172.21.15.96:0/4085488403 conn(0x7ff8280fd700 msgr2=0x7ff8280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 -- 172.21.15.96:0/4085488403 shutdown_connections 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 -- 172.21.15.96:0/4085488403 wait complete. 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.112+0000 7ff82d637640 1 Processor -- start 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 -- start start 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff82819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8281076a0 0x7ff82819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111ce0 con 0x7ff828108020 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111b60 con 0x7ff828101c90 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff82d637640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111e60 con 0x7ff8281076a0 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32988/0 (socket says 172.21.15.96:32988) 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 -- 172.21.15.96:0/1586575787 learned_addr learned my addr 172.21.15.96:0/1586575787 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8267fc640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8281076a0 0x7ff82819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff826ffd640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff82819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 -- 172.21.15.96:0/1586575787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8281076a0 msgr2=0x7ff82819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8281076a0 0x7ff82819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 -- 172.21.15.96:0/1586575787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 msgr2=0x7ff82819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff82819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 -- 172.21.15.96:0/1586575787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a5510 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff826ffd640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff82819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff8277fe640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d750 tx=0x7ff81800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.113+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818017070 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.114+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a5800 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.114+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8180054d0 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.114+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818004c60 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.114+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a5dc0 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.115+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff828102070 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.116+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff818004e00 con 0x7ff828108020 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.116+0000 7ff80ffff640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 0x7ff8100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.117+0000 7ff826ffd640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 0x7ff8100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:23.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.117+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff8180e5460 con 0x7ff828108020 2024-09-10T22:45:23.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.117+0000 7ff826ffd640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 0x7ff8100b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff820009930 tx=0x7ff820008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:23.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.121+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8180ac470 con 0x7ff828108020 2024-09-10T22:45:23.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:22 smithi121 ceph-mon[35119]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:45:23.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:22 smithi121 ceph-mon[35119]: Cluster is now healthy 2024-09-10T22:45:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:22 smithi117 ceph-mon[35081]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:45:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:22 smithi117 ceph-mon[35081]: Cluster is now healthy 2024-09-10T22:45:23.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.299+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff82802e5c0 con 0x7ff828108020 2024-09-10T22:45:23.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.300+0000 7ff80ffff640 1 -- 172.21.15.96:0/1586575787 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22291 (secure 0 0 0) 0x7ff8180abd70 con 0x7ff828108020 2024-09-10T22:45:23.302 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:23.303 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:19.551740+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:45:23.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 msgr2=0x7ff8100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:23.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 0x7ff8100b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff820009930 tx=0x7ff820008040 comp rx=0 tx=0).stop 2024-09-10T22:45:23.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 msgr2=0x7ff82819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d750 tx=0x7ff81800dc10 comp rx=0 tx=0).stop 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 shutdown_connections 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff8100b52e0 0x7ff8100b77a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff828108020 0x7ff82819abd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8281076a0 0x7ff82819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 --2- 172.21.15.96:0/1586575787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828101c90 0x7ff82819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 >> 172.21.15.96:0/1586575787 conn(0x7ff8280fd700 msgr2=0x7ff8280ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 shutdown_connections 2024-09-10T22:45:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:23.303+0000 7ff82d637640 1 -- 172.21.15.96:0/1586575787 wait complete. 2024-09-10T22:45:23.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:45:24.214 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:23 smithi096 ceph-mon[30869]: pgmap v288: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-10T22:45:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:23 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1586575787' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:24.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:23 smithi121 ceph-mon[35119]: pgmap v288: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-10T22:45:24.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:23 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1586575787' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:24.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:23 smithi117 ceph-mon[35081]: pgmap v288: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-10T22:45:24.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:23 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1586575787' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.707+0000 7f4ffeedc640 1 -- 172.21.15.96:0/819600301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff8108020 msgr2=0x7f4ff810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.707+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/819600301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff8108020 0x7f4ff810e820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009ca0 tx=0x7f4ff002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- 172.21.15.96:0/819600301 shutdown_connections 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/819600301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff8108020 0x7f4ff810e820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/819600301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff81076a0 0x7f4ff8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/819600301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8101c90 0x7f4ff8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- 172.21.15.96:0/819600301 >> 172.21.15.96:0/819600301 conn(0x7f4ff80fd700 msgr2=0x7f4ff80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- 172.21.15.96:0/819600301 shutdown_connections 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- 172.21.15.96:0/819600301 wait complete. 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 Processor -- start 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- start start 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff8101c90 0x7f4ff819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 0x7f4ff81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111ce0 con 0x7f4ff81076a0 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111b60 con 0x7f4ff8108020 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ffeedc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111e60 con 0x7f4ff8101c90 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.708+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33010/0 (socket says 172.21.15.96:33010) 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 -- 172.21.15.96:0/1519996045 learned_addr learned my addr 172.21.15.96:0/1519996045 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ffd479640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 0x7f4ff81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ffcc78640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff8101c90 0x7f4ff819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 -- 172.21.15.96:0/1519996045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff8101c90 msgr2=0x7f4ff819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff8101c90 0x7f4ff819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 -- 172.21.15.96:0/1519996045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 msgr2=0x7f4ff81a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 0x7f4ff81a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 -- 172.21.15.96:0/1519996045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81a1570 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ffd479640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 0x7f4ff81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff7fff640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400b4a0 tx=0x7f4fe400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4014070 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe4010ac0 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81a1860 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4017c30 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.709+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a7fa0 con 0x7f4ff81076a0 2024-09-10T22:45:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.710+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff8102070 con 0x7f4ff81076a0 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.712+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f4fe4010610 con 0x7f4ff81076a0 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.712+0000 7f4ff5ffb640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 0x7f4fe80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.712+0000 7f4ffcc78640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 0x7f4fe80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.712+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f4fe40e5420 con 0x7f4ff81076a0 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.713+0000 7f4ffcc78640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 0x7f4fe80b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005fd0 tx=0x7f4fec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:24.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.716+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4fe40ac430 con 0x7f4ff81076a0 2024-09-10T22:45:24.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T22:45:24.303+0000 7f7fc3311640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:45:24.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.926+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4ff81a83d0 con 0x7f4ff81076a0 2024-09-10T22:45:24.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.993+0000 7f4ff5ffb640 1 -- 172.21.15.96:0/1519996045 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4fe40abd30 con 0x7f4ff81076a0 2024-09-10T22:45:24.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 msgr2=0x7f4fe80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 0x7f4fe80b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005fd0 tx=0x7f4fec005850 comp rx=0 tx=0).stop 2024-09-10T22:45:24.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 msgr2=0x7f4ff819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:24.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400b4a0 tx=0x7f4fe400b960 comp rx=0 tx=0).stop 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 shutdown_connections 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ff8108020 0x7f4ff81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4fe80b52e0 0x7f4fe80b77a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff81076a0 0x7f4ff819ba00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 --2- 172.21.15.96:0/1519996045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ff8101c90 0x7f4ff819b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:24.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 >> 172.21.15.96:0/1519996045 conn(0x7f4ff80fd700 msgr2=0x7f4ff806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:24.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 shutdown_connections 2024-09-10T22:45:24.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:24.995+0000 7f4ffeedc640 1 -- 172.21.15.96:0/1519996045 wait complete. 2024-09-10T22:45:25.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:24 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:45:25.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:24 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:45:25.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:24 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:45:25.687 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T22:45:25.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T22:45:25.936 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:26.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:25 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:45:26.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:26 smithi121 ceph-mon[35119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:26.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:26 smithi121 ceph-mon[35119]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T22:45:26.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:26 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:45:26.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:26 smithi117 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:26.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:26 smithi117 ceph-mon[35081]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T22:45:26.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:25 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1519996045' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:45:26.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:25 smithi096 ceph-mon[30869]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:26.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:25 smithi096 ceph-mon[30869]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T22:45:26.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- 172.21.15.96:0/3729601885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c107780 msgr2=0x7fbd5c107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 --2- 172.21.15.96:0/3729601885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c107780 0x7fbd5c107be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009900 tx=0x7fbd4402a410 comp rx=0 tx=0).stop 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- 172.21.15.96:0/3729601885 shutdown_connections 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 --2- 172.21.15.96:0/3729601885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 0x7fbd5c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 --2- 172.21.15.96:0/3729601885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c107780 0x7fbd5c107be0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 --2- 172.21.15.96:0/3729601885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c102920 0x7fbd5c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- 172.21.15.96:0/3729601885 >> 172.21.15.96:0/3729601885 conn(0x7fbd5c076f50 msgr2=0x7fbd5c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- 172.21.15.96:0/3729601885 shutdown_connections 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- 172.21.15.96:0/3729601885 wait complete. 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 Processor -- start 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.418+0000 7fbd60a03640 1 -- start start 2024-09-10T22:45:26.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c107780 0x7fbd5c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 0x7fbd5c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c111ca0 con 0x7fbd5c102920 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c111b20 con 0x7fbd5c107780 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c111e20 con 0x7fbd5c108120 2024-09-10T22:45:26.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40686/0 (socket says 172.21.15.96:40686) 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 -- 172.21.15.96:0/1423716637 learned_addr learned my addr 172.21.15.96:0/1423716637 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd59d9b640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c107780 0x7fbd5c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5ad9d640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 0x7fbd5c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 -- 172.21.15.96:0/1423716637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 msgr2=0x7fbd5c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 0x7fbd5c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 -- 172.21.15.96:0/1423716637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c107780 msgr2=0x7fbd5c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c107780 0x7fbd5c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 -- 172.21.15.96:0/1423716637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c19f310 con 0x7fbd5c102920 2024-09-10T22:45:26.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd5a59c640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c890 tx=0x7fbd4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004220 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd4c0043c0 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c19f600 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.419+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004a00 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.420+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1a5e90 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.420+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c102d00 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.422+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fbd4c0079b0 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.423+0000 7fbd537fe640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 0x7fbd480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.423+0000 7fbd59d9b640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 0x7fbd480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.423+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fbd4c0e5af0 con 0x7fbd5c102920 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.423+0000 7fbd59d9b640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 0x7fbd480b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd44031440 tx=0x7fbd44005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:26.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.425+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbd4c014030 con 0x7fbd5c102920 2024-09-10T22:45:26.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:26.641+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fbd5c1a6130 con 0x7fbd5c102920 2024-09-10T22:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.004+0000 7fbd537fe640 1 -- 172.21.15.96:0/1423716637 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fbd4c0ac530 con 0x7fbd5c102920 2024-09-10T22:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 msgr2=0x7fbd480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 0x7fbd480b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd44031440 tx=0x7fbd44005b50 comp rx=0 tx=0).stop 2024-09-10T22:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 msgr2=0x7fbd5c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c890 tx=0x7fbd4c00cd50 comp rx=0 tx=0).stop 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 shutdown_connections 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd5c107780 0x7fbd5c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd480b5360 0x7fbd480b7820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd5c108120 0x7fbd5c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 --2- 172.21.15.96:0/1423716637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd5c102920 0x7fbd5c199260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 >> 172.21.15.96:0/1423716637 conn(0x7fbd5c076f50 msgr2=0x7fbd5c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 shutdown_connections 2024-09-10T22:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:27.007+0000 7fbd60a03640 1 -- 172.21.15.96:0/1423716637 wait complete. 2024-09-10T22:45:27.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:27 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:45:27.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:27 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:45:27.286 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:27 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:45:27.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T22:45:27.932 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:28.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T22:45:28.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:28 smithi121 ceph-mon[35119]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T22:45:28.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T22:45:28.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:28 smithi117 ceph-mon[35081]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:45:28.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1423716637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T22:45:28.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:28 smithi096 ceph-mon[30869]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- 172.21.15.96:0/904235044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 msgr2=0x7f0d081031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/904235044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d081031a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009ca0 tx=0x7f0cf802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- 172.21.15.96:0/904235044 shutdown_connections 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/904235044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d08108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/904235044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 0x7f0d0810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/904235044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d081031a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- 172.21.15.96:0/904235044 >> 172.21.15.96:0/904235044 conn(0x7f0d080fe830 msgr2=0x7f0d08100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- 172.21.15.96:0/904235044 shutdown_connections 2024-09-10T22:45:28.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- 172.21.15.96:0/904235044 wait complete. 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 Processor -- start 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.443+0000 7f0d0f88b640 1 -- start start 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d081997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 0x7f0d0819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112e60 con 0x7f0d08102dc0 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112ce0 con 0x7f0d081087d0 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112fe0 con 0x7f0d08109150 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40692/0 (socket says 172.21.15.96:40692) 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 -- 172.21.15.96:0/2064187370 learned_addr learned my addr 172.21.15.96:0/2064187370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:28.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0ce26640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d081997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0de28640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 0x7f0d0819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 -- 172.21.15.96:0/2064187370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 msgr2=0x7f0d0819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 0x7f0d0819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 -- 172.21.15.96:0/2064187370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 msgr2=0x7f0d081997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d081997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 -- 172.21.15.96:0/2064187370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0819f320 con 0x7f0d08102dc0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0ce26640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d081997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0d627640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8005e30 tx=0x7f0cf802f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8003bf0 con 0x7f0d08102dc0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf8003d90 con 0x7f0d08102dc0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0819f5b0 con 0x7f0d08102dc0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf802fb10 con 0x7f0d08102dc0 2024-09-10T22:45:28.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.444+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a5cd0 con 0x7f0d08102dc0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.445+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d081031a0 con 0x7f0d08102dc0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.447+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f0cf8004680 con 0x7f0d08102dc0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.447+0000 7f0d067fc640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 0x7f0cf00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.448+0000 7f0d0ce26640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 0x7f0cf00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.448+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f0cf8106fb0 con 0x7f0d08102dc0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.448+0000 7f0d0ce26640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 0x7f0cf00b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0058c0 tx=0x7f0cfc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:28.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.450+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0cf80cdfc0 con 0x7f0d08102dc0 2024-09-10T22:45:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:28.660+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0d081a6010 con 0x7f0d08102dc0 2024-09-10T22:45:29.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.022+0000 7f0d067fc640 1 -- 172.21.15.96:0/2064187370 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f0cf80cd8c0 con 0x7f0d08102dc0 2024-09-10T22:45:29.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 msgr2=0x7f0cf00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 0x7f0cf00b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0058c0 tx=0x7f0cfc005850 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 msgr2=0x7f0d08199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8005e30 tx=0x7f0cf802f3e0 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 shutdown_connections 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d081087d0 0x7f0d081997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0cf00b52e0 0x7f0cf00b77a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d08109150 0x7f0d0819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 --2- 172.21.15.96:0/2064187370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d08102dc0 0x7f0d08199270 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 >> 172.21.15.96:0/2064187370 conn(0x7f0d080fe830 msgr2=0x7f0d08100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 shutdown_connections 2024-09-10T22:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:29.025+0000 7f0d0f88b640 1 -- 172.21.15.96:0/2064187370 wait complete. 2024-09-10T22:45:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:active 2024-09-10T22:45:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:45:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T22:45:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T22:45:29.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T22:45:29.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:45:29.373 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:active 2024-09-10T22:45:29.373 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:45:29.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T22:45:29.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T22:45:29.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T22:45:29.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:29 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:45:29.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] up:active 2024-09-10T22:45:29.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:45:29.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T22:45:29.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T22:45:29.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T22:45:29.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:29 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:45:29.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T22:45:29.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T22:45:29.303+0000 7f7fc3311640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T22:45:29.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:29 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T22:45:29.303+0000 7f7fc3311640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T22:45:29.970 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:45:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:active 2024-09-10T22:45:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:45:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T22:45:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T22:45:30.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T22:45:30.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:30 smithi117 ceph-mon[35081]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-10T22:45:30.351 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:45:30.351 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:active 2024-09-10T22:45:30.352 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:45:30.352 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T22:45:30.352 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T22:45:30.352 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T22:45:30.352 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:30 smithi096 ceph-mon[30869]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-10T22:45:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 -- 172.21.15.96:0/1935063210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f8103b30 msgr2=0x7f35f8103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:30.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/1935063210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f8103b30 0x7f35f8103f10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009900 tx=0x7f35e802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 -- 172.21.15.96:0/1935063210 shutdown_connections 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/1935063210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8104530 0x7f35f810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/1935063210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f810d0e0 0x7f35f810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/1935063210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f8103b30 0x7f35f8103f10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 -- 172.21.15.96:0/1935063210 >> 172.21.15.96:0/1935063210 conn(0x7f35f80fd6c0 msgr2=0x7f35f80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 -- 172.21.15.96:0/1935063210 shutdown_connections 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.432+0000 7f35fd2ed640 1 -- 172.21.15.96:0/1935063210 wait complete. 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 Processor -- start 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 -- start start 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8103b30 0x7f35f81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f8104530 0x7f35f8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:30.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8112e20 con 0x7f35f810d0e0 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8112ca0 con 0x7f35f8103b30 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35fd2ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8112fa0 con 0x7f35f8104530 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40702/0 (socket says 172.21.15.96:40702) 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 -- 172.21.15.96:0/324018658 learned_addr learned my addr 172.21.15.96:0/324018658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f6ffd640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8103b30 0x7f35f81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f67fc640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f8104530 0x7f35f8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 -- 172.21.15.96:0/324018658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f8104530 msgr2=0x7f35f8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f8104530 0x7f35f8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.433+0000 7f35f77fe640 1 -- 172.21.15.96:0/324018658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8103b30 msgr2=0x7f35f81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35f77fe640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8103b30 0x7f35f81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35f77fe640 1 -- 172.21.15.96:0/324018658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f819f4a0 con 0x7f35f810d0e0 2024-09-10T22:45:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35f77fe640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b970 tx=0x7f35f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0019070 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f0014440 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f819f790 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00075d0 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.434+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f81a5ed0 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.435+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f8103f90 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.436+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f35f000cb40 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.437+0000 7f35dffff640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 0x7f35e00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.437+0000 7f35f6ffd640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 0x7f35e00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.437+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f35f00e5730 con 0x7f35f810d0e0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.438+0000 7f35f6ffd640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 0x7f35e00b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031440 tx=0x7f35e8005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.440+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f35f00ac050 con 0x7f35f810d0e0 2024-09-10T22:45:30.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2064187370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] up:active 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T22:45:30.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:30 smithi121 ceph-mon[35119]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-10T22:45:30.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:30.653+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f35f802e5c0 con 0x7f35f810d0e0 2024-09-10T22:45:31.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.042+0000 7f35dffff640 1 -- 172.21.15.96:0/324018658 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f35f00b0a80 con 0x7f35f810d0e0 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 msgr2=0x7f35e00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 0x7f35e00b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031440 tx=0x7f35e8005e80 comp rx=0 tx=0).stop 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 msgr2=0x7f35f819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35f000b970 tx=0x7f35f000be30 comp rx=0 tx=0).stop 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 shutdown_connections 2024-09-10T22:45:31.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35e00b52e0 0x7f35e00b77a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35f810d0e0 0x7f35f819ecd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35f8104530 0x7f35f8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 --2- 172.21.15.96:0/324018658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f8103b30 0x7f35f81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.045+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 >> 172.21.15.96:0/324018658 conn(0x7f35f80fd6c0 msgr2=0x7f35f80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.046+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 shutdown_connections 2024-09-10T22:45:31.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:31.046+0000 7f35fd2ed640 1 -- 172.21.15.96:0/324018658 wait complete. 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:active 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: Cluster is now healthy 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-10T22:45:31.338 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:31 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:45:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:active 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: Cluster is now healthy 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-10T22:45:31.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:31 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:45:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] up:active 2024-09-10T22:45:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:45:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:45:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: Cluster is now healthy 2024-09-10T22:45:31.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T22:45:31.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:45:31.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-10T22:45:31.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:31 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:45:31.695 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-10T22:45:31.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T22:45:31.942 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:32.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:45:32.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:active 2024-09-10T22:45:32.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-mon[30869]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:45:32.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:45:32.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-mon[30869]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 11 op/s 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- 172.21.15.96:0/1633001996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8103b10 msgr2=0x7f09c8103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 --2- 172.21.15.96:0/1633001996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8103b10 0x7f09c8103ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009900 tx=0x7f09b802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- 172.21.15.96:0/1633001996 shutdown_connections 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 --2- 172.21.15.96:0/1633001996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c810d0c0 0x7f09c810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 --2- 172.21.15.96:0/1633001996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8104510 0x7f09c810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 --2- 172.21.15.96:0/1633001996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8103b10 0x7f09c8103ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- 172.21.15.96:0/1633001996 >> 172.21.15.96:0/1633001996 conn(0x7f09c80fd700 msgr2=0x7f09c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- 172.21.15.96:0/1633001996 shutdown_connections 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- 172.21.15.96:0/1633001996 wait complete. 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 Processor -- start 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.429+0000 7f09ccb49640 1 -- start start 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c8103b10 0x7f09c8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c810d0c0 0x7f09c819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112d50 con 0x7f09c8104510 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112bd0 con 0x7f09c8103b10 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112ed0 con 0x7f09c810d0c0 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40720/0 (socket says 172.21.15.96:40720) 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 -- 172.21.15.96:0/3863495811 learned_addr learned my addr 172.21.15.96:0/3863495811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09c659c640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c8103b10 0x7f09c8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09c6d9d640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c810d0c0 0x7f09c819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 -- 172.21.15.96:0/3863495811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c810d0c0 msgr2=0x7f09c819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c810d0c0 0x7f09c819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 -- 172.21.15.96:0/3863495811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c8103b10 msgr2=0x7f09c8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c8103b10 0x7f09c8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 -- 172.21.15.96:0/3863495811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c819f330 con 0x7f09c8104510 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bffff640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f09b000d950 tx=0x7f09b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0017070 con 0x7f09c8104510 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b0009c30 con 0x7f09c8104510 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c819f620 con 0x7f09c8104510 2024-09-10T22:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.430+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b000f620 con 0x7f09c8104510 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.431+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81a5ce0 con 0x7f09c8104510 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.432+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c8103ef0 con 0x7f09c8104510 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.433+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f09b0010460 con 0x7f09c8104510 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.433+0000 7f09bf7fe640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 0x7f09b40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.434+0000 7f09c659c640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 0x7f09b40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.434+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f09b00e5720 con 0x7f09c8104510 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.434+0000 7f09c659c640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 0x7f09b40b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09b8031440 tx=0x7f09b8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.437+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09b00ac730 con 0x7f09c8104510 2024-09-10T22:45:32.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:32 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:45:32.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:32 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:active 2024-09-10T22:45:32.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:32 smithi121 ceph-mon[35119]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:45:32.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:32 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:45:32.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:32 smithi121 ceph-mon[35119]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 11 op/s 2024-09-10T22:45:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:32 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/324018658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:45:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:32 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] up:active 2024-09-10T22:45:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:32 smithi117 ceph-mon[35081]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:45:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:32 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:45:32.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:32 smithi117 ceph-mon[35081]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 11 op/s 2024-09-10T22:45:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.636+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f09c8063590 con 0x7f09b40b52e0 2024-09-10T22:45:32.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.656+0000 7f09bf7fe640 1 -- 172.21.15.96:0/3863495811 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09c8063590 con 0x7f09b40b52e0 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 msgr2=0x7f09b40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 0x7f09b40b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09b8031440 tx=0x7f09b8005b50 comp rx=0 tx=0).stop 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 msgr2=0x7f09c81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f09b000d950 tx=0x7f09b000de10 comp rx=0 tx=0).stop 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 shutdown_connections 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09b40b52e0 0x7f09b40b77a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c810d0c0 0x7f09c819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09c8104510 0x7f09c81997c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 --2- 172.21.15.96:0/3863495811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c8103b10 0x7f09c8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:32.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 >> 172.21.15.96:0/3863495811 conn(0x7f09c80fd700 msgr2=0x7f09c80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:32.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 shutdown_connections 2024-09-10T22:45:32.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:32.658+0000 7f09ccb49640 1 -- 172.21.15.96:0/3863495811 wait complete. 2024-09-10T22:45:32.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:45:32.641+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:45:32.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:45:32.641+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:45:32.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:45:32.641+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:45:32.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:45:32.641+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:45:32.994 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:45:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:45:32.641+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:45:33.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T22:45:33.703 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:33 smithi096 ceph-mon[30869]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:33 smithi096 ceph-mon[30869]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:45:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:33 smithi121 ceph-mon[35119]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:33 smithi121 ceph-mon[35119]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:45:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:33 smithi117 ceph-mon[35081]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:33 smithi117 ceph-mon[35081]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-10T22:45:34.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 -- 172.21.15.96:0/11057526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 msgr2=0x7f6e6c10caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:34.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 --2- 172.21.15.96:0/11057526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c10caf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009900 tx=0x7f6e6002a410 comp rx=0 tx=0).stop 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 -- 172.21.15.96:0/11057526 shutdown_connections 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 --2- 172.21.15.96:0/11057526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 0x7f6e6c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 --2- 172.21.15.96:0/11057526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c10caf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 --2- 172.21.15.96:0/11057526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 0x7f6e6c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 -- 172.21.15.96:0/11057526 >> 172.21.15.96:0/11057526 conn(0x7f6e6c0fd700 msgr2=0x7f6e6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.214+0000 7f6e72918640 1 -- 172.21.15.96:0/11057526 shutdown_connections 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 -- 172.21.15.96:0/11057526 wait complete. 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 Processor -- start 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 -- start start 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 0x7f6e6c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:34.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.215+0000 7f6e72918640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 0x7f6e6c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e72918640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c112df0 con 0x7f6e6c104600 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e72918640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c112c70 con 0x7f6e6c10d140 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e72918640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c112f70 con 0x7f6e6c103ce0 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40752/0 (socket says 172.21.15.96:40752) 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 -- 172.21.15.96:0/3689126400 learned_addr learned my addr 172.21.15.96:0/3689126400 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e70eb5640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 0x7f6e6c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6bfff640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 0x7f6e6c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 -- 172.21.15.96:0/3689126400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 msgr2=0x7f6e6c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 0x7f6e6c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 -- 172.21.15.96:0/3689126400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 msgr2=0x7f6e6c1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 0x7f6e6c1a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 -- 172.21.15.96:0/3689126400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1a1620 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e6b7fe640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e60004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6003f030 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e60004490 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60035070 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1a18b0 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.216+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c1a7fd0 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.217+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c1040c0 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.219+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f6e6003ba30 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.219+0000 7f6e697fa640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 0x7f6e540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.219+0000 7f6e6bfff640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 0x7f6e540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.219+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6e60106480 con 0x7f6e6c104600 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.220+0000 7f6e6bfff640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 0x7f6e540b7820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009990 tx=0x7f6e5c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:34.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.222+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6e600cd470 con 0x7f6e6c104600 2024-09-10T22:45:34.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.367+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f6e600c8fe0 con 0x7f6e6c104600 2024-09-10T22:45:34.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.430+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6e6c1030a0 con 0x7f6e540b5360 2024-09-10T22:45:34.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.453+0000 7f6e697fa640 1 -- 172.21.15.96:0/3689126400 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6e6c1030a0 con 0x7f6e540b5360 2024-09-10T22:45:34.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 msgr2=0x7f6e540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 0x7f6e540b7820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009990 tx=0x7f6e5c009110 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 msgr2=0x7f6e6c19bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e60004060 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 shutdown_connections 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e6c10d140 0x7f6e6c1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e540b5360 0x7f6e540b7820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e6c104600 0x7f6e6c19bab0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 --2- 172.21.15.96:0/3689126400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e6c103ce0 0x7f6e6c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 >> 172.21.15.96:0/3689126400 conn(0x7f6e6c0fd700 msgr2=0x7f6e6c10f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 shutdown_connections 2024-09-10T22:45:34.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:34.456+0000 7f6e72918640 1 -- 172.21.15.96:0/3689126400 wait complete. 2024-09-10T22:45:34.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 22:45:34 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T22:45:34.304+0000 7f7fc3311640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T22:45:35.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T22:45:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:35.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:35 smithi096 ceph-mon[30869]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:45:35.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:35 smithi096 ceph-mon[30869]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:35.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:35 smithi096 ceph-mon[30869]: pgmap v294: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-09-10T22:45:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:35 smithi121 ceph-mon[35119]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:45:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:35 smithi121 ceph-mon[35119]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:35 smithi121 ceph-mon[35119]: pgmap v294: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-09-10T22:45:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:35 smithi117 ceph-mon[35081]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T22:45:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:35 smithi117 ceph-mon[35081]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:35 smithi117 ceph-mon[35081]: pgmap v294: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 4.9 KiB/s wr, 18 op/s 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 -- 172.21.15.96:0/2178507095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101c90 msgr2=0x7f2188102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 --2- 172.21.15.96:0/2178507095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101c90 0x7f2188102070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2178009900 tx=0x7f217802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 -- 172.21.15.96:0/2178507095 shutdown_connections 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 --2- 172.21.15.96:0/2178507095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21881076a0 0x7f2188107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 --2- 172.21.15.96:0/2178507095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2188108020 0x7f218810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 --2- 172.21.15.96:0/2178507095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188101c90 0x7f2188102070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:35.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 -- 172.21.15.96:0/2178507095 >> 172.21.15.96:0/2178507095 conn(0x7f21880fd700 msgr2=0x7f21880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 -- 172.21.15.96:0/2178507095 shutdown_connections 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.829+0000 7f218dc92640 1 -- 172.21.15.96:0/2178507095 wait complete. 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 Processor -- start 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 -- start start 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 0x7f2188198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21881076a0 0x7f2188198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2188111ce0 con 0x7f2188108020 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2188111b60 con 0x7f2188101c90 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f218dc92640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2188111e60 con 0x7f21881076a0 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f2187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f2187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40770/0 (socket says 172.21.15.96:40770) 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f2187fff640 1 -- 172.21.15.96:0/2335728579 learned_addr learned my addr 172.21.15.96:0/2335728579 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:35.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f217ffff640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21881076a0 0x7f2188198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f21877fe640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 0x7f2188198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.830+0000 7f2187fff640 1 -- 172.21.15.96:0/2335728579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21881076a0 msgr2=0x7f2188198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f2187fff640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21881076a0 0x7f2188198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f2187fff640 1 -- 172.21.15.96:0/2335728579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 msgr2=0x7f2188198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f2187fff640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 0x7f2188198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f2187fff640 1 -- 172.21.15.96:0/2335728579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218819e1e0 con 0x7f2188108020 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f21877fe640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 0x7f2188198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f2187fff640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f218000cad0 tx=0x7f218000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000ae20 con 0x7f2188108020 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218819e4d0 con 0x7f2188108020 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2180005020 con 0x7f2188108020 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000f6b0 con 0x7f2188108020 2024-09-10T22:45:35.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.831+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881a4b90 con 0x7f2188108020 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.832+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2188102070 con 0x7f2188108020 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.833+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f21800054f0 con 0x7f2188108020 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.834+0000 7f21857fa640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 0x7f21740b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.834+0000 7f21877fe640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 0x7f21740b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.834+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f21800e6060 con 0x7f2188108020 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.834+0000 7f21877fe640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 0x7f21740b7830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:35.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:35.837+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2180014030 con 0x7f2188108020 2024-09-10T22:45:36.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.040+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2188063590 con 0x7f21740b5370 2024-09-10T22:45:36.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.045+0000 7f21857fa640 1 -- 172.21.15.96:0/2335728579 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2188063590 con 0x7f21740b5370 2024-09-10T22:45:36.046 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_0/b6eefe90-2020-49ea-af47-9ca32e840fc6 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 msgr2=0x7f21740b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 0x7f21740b7830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178031400 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 msgr2=0x7f218819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f218000cad0 tx=0x7f218000cf90 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 shutdown_connections 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f21740b5370 0x7f21740b7830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2188108020 0x7f218819da10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21881076a0 0x7f2188198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 --2- 172.21.15.96:0/2335728579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2188101c90 0x7f2188198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:36.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 >> 172.21.15.96:0/2335728579 conn(0x7f21880fd700 msgr2=0x7f21880ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:36.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 shutdown_connections 2024-09-10T22:45:36.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:36.047+0000 7f218dc92640 1 -- 172.21.15.96:0/2335728579 wait complete. 2024-09-10T22:45:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:36 smithi096 ceph-mon[30869]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:36 smithi121 ceph-mon[35119]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:36 smithi117 ceph-mon[35081]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:36.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T22:45:37.125 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:37.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:37 smithi096 ceph-mon[30869]: pgmap v295: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-10T22:45:37.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1676218002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd41087d0 msgr2=0x7fcdd4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:37.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1676218002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd41087d0 0x7fcdd4108c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009900 tx=0x7fcdc802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:37.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1676218002 shutdown_connections 2024-09-10T22:45:37.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1676218002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4109150 0x7fcdd410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1676218002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd41087d0 0x7fcdd4108c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1676218002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd4102dc0 0x7fcdd41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1676218002 >> 172.21.15.96:0/1676218002 conn(0x7fcdd40fe830 msgr2=0x7fcdd4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1676218002 shutdown_connections 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1676218002 wait complete. 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 Processor -- start 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.638+0000 7fcdda9a4640 1 -- start start 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 0x7fcdd4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd41087d0 0x7fcdd41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4112da0 con 0x7fcdd4109150 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4112c20 con 0x7fcdd41087d0 2024-09-10T22:45:37.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4112f20 con 0x7fcdd4102dc0 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44442/0 (socket says 172.21.15.96:44442) 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 -- 172.21.15.96:0/1077636303 learned_addr learned my addr 172.21.15.96:0/1077636303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd37fe640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd41087d0 0x7fcdd41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd3fff640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 0x7fcdd4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 -- 172.21.15.96:0/1077636303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 msgr2=0x7fcdd4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 0x7fcdd4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 -- 172.21.15.96:0/1077636303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd41087d0 msgr2=0x7fcdd41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd41087d0 0x7fcdd41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 -- 172.21.15.96:0/1077636303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd419f320 con 0x7fcdd4109150 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd3fff640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 0x7fcdd4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:37.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd8f41640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00cad0 tx=0x7fcdcc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00ae20 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc005020 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd419f610 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.639+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00f6b0 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.640+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd41a5cd0 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.640+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd41031a0 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.642+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fcdcc0054f0 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.642+0000 7fcdd17fa640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 0x7fcdbc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.642+0000 7fcdd3fff640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 0x7fcdbc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.643+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fcdcc0e5fe0 con 0x7fcdd4109150 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.643+0000 7fcdd3fff640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 0x7fcdbc0b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40058c0 tx=0x7fcdc4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:37.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.645+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcdcc014030 con 0x7fcdd4109150 2024-09-10T22:45:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:37 smithi121 ceph-mon[35119]: pgmap v295: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-10T22:45:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:37 smithi117 ceph-mon[35081]: pgmap v295: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-10T22:45:37.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.848+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcdd4063590 con 0x7fcdbc0b5370 2024-09-10T22:45:37.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.877+0000 7fcdd17fa640 1 -- 172.21.15.96:0/1077636303 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdd4063590 con 0x7fcdbc0b5370 2024-09-10T22:45:37.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 msgr2=0x7fcdbc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 0x7fcdbc0b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40058c0 tx=0x7fcdc4005ec0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 msgr2=0x7fcdd419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00cad0 tx=0x7fcdcc00cf90 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 shutdown_connections 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcdd41087d0 0x7fcdd41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcdbc0b5370 0x7fcdbc0b7830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdd4109150 0x7fcdd419eb50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 --2- 172.21.15.96:0/1077636303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdd4102dc0 0x7fcdd4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 >> 172.21.15.96:0/1077636303 conn(0x7fcdd40fe830 msgr2=0x7fcdd4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 shutdown_connections 2024-09-10T22:45:37.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:37.879+0000 7fcdda9a4640 1 -- 172.21.15.96:0/1077636303 wait complete. 2024-09-10T22:45:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:38 smithi096 ceph-mon[30869]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:38.588 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T22:45:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:38 smithi121 ceph-mon[35119]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:38.782 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:38 smithi117 ceph-mon[35081]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:39.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 -- 172.21.15.96:0/1666142380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b4108060 msgr2=0x7f54b410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/1666142380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b4108060 0x7f54b410e860 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009ca0 tx=0x7f54ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 -- 172.21.15.96:0/1666142380 shutdown_connections 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/1666142380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b4108060 0x7f54b410e860 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/1666142380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b41076e0 0x7f54b4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/1666142380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 0x7f54b41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 -- 172.21.15.96:0/1666142380 >> 172.21.15.96:0/1666142380 conn(0x7f54b40fd700 msgr2=0x7f54b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 -- 172.21.15.96:0/1666142380 shutdown_connections 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.273+0000 7f54b8a7d640 1 -- 172.21.15.96:0/1666142380 wait complete. 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 Processor -- start 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 -- start start 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 0x7f54b4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b4108060 0x7f54b419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111d20 con 0x7f54b41076e0 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111ba0 con 0x7f54b4101cd0 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b8a7d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4111ea0 con 0x7f54b4108060 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44458/0 (socket says 172.21.15.96:44458) 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54a9d9b640 1 -- 172.21.15.96:0/2757852138 learned_addr learned my addr 172.21.15.96:0/2757852138 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b2d9d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b4108060 0x7f54b419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.274+0000 7f54b259c640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 0x7f54b4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 -- 172.21.15.96:0/2757852138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b4108060 msgr2=0x7f54b419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b4108060 0x7f54b419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 -- 172.21.15.96:0/2757852138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 msgr2=0x7f54b4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 0x7f54b4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 -- 172.21.15.96:0/2757852138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b419e1c0 con 0x7f54b41076e0 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54a9d9b640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b4a0 tx=0x7f549c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c014070 con 0x7f54b41076e0 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c010ac0 con 0x7f54b41076e0 2024-09-10T22:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b419e4b0 con 0x7f54b41076e0 2024-09-10T22:45:39.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017c30 con 0x7f54b41076e0 2024-09-10T22:45:39.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.275+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41a4bf0 con 0x7f54b41076e0 2024-09-10T22:45:39.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.276+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b41020b0 con 0x7f54b41076e0 2024-09-10T22:45:39.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.277+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f549c010610 con 0x7f54b41076e0 2024-09-10T22:45:39.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.278+0000 7f54abfff640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 0x7f54a00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:39.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.278+0000 7f54b259c640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 0x7f54a00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:39.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.278+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f549c0e5220 con 0x7f54b41076e0 2024-09-10T22:45:39.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.278+0000 7f54b259c640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 0x7f54a00b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005e10 tx=0x7f54a4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:39.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.282+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f549c0ac130 con 0x7f54b41076e0 2024-09-10T22:45:39.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.488+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f54b4104cc0 con 0x7f54a00b5370 2024-09-10T22:45:39.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.493+0000 7f54abfff640 1 -- 172.21.15.96:0/2757852138 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f54b4104cc0 con 0x7f54a00b5370 2024-09-10T22:45:39.495 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_1/83316ff1-820e-4d07-b614-9a4f7ce8988b 2024-09-10T22:45:39.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 msgr2=0x7f54a00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 0x7f54a00b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005e10 tx=0x7f54a4005da0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 msgr2=0x7f54b4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b4a0 tx=0x7f549c00b960 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 shutdown_connections 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54a00b5370 0x7f54a00b7830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54b4108060 0x7f54b419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54b41076e0 0x7f54b4198650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 --2- 172.21.15.96:0/2757852138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54b4101cd0 0x7f54b4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 >> 172.21.15.96:0/2757852138 conn(0x7f54b40fd700 msgr2=0x7f54b40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:39.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 shutdown_connections 2024-09-10T22:45:39.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:39.495+0000 7f54b8a7d640 1 -- 172.21.15.96:0/2757852138 wait complete. 2024-09-10T22:45:39.677 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:39 smithi117 ceph-mon[35081]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-09-10T22:45:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:39 smithi096 ceph-mon[30869]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-09-10T22:45:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:39 smithi121 ceph-mon[35119]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-09-10T22:45:40.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:45:40.428 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:40.673 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:40 smithi096 ceph-mon[30869]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:40.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:40 smithi121 ceph-mon[35119]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:40.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:40 smithi117 ceph-mon[35081]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:45:40.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.883+0000 7f3fed24d640 1 -- 172.21.15.96:0/877978293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe81076a0 msgr2=0x7f3fe8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:40.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.883+0000 7f3fed24d640 1 --2- 172.21.15.96:0/877978293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe81076a0 0x7f3fe8107ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009900 tx=0x7f3fd002a410 comp rx=0 tx=0).stop 2024-09-10T22:45:40.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 -- 172.21.15.96:0/877978293 shutdown_connections 2024-09-10T22:45:40.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 --2- 172.21.15.96:0/877978293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8108020 0x7f3fe810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 --2- 172.21.15.96:0/877978293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe81076a0 0x7f3fe8107ae0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 --2- 172.21.15.96:0/877978293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe8101c90 0x7f3fe8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 -- 172.21.15.96:0/877978293 >> 172.21.15.96:0/877978293 conn(0x7f3fe80fd700 msgr2=0x7f3fe80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 -- 172.21.15.96:0/877978293 shutdown_connections 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 -- 172.21.15.96:0/877978293 wait complete. 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.884+0000 7f3fed24d640 1 Processor -- start 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 -- start start 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 0x7f3fe8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe81076a0 0x7f3fe8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe8111ce0 con 0x7f3fe8108020 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe8111b60 con 0x7f3fe8101c90 2024-09-10T22:45:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fed24d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe8111e60 con 0x7f3fe81076a0 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44474/0 (socket says 172.21.15.96:44474) 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 -- 172.21.15.96:0/2495509981 learned_addr learned my addr 172.21.15.96:0/2495509981 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe6d9d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 0x7f3fe8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe659c640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe81076a0 0x7f3fe8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 -- 172.21.15.96:0/2495509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe81076a0 msgr2=0x7f3fe8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe81076a0 0x7f3fe8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 -- 172.21.15.96:0/2495509981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 msgr2=0x7f3fe8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 0x7f3fe8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 -- 172.21.15.96:0/2495509981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe819f3c0 con 0x7f3fe8108020 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe6d9d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 0x7f3fe8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:45:40.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fe759e640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800e970 tx=0x7f3fd800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.885+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800ccc0 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.886+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe819f6b0 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.886+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd800ce60 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.886+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8010430 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.886+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe81a5d70 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.887+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe8102070 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.889+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f3fd8004fb0 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.889+0000 7f3fdffff640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 0x7f3fd40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.890+0000 7f3fe6d9d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 0x7f3fd40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.890+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3fd80e60c0 con 0x7f3fe8108020 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.890+0000 7f3fe6d9d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 0x7f3fd40b7830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009fd0 tx=0x7f3fe0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:40.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:40.893+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3fd8056220 con 0x7f3fe8108020 2024-09-10T22:45:41.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.107+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fe819a440 con 0x7f3fe8108020 2024-09-10T22:45:41.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.107+0000 7f3fdffff640 1 -- 172.21.15.96:0/2495509981 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f3fe819a440 con 0x7f3fe8108020 2024-09-10T22:45:41.109 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:41.109 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:45:41.109 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 msgr2=0x7f3fd40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 0x7f3fd40b7830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009fd0 tx=0x7f3fe0009680 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 msgr2=0x7f3fe819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800e970 tx=0x7f3fd800ed70 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 shutdown_connections 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3fd40b5370 0x7f3fd40b7830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3fe8108020 0x7f3fe819ebf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fe81076a0 0x7f3fe8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 --2- 172.21.15.96:0/2495509981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fe8101c90 0x7f3fe8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:41.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 >> 172.21.15.96:0/2495509981 conn(0x7f3fe80fd700 msgr2=0x7f3fe806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:41.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 shutdown_connections 2024-09-10T22:45:41.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:41.109+0000 7f3fed24d640 1 -- 172.21.15.96:0/2495509981 wait complete. 2024-09-10T22:45:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:41 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2495509981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:41 smithi096 ceph-mon[30869]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-10T22:45:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:41 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2495509981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:41 smithi121 ceph-mon[35119]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-10T22:45:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2495509981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:45:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:41 smithi117 ceph-mon[35081]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-09-10T22:45:41.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:45:42.044 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 -- 172.21.15.96:0/3039844002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 msgr2=0x7f587010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 --2- 172.21.15.96:0/3039844002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587010f950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5860009900 tx=0x7f586002a410 comp rx=0 tx=0).stop 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 -- 172.21.15.96:0/3039844002 shutdown_connections 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 --2- 172.21.15.96:0/3039844002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587010f950 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 --2- 172.21.15.96:0/3039844002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 0x7f5870108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 --2- 172.21.15.96:0/3039844002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 -- 172.21.15.96:0/3039844002 >> 172.21.15.96:0/3039844002 conn(0x7f58700fe830 msgr2=0x7f5870100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 -- 172.21.15.96:0/3039844002 shutdown_connections 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 -- 172.21.15.96:0/3039844002 wait complete. 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.519+0000 7f58756bb640 1 Processor -- start 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 -- start start 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 0x7f5870199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5870112e10 con 0x7f5870109150 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5870112c90 con 0x7f5870102dc0 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f58756bb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5870112f90 con 0x7f58701087d0 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:42.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44502/0 (socket says 172.21.15.96:44502) 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 -- 172.21.15.96:0/616399250 learned_addr learned my addr 172.21.15.96:0/616399250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586e7fc640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 0x7f5870199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586effd640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 -- 172.21.15.96:0/616399250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 msgr2=0x7f5870199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 0x7f5870199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 -- 172.21.15.96:0/616399250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 msgr2=0x7f58701991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586f7fe640 1 -- 172.21.15.96:0/616399250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587019f290 con 0x7f5870109150 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.520+0000 7f586effd640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f586f7fe640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5860004770 tx=0x7f5860004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586003f030 con 0x7f5870109150 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587019f520 con 0x7f5870109150 2024-09-10T22:45:42.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860003aa0 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860035070 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.521+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a5c40 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.522+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58701031a0 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.523+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f586003bcb0 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.524+0000 7f5867fff640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 0x7f585c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.524+0000 7f586effd640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 0x7f585c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.524+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5860106890 con 0x7f5870109150 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.524+0000 7f586effd640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 0x7f585c0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5868005fd0 tx=0x7f58680093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:42.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.527+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58600cd7a0 con 0x7f5870109150 2024-09-10T22:45:42.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.704+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58701a5ea0 con 0x7f5870109150 2024-09-10T22:45:42.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.705+0000 7f5867fff640 1 -- 172.21.15.96:0/616399250 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22291 (secure 0 0 0) 0x7f58600ee820 con 0x7f5870109150 2024-09-10T22:45:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:42.707 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:19.551740+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 msgr2=0x7f585c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 0x7f585c0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5868005fd0 tx=0x7f58680093e0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 msgr2=0x7f587019eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5860004770 tx=0x7f5860004270 comp rx=0 tx=0).stop 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 shutdown_connections 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f585c0b5370 0x7f585c0b7830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5870109150 0x7f587019eac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58701087d0 0x7f5870199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 --2- 172.21.15.96:0/616399250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5870102dc0 0x7f58701991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:42.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.707+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 >> 172.21.15.96:0/616399250 conn(0x7f58700fe830 msgr2=0x7f5870101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:42.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.708+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 shutdown_connections 2024-09-10T22:45:42.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:42.708+0000 7f58756bb640 1 -- 172.21.15.96:0/616399250 wait complete. 2024-09-10T22:45:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:42 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/616399250' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:42 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/616399250' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:42 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/616399250' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:43.367 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:45:43.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:45:43.612 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:43 smithi096 ceph-mon[30869]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-10T22:45:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:43 smithi117 ceph-mon[35081]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-10T22:45:44.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:43 smithi121 ceph-mon[35119]: pgmap v298: 97 pgs: 97 active+clean; 623 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 19 op/s 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 -- 172.21.15.96:0/3560545732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 msgr2=0x7f424410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 --2- 172.21.15.96:0/3560545732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410cb50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f422c009900 tx=0x7f422c02a410 comp rx=0 tx=0).stop 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 -- 172.21.15.96:0/3560545732 shutdown_connections 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 --2- 172.21.15.96:0/3560545732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 0x7f424410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 --2- 172.21.15.96:0/3560545732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410cb50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 --2- 172.21.15.96:0/3560545732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f4244103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 -- 172.21.15.96:0/3560545732 >> 172.21.15.96:0/3560545732 conn(0x7f42440fd700 msgr2=0x7f42440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 -- 172.21.15.96:0/3560545732 shutdown_connections 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.120+0000 7f424a650640 1 -- 172.21.15.96:0/3560545732 wait complete. 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 Processor -- start 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 -- start start 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f424410be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 0x7f424419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112d60 con 0x7f4244104570 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112be0 con 0x7f4244103b70 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f424a650640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112ee0 con 0x7f424410d120 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44516/0 (socket says 172.21.15.96:44516) 2024-09-10T22:45:44.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 -- 172.21.15.96:0/992045548 learned_addr learned my addr 172.21.15.96:0/992045548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f4248bed640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 0x7f424419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f4243fff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f424410be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 -- 172.21.15.96:0/992045548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 msgr2=0x7f424419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 0x7f424419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 -- 172.21.15.96:0/992045548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 msgr2=0x7f424410be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f424410be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f423bfff640 1 -- 172.21.15.96:0/992045548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42441a01e0 con 0x7f4244104570 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.121+0000 7f4243fff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f424410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f423bfff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f422c005940 tx=0x7f422c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c03f030 con 0x7f4244104570 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c003cb0 con 0x7f4244104570 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42441a0470 con 0x7f4244104570 2024-09-10T22:45:44.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c035070 con 0x7f4244104570 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.122+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424419c990 con 0x7f4244104570 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.123+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244103f50 con 0x7f4244104570 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.124+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f422c03bbf0 con 0x7f4244104570 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.125+0000 7f4241ffb640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 0x7f42300b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.125+0000 7f4243fff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 0x7f42300b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.125+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f422c1069b0 con 0x7f4244104570 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.125+0000 7f4243fff640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 0x7f42300b7830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f423400baa0 tx=0x7f4234005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:44.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.128+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f422c0cd8c0 con 0x7f4244104570 2024-09-10T22:45:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.300+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42440ff310 con 0x7f42300b5370 2024-09-10T22:45:44.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.305+0000 7f4241ffb640 1 -- 172.21.15.96:0/992045548 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258881 (secure 0 0 0) 0x7f42440ff310 con 0x7f42300b5370 2024-09-10T22:45:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:44.310 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 msgr2=0x7f42300b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 0x7f42300b7830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f423400baa0 tx=0x7f4234005d90 comp rx=0 tx=0).stop 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 msgr2=0x7f424410c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f422c005940 tx=0x7f422c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 shutdown_connections 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42300b5370 0x7f42300b7830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f424410d120 0x7f424419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4244104570 0x7f424410c3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 --2- 172.21.15.96:0/992045548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4244103b70 0x7f424410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 >> 172.21.15.96:0/992045548 conn(0x7f42440fd700 msgr2=0x7f424410f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 shutdown_connections 2024-09-10T22:45:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:44.308+0000 7f424a650640 1 -- 172.21.15.96:0/992045548 wait complete. 2024-09-10T22:45:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:44 smithi117 ceph-mon[35081]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:45.073 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:44 smithi096 ceph-mon[30869]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:45.101 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-10T22:45:43.302824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638087,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":205,"num_read_kb":181,"num_write":353,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355164,"kb_used_data":10076,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586660,"statfs":{"total":1151940427776,"available":1151576739840,"internally_reserved":0,"allocated":10317824,"data_stored":6574486,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42407,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":129,"num_read_kb":117,"num_write":104,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002860"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962468+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:45:19.962468+0000","last_peered":"2024-09-10T22:45:19.962468+0000","last_clean":"2024-09-10T22:45:19.962468+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:45:19.962468+0000","last_undegraded":"2024-09-10T22:45:19.962468+0000","last_fullsized":"2024-09-10T22:45:19.962468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969633+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:45:19.969633+0000","last_peered":"2024-09-10T22:45:19.969633+0000","last_clean":"2024-09-10T22:45:19.969633+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:45:19.969633+0000","last_undegraded":"2024-09-10T22:45:19.969633+0000","last_fullsized":"2024-09-10T22:45:19.969633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969422+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:45:19.969422+0000","last_peered":"2024-09-10T22:45:19.969422+0000","last_clean":"2024-09-10T22:45:19.969422+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:45:19.969422+0000","last_undegraded":"2024-09-10T22:45:19.969422+0000","last_fullsized":"2024-09-10T22:45:19.969422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966885+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:45:19.966885+0000","last_peered":"2024-09-10T22:45:19.966885+0000","last_clean":"2024-09-10T22:45:19.966885+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:45:19.966885+0000","last_undegraded":"2024-09-10T22:45:19.966885+0000","last_fullsized":"2024-09-10T22:45:19.966885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965493+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:45:19.965493+0000","last_peered":"2024-09-10T22:45:19.965493+0000","last_clean":"2024-09-10T22:45:19.965493+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:45:19.965493+0000","last_undegraded":"2024-09-10T22:45:19.965493+0000","last_fullsized":"2024-09-10T22:45:19.965493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958894+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:45:19.958894+0000","last_peered":"2024-09-10T22:45:19.958894+0000","last_clean":"2024-09-10T22:45:19.958894+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:45:19.958894+0000","last_undegraded":"2024-09-10T22:45:19.958894+0000","last_fullsized":"2024-09-10T22:45:19.958894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965027+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:45:19.965027+0000","last_peered":"2024-09-10T22:45:19.965027+0000","last_clean":"2024-09-10T22:45:19.965027+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:45:19.965027+0000","last_undegraded":"2024-09-10T22:45:19.965027+0000","last_fullsized":"2024-09-10T22:45:19.965027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964367+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:45:19.964367+0000","last_peered":"2024-09-10T22:45:19.964367+0000","last_clean":"2024-09-10T22:45:19.964367+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:45:19.964367+0000","last_undegraded":"2024-09-10T22:45:19.964367+0000","last_fullsized":"2024-09-10T22:45:19.964367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969578+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:45:19.969578+0000","last_peered":"2024-09-10T22:45:19.969578+0000","last_clean":"2024-09-10T22:45:19.969578+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:45:19.969578+0000","last_undegraded":"2024-09-10T22:45:19.969578+0000","last_fullsized":"2024-09-10T22:45:19.969578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959308+0000","last_change":"2024-09-10T22:45:18.949748+0000","last_active":"2024-09-10T22:45:19.959308+0000","last_peered":"2024-09-10T22:45:19.959308+0000","last_clean":"2024-09-10T22:45:19.959308+0000","last_became_active":"2024-09-10T22:45:18.949414+0000","last_became_peered":"2024-09-10T22:45:18.949414+0000","last_unstale":"2024-09-10T22:45:19.959308+0000","last_undegraded":"2024-09-10T22:45:19.959308+0000","last_fullsized":"2024-09-10T22:45:19.959308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:33.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967010+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:45:19.967010+0000","last_peered":"2024-09-10T22:45:19.967010+0000","last_clean":"2024-09-10T22:45:19.967010+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:45:19.967010+0000","last_undegraded":"2024-09-10T22:45:19.967010+0000","last_fullsized":"2024-09-10T22:45:19.967010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967117+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:45:19.967117+0000","last_peered":"2024-09-10T22:45:19.967117+0000","last_clean":"2024-09-10T22:45:19.967117+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:45:19.967117+0000","last_undegraded":"2024-09-10T22:45:19.967117+0000","last_fullsized":"2024-09-10T22:45:19.967117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964910+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:45:19.964910+0000","last_peered":"2024-09-10T22:45:19.964910+0000","last_clean":"2024-09-10T22:45:19.964910+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:45:19.964910+0000","last_undegraded":"2024-09-10T22:45:19.964910+0000","last_fullsized":"2024-09-10T22:45:19.964910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969290+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:45:19.969290+0000","last_peered":"2024-09-10T22:45:19.969290+0000","last_clean":"2024-09-10T22:45:19.969290+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:45:19.969290+0000","last_undegraded":"2024-09-10T22:45:19.969290+0000","last_fullsized":"2024-09-10T22:45:19.969290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969199+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:45:19.969199+0000","last_peered":"2024-09-10T22:45:19.969199+0000","last_clean":"2024-09-10T22:45:19.969199+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:45:19.969199+0000","last_undegraded":"2024-09-10T22:45:19.969199+0000","last_fullsized":"2024-09-10T22:45:19.969199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964661+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:45:19.964661+0000","last_peered":"2024-09-10T22:45:19.964661+0000","last_clean":"2024-09-10T22:45:19.964661+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:45:19.964661+0000","last_undegraded":"2024-09-10T22:45:19.964661+0000","last_fullsized":"2024-09-10T22:45:19.964661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962606+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:45:19.962606+0000","last_peered":"2024-09-10T22:45:19.962606+0000","last_clean":"2024-09-10T22:45:19.962606+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:45:19.962606+0000","last_undegraded":"2024-09-10T22:45:19.962606+0000","last_fullsized":"2024-09-10T22:45:19.962606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959934+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:45:19.959934+0000","last_peered":"2024-09-10T22:45:19.959934+0000","last_clean":"2024-09-10T22:45:19.959934+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:45:19.959934+0000","last_undegraded":"2024-09-10T22:45:19.959934+0000","last_fullsized":"2024-09-10T22:45:19.959934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968136+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:45:19.968136+0000","last_peered":"2024-09-10T22:45:19.968136+0000","last_clean":"2024-09-10T22:45:19.968136+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:45:19.968136+0000","last_undegraded":"2024-09-10T22:45:19.968136+0000","last_fullsized":"2024-09-10T22:45:19.968136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962519+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:45:19.962519+0000","last_peered":"2024-09-10T22:45:19.962519+0000","last_clean":"2024-09-10T22:45:19.962519+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:45:19.962519+0000","last_undegraded":"2024-09-10T22:45:19.962519+0000","last_fullsized":"2024-09-10T22:45:19.962519+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965110+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:45:19.965110+0000","last_peered":"2024-09-10T22:45:19.965110+0000","last_clean":"2024-09-10T22:45:19.965110+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:45:19.965110+0000","last_undegraded":"2024-09-10T22:45:19.965110+0000","last_fullsized":"2024-09-10T22:45:19.965110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965783+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:45:19.965783+0000","last_peered":"2024-09-10T22:45:19.965783+0000","last_clean":"2024-09-10T22:45:19.965783+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:45:19.965783+0000","last_undegraded":"2024-09-10T22:45:19.965783+0000","last_fullsized":"2024-09-10T22:45:19.965783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965374+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:45:19.965374+0000","last_peered":"2024-09-10T22:45:19.965374+0000","last_clean":"2024-09-10T22:45:19.965374+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:45:19.965374+0000","last_undegraded":"2024-09-10T22:45:19.965374+0000","last_fullsized":"2024-09-10T22:45:19.965374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970979+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:45:19.970979+0000","last_peered":"2024-09-10T22:45:19.970979+0000","last_clean":"2024-09-10T22:45:19.970979+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:45:19.970979+0000","last_undegraded":"2024-09-10T22:45:19.970979+0000","last_fullsized":"2024-09-10T22:45:19.970979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965237+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:45:19.965237+0000","last_peered":"2024-09-10T22:45:19.965237+0000","last_clean":"2024-09-10T22:45:19.965237+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:45:19.965237+0000","last_undegraded":"2024-09-10T22:45:19.965237+0000","last_fullsized":"2024-09-10T22:45:19.965237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969102+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:45:19.969102+0000","last_peered":"2024-09-10T22:45:19.969102+0000","last_clean":"2024-09-10T22:45:19.969102+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:45:19.969102+0000","last_undegraded":"2024-09-10T22:45:19.969102+0000","last_fullsized":"2024-09-10T22:45:19.969102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962488+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:45:19.962488+0000","last_peered":"2024-09-10T22:45:19.962488+0000","last_clean":"2024-09-10T22:45:19.962488+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:45:19.962488+0000","last_undegraded":"2024-09-10T22:45:19.962488+0000","last_fullsized":"2024-09-10T22:45:19.962488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964845+0000","last_change":"2024-09-10T22:45:18.943019+0000","last_active":"2024-09-10T22:45:19.964845+0000","last_peered":"2024-09-10T22:45:19.964845+0000","last_clean":"2024-09-10T22:45:19.964845+0000","last_became_active":"2024-09-10T22:45:18.942707+0000","last_became_peered":"2024-09-10T22:45:18.942707+0000","last_unstale":"2024-09-10T22:45:19.964845+0000","last_undegraded":"2024-09-10T22:45:19.964845+0000","last_fullsized":"2024-09-10T22:45:19.964845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:05.709020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970238+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:45:19.970238+0000","last_peered":"2024-09-10T22:45:19.970238+0000","last_clean":"2024-09-10T22:45:19.970238+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:45:19.970238+0000","last_undegraded":"2024-09-10T22:45:19.970238+0000","last_fullsized":"2024-09-10T22:45:19.970238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968538+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:45:19.968538+0000","last_peered":"2024-09-10T22:45:19.968538+0000","last_clean":"2024-09-10T22:45:19.968538+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:45:19.968538+0000","last_undegraded":"2024-09-10T22:45:19.968538+0000","last_fullsized":"2024-09-10T22:45:19.968538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966825+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:45:19.966825+0000","last_peered":"2024-09-10T22:45:19.966825+0000","last_clean":"2024-09-10T22:45:19.966825+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:45:19.966825+0000","last_undegraded":"2024-09-10T22:45:19.966825+0000","last_fullsized":"2024-09-10T22:45:19.966825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962176+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:45:19.962176+0000","last_peered":"2024-09-10T22:45:19.962176+0000","last_clean":"2024-09-10T22:45:19.962176+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:45:19.962176+0000","last_undegraded":"2024-09-10T22:45:19.962176+0000","last_fullsized":"2024-09-10T22:45:19.962176+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965726+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:45:19.965726+0000","last_peered":"2024-09-10T22:45:19.965726+0000","last_clean":"2024-09-10T22:45:19.965726+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:45:19.965726+0000","last_undegraded":"2024-09-10T22:45:19.965726+0000","last_fullsized":"2024-09-10T22:45:19.965726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.073141+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:45:41.073141+0000","last_peered":"2024-09-10T22:45:41.073141+0000","last_clean":"2024-09-10T22:45:41.073141+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:45:41.073141+0000","last_undegraded":"2024-09-10T22:45:41.073141+0000","last_fullsized":"2024-09-10T22:45:41.073141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971237+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:45:19.971237+0000","last_peered":"2024-09-10T22:45:19.971237+0000","last_clean":"2024-09-10T22:45:19.971237+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:45:19.971237+0000","last_undegraded":"2024-09-10T22:45:19.971237+0000","last_fullsized":"2024-09-10T22:45:19.971237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.037546+0000","last_change":"2024-09-10T22:45:16.932623+0000","last_active":"2024-09-10T22:45:29.037546+0000","last_peered":"2024-09-10T22:45:29.037546+0000","last_clean":"2024-09-10T22:45:29.037546+0000","last_became_active":"2024-09-10T22:45:16.932281+0000","last_became_peered":"2024-09-10T22:45:16.932281+0000","last_unstale":"2024-09-10T22:45:29.037546+0000","last_undegraded":"2024-09-10T22:45:29.037546+0000","last_fullsized":"2024-09-10T22:45:29.037546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:27.413508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:28.030416+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:28.030416+0000","last_peered":"2024-09-10T22:45:28.030416+0000","last_clean":"2024-09-10T22:45:28.030416+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:28.030416+0000","last_undegraded":"2024-09-10T22:45:28.030416+0000","last_fullsized":"2024-09-10T22:45:28.030416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964456+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:45:19.964456+0000","last_peered":"2024-09-10T22:45:19.964456+0000","last_clean":"2024-09-10T22:45:19.964456+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:45:19.964456+0000","last_undegraded":"2024-09-10T22:45:19.964456+0000","last_fullsized":"2024-09-10T22:45:19.964456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052698+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:30.052698+0000","last_peered":"2024-09-10T22:45:30.052698+0000","last_clean":"2024-09-10T22:45:30.052698+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:30.052698+0000","last_undegraded":"2024-09-10T22:45:30.052698+0000","last_fullsized":"2024-09-10T22:45:30.052698+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959615+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:45:19.959615+0000","last_peered":"2024-09-10T22:45:19.959615+0000","last_clean":"2024-09-10T22:45:19.959615+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:45:19.959615+0000","last_undegraded":"2024-09-10T22:45:19.959615+0000","last_fullsized":"2024-09-10T22:45:19.959615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:38.068553+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:45:38.068553+0000","last_peered":"2024-09-10T22:45:38.068553+0000","last_clean":"2024-09-10T22:45:38.068553+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:45:38.068553+0000","last_undegraded":"2024-09-10T22:45:38.068553+0000","last_fullsized":"2024-09-10T22:45:38.068553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968394+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:45:19.968394+0000","last_peered":"2024-09-10T22:45:19.968394+0000","last_clean":"2024-09-10T22:45:19.968394+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:45:19.968394+0000","last_undegraded":"2024-09-10T22:45:19.968394+0000","last_fullsized":"2024-09-10T22:45:19.968394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054170+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:30.054170+0000","last_peered":"2024-09-10T22:45:30.054170+0000","last_clean":"2024-09-10T22:45:30.054170+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:30.054170+0000","last_undegraded":"2024-09-10T22:45:30.054170+0000","last_fullsized":"2024-09-10T22:45:30.054170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970192+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:45:19.970192+0000","last_peered":"2024-09-10T22:45:19.970192+0000","last_clean":"2024-09-10T22:45:19.970192+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:45:19.970192+0000","last_undegraded":"2024-09-10T22:45:19.970192+0000","last_fullsized":"2024-09-10T22:45:19.970192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.036434+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:29.036434+0000","last_peered":"2024-09-10T22:45:29.036434+0000","last_clean":"2024-09-10T22:45:29.036434+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:29.036434+0000","last_undegraded":"2024-09-10T22:45:29.036434+0000","last_fullsized":"2024-09-10T22:45:29.036434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968986+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:45:19.968986+0000","last_peered":"2024-09-10T22:45:19.968986+0000","last_clean":"2024-09-10T22:45:19.968986+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:45:19.968986+0000","last_undegraded":"2024-09-10T22:45:19.968986+0000","last_fullsized":"2024-09-10T22:45:19.968986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'22","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:38.070091+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:45:38.070091+0000","last_peered":"2024-09-10T22:45:38.070091+0000","last_clean":"2024-09-10T22:45:38.070091+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:45:38.070091+0000","last_undegraded":"2024-09-10T22:45:38.070091+0000","last_fullsized":"2024-09-10T22:45:38.070091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41063,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":50,"num_write":24,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966242+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:45:19.966242+0000","last_peered":"2024-09-10T22:45:19.966242+0000","last_clean":"2024-09-10T22:45:19.966242+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:45:19.966242+0000","last_undegraded":"2024-09-10T22:45:19.966242+0000","last_fullsized":"2024-09-10T22:45:19.966242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.055849+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:31.055849+0000","last_peered":"2024-09-10T22:45:31.055849+0000","last_clean":"2024-09-10T22:45:31.055849+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:31.055849+0000","last_undegraded":"2024-09-10T22:45:31.055849+0000","last_fullsized":"2024-09-10T22:45:31.055849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964611+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:45:19.964611+0000","last_peered":"2024-09-10T22:45:19.964611+0000","last_clean":"2024-09-10T22:45:19.964611+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:45:19.964611+0000","last_undegraded":"2024-09-10T22:45:19.964611+0000","last_fullsized":"2024-09-10T22:45:19.964611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052801+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:30.052801+0000","last_peered":"2024-09-10T22:45:30.052801+0000","last_clean":"2024-09-10T22:45:30.052801+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:30.052801+0000","last_undegraded":"2024-09-10T22:45:30.052801+0000","last_fullsized":"2024-09-10T22:45:30.052801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965982+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:45:19.965982+0000","last_peered":"2024-09-10T22:45:19.965982+0000","last_clean":"2024-09-10T22:45:19.965982+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:45:19.965982+0000","last_undegraded":"2024-09-10T22:45:19.965982+0000","last_fullsized":"2024-09-10T22:45:19.965982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:37.067183+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:45:37.067183+0000","last_peered":"2024-09-10T22:45:37.067183+0000","last_clean":"2024-09-10T22:45:37.067183+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:45:37.067183+0000","last_undegraded":"2024-09-10T22:45:37.067183+0000","last_fullsized":"2024-09-10T22:45:37.067183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965438+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:45:19.965438+0000","last_peered":"2024-09-10T22:45:19.965438+0000","last_clean":"2024-09-10T22:45:19.965438+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:45:19.965438+0000","last_undegraded":"2024-09-10T22:45:19.965438+0000","last_fullsized":"2024-09-10T22:45:19.965438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052098+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:30.052098+0000","last_peered":"2024-09-10T22:45:30.052098+0000","last_clean":"2024-09-10T22:45:30.052098+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:30.052098+0000","last_undegraded":"2024-09-10T22:45:30.052098+0000","last_fullsized":"2024-09-10T22:45:30.052098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971123+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:45:19.971123+0000","last_peered":"2024-09-10T22:45:19.971123+0000","last_clean":"2024-09-10T22:45:19.971123+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:45:19.971123+0000","last_undegraded":"2024-09-10T22:45:19.971123+0000","last_fullsized":"2024-09-10T22:45:19.971123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.025586+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:30.025586+0000","last_peered":"2024-09-10T22:45:30.025586+0000","last_clean":"2024-09-10T22:45:30.025586+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:30.025586+0000","last_undegraded":"2024-09-10T22:45:30.025586+0000","last_fullsized":"2024-09-10T22:45:30.025586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962244+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:45:19.962244+0000","last_peered":"2024-09-10T22:45:19.962244+0000","last_clean":"2024-09-10T22:45:19.962244+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:45:19.962244+0000","last_undegraded":"2024-09-10T22:45:19.962244+0000","last_fullsized":"2024-09-10T22:45:19.962244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.051784+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:30.051784+0000","last_peered":"2024-09-10T22:45:30.051784+0000","last_clean":"2024-09-10T22:45:30.051784+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:30.051784+0000","last_undegraded":"2024-09-10T22:45:30.051784+0000","last_fullsized":"2024-09-10T22:45:30.051784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966650+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:45:19.966650+0000","last_peered":"2024-09-10T22:45:19.966650+0000","last_clean":"2024-09-10T22:45:19.966650+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:45:19.966650+0000","last_undegraded":"2024-09-10T22:45:19.966650+0000","last_fullsized":"2024-09-10T22:45:19.966650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054625+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:30.054625+0000","last_peered":"2024-09-10T22:45:30.054625+0000","last_clean":"2024-09-10T22:45:30.054625+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:30.054625+0000","last_undegraded":"2024-09-10T22:45:30.054625+0000","last_fullsized":"2024-09-10T22:45:30.054625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966333+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:45:19.966333+0000","last_peered":"2024-09-10T22:45:19.966333+0000","last_clean":"2024-09-10T22:45:19.966333+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:45:19.966333+0000","last_undegraded":"2024-09-10T22:45:19.966333+0000","last_fullsized":"2024-09-10T22:45:19.966333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.072867+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:45:41.072867+0000","last_peered":"2024-09-10T22:45:41.072867+0000","last_clean":"2024-09-10T22:45:41.072867+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:45:41.072867+0000","last_undegraded":"2024-09-10T22:45:41.072867+0000","last_fullsized":"2024-09-10T22:45:41.072867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967248+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:45:19.967248+0000","last_peered":"2024-09-10T22:45:19.967248+0000","last_clean":"2024-09-10T22:45:19.967248+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:45:19.967248+0000","last_undegraded":"2024-09-10T22:45:19.967248+0000","last_fullsized":"2024-09-10T22:45:19.967248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965696+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:19.965696+0000","last_peered":"2024-09-10T22:45:19.965696+0000","last_clean":"2024-09-10T22:45:19.965696+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:19.965696+0000","last_undegraded":"2024-09-10T22:45:19.965696+0000","last_fullsized":"2024-09-10T22:45:19.965696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.055797+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:31.055797+0000","last_peered":"2024-09-10T22:45:31.055797+0000","last_clean":"2024-09-10T22:45:31.055797+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:31.055797+0000","last_undegraded":"2024-09-10T22:45:31.055797+0000","last_fullsized":"2024-09-10T22:45:31.055797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965000+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:45:19.965000+0000","last_peered":"2024-09-10T22:45:19.965000+0000","last_clean":"2024-09-10T22:45:19.965000+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:45:19.965000+0000","last_undegraded":"2024-09-10T22:45:19.965000+0000","last_fullsized":"2024-09-10T22:45:19.965000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:28.029817+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:28.029817+0000","last_peered":"2024-09-10T22:45:28.029817+0000","last_clean":"2024-09-10T22:45:28.029817+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:28.029817+0000","last_undegraded":"2024-09-10T22:45:28.029817+0000","last_fullsized":"2024-09-10T22:45:28.029817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964385+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:45:19.964385+0000","last_peered":"2024-09-10T22:45:19.964385+0000","last_clean":"2024-09-10T22:45:19.964385+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:45:19.964385+0000","last_undegraded":"2024-09-10T22:45:19.964385+0000","last_fullsized":"2024-09-10T22:45:19.964385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:38.068070+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:45:38.068070+0000","last_peered":"2024-09-10T22:45:38.068070+0000","last_clean":"2024-09-10T22:45:38.068070+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:45:38.068070+0000","last_undegraded":"2024-09-10T22:45:38.068070+0000","last_fullsized":"2024-09-10T22:45:38.068070+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971192+0000","last_change":"2024-09-10T22:45:18.951851+0000","last_active":"2024-09-10T22:45:19.971192+0000","last_peered":"2024-09-10T22:45:19.971192+0000","last_clean":"2024-09-10T22:45:19.971192+0000","last_became_active":"2024-09-10T22:45:18.951516+0000","last_became_peered":"2024-09-10T22:45:18.951516+0000","last_unstale":"2024-09-10T22:45:19.971192+0000","last_undegraded":"2024-09-10T22:45:19.971192+0000","last_fullsized":"2024-09-10T22:45:19.971192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:00.873675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:38.068616+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:45:38.068616+0000","last_peered":"2024-09-10T22:45:38.068616+0000","last_clean":"2024-09-10T22:45:38.068616+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:45:38.068616+0000","last_undegraded":"2024-09-10T22:45:38.068616+0000","last_fullsized":"2024-09-10T22:45:38.068616+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958650+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:45:19.958650+0000","last_peered":"2024-09-10T22:45:19.958650+0000","last_clean":"2024-09-10T22:45:19.958650+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:45:19.958650+0000","last_undegraded":"2024-09-10T22:45:19.958650+0000","last_fullsized":"2024-09-10T22:45:19.958650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959826+0000","last_change":"2024-09-10T22:45:16.990874+0000","last_active":"2024-09-10T22:45:19.959826+0000","last_peered":"2024-09-10T22:45:19.959826+0000","last_clean":"2024-09-10T22:45:19.959826+0000","last_became_active":"2024-09-10T22:45:16.990510+0000","last_became_peered":"2024-09-10T22:45:16.990510+0000","last_unstale":"2024-09-10T22:45:19.959826+0000","last_undegraded":"2024-09-10T22:45:19.959826+0000","last_fullsized":"2024-09-10T22:45:19.959826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:37.436782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958947+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:45:19.958947+0000","last_peered":"2024-09-10T22:45:19.958947+0000","last_clean":"2024-09-10T22:45:19.958947+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:45:19.958947+0000","last_undegraded":"2024-09-10T22:45:19.958947+0000","last_fullsized":"2024-09-10T22:45:19.958947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052501+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:30.052501+0000","last_peered":"2024-09-10T22:45:30.052501+0000","last_clean":"2024-09-10T22:45:30.052501+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:30.052501+0000","last_undegraded":"2024-09-10T22:45:30.052501+0000","last_fullsized":"2024-09-10T22:45:30.052501+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965079+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:45:19.965079+0000","last_peered":"2024-09-10T22:45:19.965079+0000","last_clean":"2024-09-10T22:45:19.965079+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:45:19.965079+0000","last_undegraded":"2024-09-10T22:45:19.965079+0000","last_fullsized":"2024-09-10T22:45:19.965079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'12","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:39.965464+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:39.965464+0000","last_peered":"2024-09-10T22:45:39.965464+0000","last_clean":"2024-09-10T22:45:39.965464+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:39.965464+0000","last_undegraded":"2024-09-10T22:45:39.965464+0000","last_fullsized":"2024-09-10T22:45:39.965464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965399+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:45:19.965399+0000","last_peered":"2024-09-10T22:45:19.965399+0000","last_clean":"2024-09-10T22:45:19.965399+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:45:19.965399+0000","last_undegraded":"2024-09-10T22:45:19.965399+0000","last_fullsized":"2024-09-10T22:45:19.965399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052806+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:30.052806+0000","last_peered":"2024-09-10T22:45:30.052806+0000","last_clean":"2024-09-10T22:45:30.052806+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:30.052806+0000","last_undegraded":"2024-09-10T22:45:30.052806+0000","last_fullsized":"2024-09-10T22:45:30.052806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969837+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:45:19.969837+0000","last_peered":"2024-09-10T22:45:19.969837+0000","last_clean":"2024-09-10T22:45:19.969837+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:45:19.969837+0000","last_undegraded":"2024-09-10T22:45:19.969837+0000","last_fullsized":"2024-09-10T22:45:19.969837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:27.021098+0000","last_change":"2024-09-10T22:45:16.990623+0000","last_active":"2024-09-10T22:45:27.021098+0000","last_peered":"2024-09-10T22:45:27.021098+0000","last_clean":"2024-09-10T22:45:27.021098+0000","last_became_active":"2024-09-10T22:45:16.990342+0000","last_became_peered":"2024-09-10T22:45:16.990342+0000","last_unstale":"2024-09-10T22:45:27.021098+0000","last_undegraded":"2024-09-10T22:45:27.021098+0000","last_fullsized":"2024-09-10T22:45:27.021098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:12.470779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966362+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:45:19.966362+0000","last_peered":"2024-09-10T22:45:19.966362+0000","last_clean":"2024-09-10T22:45:19.966362+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:45:19.966362+0000","last_undegraded":"2024-09-10T22:45:19.966362+0000","last_fullsized":"2024-09-10T22:45:19.966362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054187+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:30.054187+0000","last_peered":"2024-09-10T22:45:30.054187+0000","last_clean":"2024-09-10T22:45:30.054187+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:30.054187+0000","last_undegraded":"2024-09-10T22:45:30.054187+0000","last_fullsized":"2024-09-10T22:45:30.054187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962197+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:45:19.962197+0000","last_peered":"2024-09-10T22:45:19.962197+0000","last_clean":"2024-09-10T22:45:19.962197+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:45:19.962197+0000","last_undegraded":"2024-09-10T22:45:19.962197+0000","last_fullsized":"2024-09-10T22:45:19.962197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.038048+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:29.038048+0000","last_peered":"2024-09-10T22:45:29.038048+0000","last_clean":"2024-09-10T22:45:29.038048+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:29.038048+0000","last_undegraded":"2024-09-10T22:45:29.038048+0000","last_fullsized":"2024-09-10T22:45:29.038048+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965043+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:45:19.965043+0000","last_peered":"2024-09-10T22:45:19.965043+0000","last_clean":"2024-09-10T22:45:19.965043+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:45:19.965043+0000","last_undegraded":"2024-09-10T22:45:19.965043+0000","last_fullsized":"2024-09-10T22:45:19.965043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:27.020435+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:27.020435+0000","last_peered":"2024-09-10T22:45:27.020435+0000","last_clean":"2024-09-10T22:45:27.020435+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:27.020435+0000","last_undegraded":"2024-09-10T22:45:27.020435+0000","last_fullsized":"2024-09-10T22:45:27.020435+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965221+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:45:19.965221+0000","last_peered":"2024-09-10T22:45:19.965221+0000","last_clean":"2024-09-10T22:45:19.965221+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:45:19.965221+0000","last_undegraded":"2024-09-10T22:45:19.965221+0000","last_fullsized":"2024-09-10T22:45:19.965221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971942+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:45:19.971942+0000","last_peered":"2024-09-10T22:45:19.971942+0000","last_clean":"2024-09-10T22:45:19.971942+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:45:19.971942+0000","last_undegraded":"2024-09-10T22:45:19.971942+0000","last_fullsized":"2024-09-10T22:45:19.971942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.053034+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:30.053034+0000","last_peered":"2024-09-10T22:45:30.053034+0000","last_clean":"2024-09-10T22:45:30.053034+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:30.053034+0000","last_undegraded":"2024-09-10T22:45:30.053034+0000","last_fullsized":"2024-09-10T22:45:30.053034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967833+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:45:19.967833+0000","last_peered":"2024-09-10T22:45:19.967833+0000","last_clean":"2024-09-10T22:45:19.967833+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:45:19.967833+0000","last_undegraded":"2024-09-10T22:45:19.967833+0000","last_fullsized":"2024-09-10T22:45:19.967833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.048510+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:31.048510+0000","last_peered":"2024-09-10T22:45:31.048510+0000","last_clean":"2024-09-10T22:45:31.048510+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:31.048510+0000","last_undegraded":"2024-09-10T22:45:31.048510+0000","last_fullsized":"2024-09-10T22:45:31.048510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970070+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:45:19.970070+0000","last_peered":"2024-09-10T22:45:19.970070+0000","last_clean":"2024-09-10T22:45:19.970070+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:45:19.970070+0000","last_undegraded":"2024-09-10T22:45:19.970070+0000","last_fullsized":"2024-09-10T22:45:19.970070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.972695+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:19.972695+0000","last_peered":"2024-09-10T22:45:19.972695+0000","last_clean":"2024-09-10T22:45:19.972695+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:19.972695+0000","last_undegraded":"2024-09-10T22:45:19.972695+0000","last_fullsized":"2024-09-10T22:45:19.972695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964827+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:45:19.964827+0000","last_peered":"2024-09-10T22:45:19.964827+0000","last_clean":"2024-09-10T22:45:19.964827+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:45:19.964827+0000","last_undegraded":"2024-09-10T22:45:19.964827+0000","last_fullsized":"2024-09-10T22:45:19.964827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.071902+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:45:41.071902+0000","last_peered":"2024-09-10T22:45:41.071902+0000","last_clean":"2024-09-10T22:45:41.071902+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:45:41.071902+0000","last_undegraded":"2024-09-10T22:45:41.071902+0000","last_fullsized":"2024-09-10T22:45:41.071902+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47485,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":129,"num_read_kb":117,"num_write":228,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":148192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":389066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939669,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1021158,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103196,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":388996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":389374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":981501,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":388713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":981179,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":388574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":388538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":436646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":430689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":390052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:45:45.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:45:45.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:44 smithi121 ceph-mon[35119]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:45.353 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:45.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 -- 172.21.15.96:0/3332898321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178104570 msgr2=0x7fb17810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:45.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 --2- 172.21.15.96:0/3332898321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178104570 0x7fb17810cb50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-09-10T22:45:45.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 -- 172.21.15.96:0/3332898321 shutdown_connections 2024-09-10T22:45:45.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 --2- 172.21.15.96:0/3332898321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb17810d120 0x7fb17810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:45.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 --2- 172.21.15.96:0/3332898321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178104570 0x7fb17810cb50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 --2- 172.21.15.96:0/3332898321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb178103b70 0x7fb178103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 -- 172.21.15.96:0/3332898321 >> 172.21.15.96:0/3332898321 conn(0x7fb1780fd700 msgr2=0x7fb1780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 -- 172.21.15.96:0/3332898321 shutdown_connections 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.831+0000 7fb17f070640 1 -- 172.21.15.96:0/3332898321 wait complete. 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 Processor -- start 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 -- start start 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 0x7fb178199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 0x7fb17819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb178112e60 con 0x7fb178103b70 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb178112ce0 con 0x7fb178104570 2024-09-10T22:45:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17f070640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb178112fe0 con 0x7fb17810d120 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44536/0 (socket says 172.21.15.96:44536) 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 -- 172.21.15.96:0/796355086 learned_addr learned my addr 172.21.15.96:0/796355086 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17d60d640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 0x7fb17819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb177fff640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 0x7fb178199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 -- 172.21.15.96:0/796355086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 msgr2=0x7fb17819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 0x7fb17819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 -- 172.21.15.96:0/796355086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 msgr2=0x7fb178199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 0x7fb178199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17ce0c640 1 -- 172.21.15.96:0/796355086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17819f480 con 0x7fb178103b70 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.832+0000 7fb17d60d640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 0x7fb17819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb177fff640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 0x7fb178199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb17ce0c640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb16800b4a0 tx=0x7fb16800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168014070 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17819f770 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb168010ac0 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168017c30 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.833+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1781a5eb0 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.834+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb178103fd0 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.835+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb168010610 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.836+0000 7fb175ffb640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 0x7fb1640b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.836+0000 7fb177fff640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 0x7fb1640b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.836+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fb1680e5520 con 0x7fb178103b70 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.837+0000 7fb177fff640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 0x7fb1640b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb160031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:45.840+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1680ac430 con 0x7fb178103b70 2024-09-10T22:45:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.011+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb178063590 con 0x7fb1640b5370 2024-09-10T22:45:46.013 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:45 smithi096 ceph-mon[30869]: pgmap v299: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2024-09-10T22:45:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.015+0000 7fb175ffb640 1 -- 172.21.15.96:0/796355086 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258880 (secure 0 0 0) 0x7fb178063590 con 0x7fb1640b5370 2024-09-10T22:45:46.019 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:46.020 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 msgr2=0x7fb1640b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 0x7fb1640b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb160031400 comp rx=0 tx=0).stop 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 msgr2=0x7fb1781993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb16800b4a0 tx=0x7fb16800b960 comp rx=0 tx=0).stop 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 shutdown_connections 2024-09-10T22:45:46.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb178104570 0x7fb178199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb1640b5370 0x7fb1640b7830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb17810d120 0x7fb17819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 --2- 172.21.15.96:0/796355086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb178103b70 0x7fb1781993d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 >> 172.21.15.96:0/796355086 conn(0x7fb1780fd700 msgr2=0x7fb1780fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 shutdown_connections 2024-09-10T22:45:46.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:46.019+0000 7fb17f070640 1 -- 172.21.15.96:0/796355086 wait complete. 2024-09-10T22:45:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:45 smithi121 ceph-mon[35119]: pgmap v299: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2024-09-10T22:45:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:45 smithi117 ceph-mon[35081]: pgmap v299: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 17 op/s 2024-09-10T22:45:46.848 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T22:45:45.303202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643824,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":355,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355180,"kb_used_data":10092,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586644,"statfs":{"total":1151940427776,"available":1151576723456,"internally_reserved":0,"allocated":10334208,"data_stored":6585960,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46178,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":62,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002777"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962468+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:45:19.962468+0000","last_peered":"2024-09-10T22:45:19.962468+0000","last_clean":"2024-09-10T22:45:19.962468+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:45:19.962468+0000","last_undegraded":"2024-09-10T22:45:19.962468+0000","last_fullsized":"2024-09-10T22:45:19.962468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969633+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:45:19.969633+0000","last_peered":"2024-09-10T22:45:19.969633+0000","last_clean":"2024-09-10T22:45:19.969633+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:45:19.969633+0000","last_undegraded":"2024-09-10T22:45:19.969633+0000","last_fullsized":"2024-09-10T22:45:19.969633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969422+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:45:19.969422+0000","last_peered":"2024-09-10T22:45:19.969422+0000","last_clean":"2024-09-10T22:45:19.969422+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:45:19.969422+0000","last_undegraded":"2024-09-10T22:45:19.969422+0000","last_fullsized":"2024-09-10T22:45:19.969422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966885+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:45:19.966885+0000","last_peered":"2024-09-10T22:45:19.966885+0000","last_clean":"2024-09-10T22:45:19.966885+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:45:19.966885+0000","last_undegraded":"2024-09-10T22:45:19.966885+0000","last_fullsized":"2024-09-10T22:45:19.966885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965493+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:45:19.965493+0000","last_peered":"2024-09-10T22:45:19.965493+0000","last_clean":"2024-09-10T22:45:19.965493+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:45:19.965493+0000","last_undegraded":"2024-09-10T22:45:19.965493+0000","last_fullsized":"2024-09-10T22:45:19.965493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958894+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:45:19.958894+0000","last_peered":"2024-09-10T22:45:19.958894+0000","last_clean":"2024-09-10T22:45:19.958894+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:45:19.958894+0000","last_undegraded":"2024-09-10T22:45:19.958894+0000","last_fullsized":"2024-09-10T22:45:19.958894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965027+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:45:19.965027+0000","last_peered":"2024-09-10T22:45:19.965027+0000","last_clean":"2024-09-10T22:45:19.965027+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:45:19.965027+0000","last_undegraded":"2024-09-10T22:45:19.965027+0000","last_fullsized":"2024-09-10T22:45:19.965027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964367+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:45:19.964367+0000","last_peered":"2024-09-10T22:45:19.964367+0000","last_clean":"2024-09-10T22:45:19.964367+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:45:19.964367+0000","last_undegraded":"2024-09-10T22:45:19.964367+0000","last_fullsized":"2024-09-10T22:45:19.964367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969578+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:45:19.969578+0000","last_peered":"2024-09-10T22:45:19.969578+0000","last_clean":"2024-09-10T22:45:19.969578+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:45:19.969578+0000","last_undegraded":"2024-09-10T22:45:19.969578+0000","last_fullsized":"2024-09-10T22:45:19.969578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959308+0000","last_change":"2024-09-10T22:45:18.949748+0000","last_active":"2024-09-10T22:45:19.959308+0000","last_peered":"2024-09-10T22:45:19.959308+0000","last_clean":"2024-09-10T22:45:19.959308+0000","last_became_active":"2024-09-10T22:45:18.949414+0000","last_became_peered":"2024-09-10T22:45:18.949414+0000","last_unstale":"2024-09-10T22:45:19.959308+0000","last_undegraded":"2024-09-10T22:45:19.959308+0000","last_fullsized":"2024-09-10T22:45:19.959308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:33.182950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967010+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:45:19.967010+0000","last_peered":"2024-09-10T22:45:19.967010+0000","last_clean":"2024-09-10T22:45:19.967010+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:45:19.967010+0000","last_undegraded":"2024-09-10T22:45:19.967010+0000","last_fullsized":"2024-09-10T22:45:19.967010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967117+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:45:19.967117+0000","last_peered":"2024-09-10T22:45:19.967117+0000","last_clean":"2024-09-10T22:45:19.967117+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:45:19.967117+0000","last_undegraded":"2024-09-10T22:45:19.967117+0000","last_fullsized":"2024-09-10T22:45:19.967117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964910+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:45:19.964910+0000","last_peered":"2024-09-10T22:45:19.964910+0000","last_clean":"2024-09-10T22:45:19.964910+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:45:19.964910+0000","last_undegraded":"2024-09-10T22:45:19.964910+0000","last_fullsized":"2024-09-10T22:45:19.964910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969290+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:45:19.969290+0000","last_peered":"2024-09-10T22:45:19.969290+0000","last_clean":"2024-09-10T22:45:19.969290+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:45:19.969290+0000","last_undegraded":"2024-09-10T22:45:19.969290+0000","last_fullsized":"2024-09-10T22:45:19.969290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969199+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:45:19.969199+0000","last_peered":"2024-09-10T22:45:19.969199+0000","last_clean":"2024-09-10T22:45:19.969199+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:45:19.969199+0000","last_undegraded":"2024-09-10T22:45:19.969199+0000","last_fullsized":"2024-09-10T22:45:19.969199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964661+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:45:19.964661+0000","last_peered":"2024-09-10T22:45:19.964661+0000","last_clean":"2024-09-10T22:45:19.964661+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:45:19.964661+0000","last_undegraded":"2024-09-10T22:45:19.964661+0000","last_fullsized":"2024-09-10T22:45:19.964661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962606+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:45:19.962606+0000","last_peered":"2024-09-10T22:45:19.962606+0000","last_clean":"2024-09-10T22:45:19.962606+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:45:19.962606+0000","last_undegraded":"2024-09-10T22:45:19.962606+0000","last_fullsized":"2024-09-10T22:45:19.962606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959934+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:45:19.959934+0000","last_peered":"2024-09-10T22:45:19.959934+0000","last_clean":"2024-09-10T22:45:19.959934+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:45:19.959934+0000","last_undegraded":"2024-09-10T22:45:19.959934+0000","last_fullsized":"2024-09-10T22:45:19.959934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968136+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:45:19.968136+0000","last_peered":"2024-09-10T22:45:19.968136+0000","last_clean":"2024-09-10T22:45:19.968136+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:45:19.968136+0000","last_undegraded":"2024-09-10T22:45:19.968136+0000","last_fullsized":"2024-09-10T22:45:19.968136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962519+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:45:19.962519+0000","last_peered":"2024-09-10T22:45:19.962519+0000","last_clean":"2024-09-10T22:45:19.962519+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:45:19.962519+0000","last_undegraded":"2024-09-10T22:45:19.962519+0000","last_fullsized":"2024-09-10T22:45:19.962519+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965110+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:45:19.965110+0000","last_peered":"2024-09-10T22:45:19.965110+0000","last_clean":"2024-09-10T22:45:19.965110+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:45:19.965110+0000","last_undegraded":"2024-09-10T22:45:19.965110+0000","last_fullsized":"2024-09-10T22:45:19.965110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965783+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:45:19.965783+0000","last_peered":"2024-09-10T22:45:19.965783+0000","last_clean":"2024-09-10T22:45:19.965783+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:45:19.965783+0000","last_undegraded":"2024-09-10T22:45:19.965783+0000","last_fullsized":"2024-09-10T22:45:19.965783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965374+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:45:19.965374+0000","last_peered":"2024-09-10T22:45:19.965374+0000","last_clean":"2024-09-10T22:45:19.965374+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:45:19.965374+0000","last_undegraded":"2024-09-10T22:45:19.965374+0000","last_fullsized":"2024-09-10T22:45:19.965374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970979+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:45:19.970979+0000","last_peered":"2024-09-10T22:45:19.970979+0000","last_clean":"2024-09-10T22:45:19.970979+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:45:19.970979+0000","last_undegraded":"2024-09-10T22:45:19.970979+0000","last_fullsized":"2024-09-10T22:45:19.970979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965237+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:45:19.965237+0000","last_peered":"2024-09-10T22:45:19.965237+0000","last_clean":"2024-09-10T22:45:19.965237+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:45:19.965237+0000","last_undegraded":"2024-09-10T22:45:19.965237+0000","last_fullsized":"2024-09-10T22:45:19.965237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969102+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:45:19.969102+0000","last_peered":"2024-09-10T22:45:19.969102+0000","last_clean":"2024-09-10T22:45:19.969102+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:45:19.969102+0000","last_undegraded":"2024-09-10T22:45:19.969102+0000","last_fullsized":"2024-09-10T22:45:19.969102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962488+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:45:19.962488+0000","last_peered":"2024-09-10T22:45:19.962488+0000","last_clean":"2024-09-10T22:45:19.962488+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:45:19.962488+0000","last_undegraded":"2024-09-10T22:45:19.962488+0000","last_fullsized":"2024-09-10T22:45:19.962488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964845+0000","last_change":"2024-09-10T22:45:18.943019+0000","last_active":"2024-09-10T22:45:19.964845+0000","last_peered":"2024-09-10T22:45:19.964845+0000","last_clean":"2024-09-10T22:45:19.964845+0000","last_became_active":"2024-09-10T22:45:18.942707+0000","last_became_peered":"2024-09-10T22:45:18.942707+0000","last_unstale":"2024-09-10T22:45:19.964845+0000","last_undegraded":"2024-09-10T22:45:19.964845+0000","last_fullsized":"2024-09-10T22:45:19.964845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:05.709020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970238+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:45:19.970238+0000","last_peered":"2024-09-10T22:45:19.970238+0000","last_clean":"2024-09-10T22:45:19.970238+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:45:19.970238+0000","last_undegraded":"2024-09-10T22:45:19.970238+0000","last_fullsized":"2024-09-10T22:45:19.970238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968538+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:45:19.968538+0000","last_peered":"2024-09-10T22:45:19.968538+0000","last_clean":"2024-09-10T22:45:19.968538+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:45:19.968538+0000","last_undegraded":"2024-09-10T22:45:19.968538+0000","last_fullsized":"2024-09-10T22:45:19.968538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966825+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:45:19.966825+0000","last_peered":"2024-09-10T22:45:19.966825+0000","last_clean":"2024-09-10T22:45:19.966825+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:45:19.966825+0000","last_undegraded":"2024-09-10T22:45:19.966825+0000","last_fullsized":"2024-09-10T22:45:19.966825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962176+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:45:19.962176+0000","last_peered":"2024-09-10T22:45:19.962176+0000","last_clean":"2024-09-10T22:45:19.962176+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:45:19.962176+0000","last_undegraded":"2024-09-10T22:45:19.962176+0000","last_fullsized":"2024-09-10T22:45:19.962176+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965726+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:45:19.965726+0000","last_peered":"2024-09-10T22:45:19.965726+0000","last_clean":"2024-09-10T22:45:19.965726+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:45:19.965726+0000","last_undegraded":"2024-09-10T22:45:19.965726+0000","last_fullsized":"2024-09-10T22:45:19.965726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.073141+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:45:41.073141+0000","last_peered":"2024-09-10T22:45:41.073141+0000","last_clean":"2024-09-10T22:45:41.073141+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:45:41.073141+0000","last_undegraded":"2024-09-10T22:45:41.073141+0000","last_fullsized":"2024-09-10T22:45:41.073141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971237+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:45:19.971237+0000","last_peered":"2024-09-10T22:45:19.971237+0000","last_clean":"2024-09-10T22:45:19.971237+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:45:19.971237+0000","last_undegraded":"2024-09-10T22:45:19.971237+0000","last_fullsized":"2024-09-10T22:45:19.971237+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.037546+0000","last_change":"2024-09-10T22:45:16.932623+0000","last_active":"2024-09-10T22:45:29.037546+0000","last_peered":"2024-09-10T22:45:29.037546+0000","last_clean":"2024-09-10T22:45:29.037546+0000","last_became_active":"2024-09-10T22:45:16.932281+0000","last_became_peered":"2024-09-10T22:45:16.932281+0000","last_unstale":"2024-09-10T22:45:29.037546+0000","last_undegraded":"2024-09-10T22:45:29.037546+0000","last_fullsized":"2024-09-10T22:45:29.037546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:27.413508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:28.030416+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:28.030416+0000","last_peered":"2024-09-10T22:45:28.030416+0000","last_clean":"2024-09-10T22:45:28.030416+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:28.030416+0000","last_undegraded":"2024-09-10T22:45:28.030416+0000","last_fullsized":"2024-09-10T22:45:28.030416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964456+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:45:19.964456+0000","last_peered":"2024-09-10T22:45:19.964456+0000","last_clean":"2024-09-10T22:45:19.964456+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:45:19.964456+0000","last_undegraded":"2024-09-10T22:45:19.964456+0000","last_fullsized":"2024-09-10T22:45:19.964456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052698+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:30.052698+0000","last_peered":"2024-09-10T22:45:30.052698+0000","last_clean":"2024-09-10T22:45:30.052698+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:30.052698+0000","last_undegraded":"2024-09-10T22:45:30.052698+0000","last_fullsized":"2024-09-10T22:45:30.052698+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959615+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:45:19.959615+0000","last_peered":"2024-09-10T22:45:19.959615+0000","last_clean":"2024-09-10T22:45:19.959615+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:45:19.959615+0000","last_undegraded":"2024-09-10T22:45:19.959615+0000","last_fullsized":"2024-09-10T22:45:19.959615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":49,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:43.075293+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:45:43.075293+0000","last_peered":"2024-09-10T22:45:43.075293+0000","last_clean":"2024-09-10T22:45:43.075293+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:45:43.075293+0000","last_undegraded":"2024-09-10T22:45:43.075293+0000","last_fullsized":"2024-09-10T22:45:43.075293+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968394+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:45:19.968394+0000","last_peered":"2024-09-10T22:45:19.968394+0000","last_clean":"2024-09-10T22:45:19.968394+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:45:19.968394+0000","last_undegraded":"2024-09-10T22:45:19.968394+0000","last_fullsized":"2024-09-10T22:45:19.968394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054170+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:30.054170+0000","last_peered":"2024-09-10T22:45:30.054170+0000","last_clean":"2024-09-10T22:45:30.054170+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:30.054170+0000","last_undegraded":"2024-09-10T22:45:30.054170+0000","last_fullsized":"2024-09-10T22:45:30.054170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970192+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:45:19.970192+0000","last_peered":"2024-09-10T22:45:19.970192+0000","last_clean":"2024-09-10T22:45:19.970192+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:45:19.970192+0000","last_undegraded":"2024-09-10T22:45:19.970192+0000","last_fullsized":"2024-09-10T22:45:19.970192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.036434+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:29.036434+0000","last_peered":"2024-09-10T22:45:29.036434+0000","last_clean":"2024-09-10T22:45:29.036434+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:29.036434+0000","last_undegraded":"2024-09-10T22:45:29.036434+0000","last_fullsized":"2024-09-10T22:45:29.036434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.968986+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:45:19.968986+0000","last_peered":"2024-09-10T22:45:19.968986+0000","last_clean":"2024-09-10T22:45:19.968986+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:45:19.968986+0000","last_undegraded":"2024-09-10T22:45:19.968986+0000","last_fullsized":"2024-09-10T22:45:19.968986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'24","reported_seq":75,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:43.078664+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:45:43.078664+0000","last_peered":"2024-09-10T22:45:43.078664+0000","last_clean":"2024-09-10T22:45:43.078664+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:45:43.078664+0000","last_undegraded":"2024-09-10T22:45:43.078664+0000","last_fullsized":"2024-09-10T22:45:43.078664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46800,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966242+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:45:19.966242+0000","last_peered":"2024-09-10T22:45:19.966242+0000","last_clean":"2024-09-10T22:45:19.966242+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:45:19.966242+0000","last_undegraded":"2024-09-10T22:45:19.966242+0000","last_fullsized":"2024-09-10T22:45:19.966242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.055849+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:31.055849+0000","last_peered":"2024-09-10T22:45:31.055849+0000","last_clean":"2024-09-10T22:45:31.055849+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:31.055849+0000","last_undegraded":"2024-09-10T22:45:31.055849+0000","last_fullsized":"2024-09-10T22:45:31.055849+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964611+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:45:19.964611+0000","last_peered":"2024-09-10T22:45:19.964611+0000","last_clean":"2024-09-10T22:45:19.964611+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:45:19.964611+0000","last_undegraded":"2024-09-10T22:45:19.964611+0000","last_fullsized":"2024-09-10T22:45:19.964611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052801+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:30.052801+0000","last_peered":"2024-09-10T22:45:30.052801+0000","last_clean":"2024-09-10T22:45:30.052801+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:30.052801+0000","last_undegraded":"2024-09-10T22:45:30.052801+0000","last_fullsized":"2024-09-10T22:45:30.052801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965982+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:45:19.965982+0000","last_peered":"2024-09-10T22:45:19.965982+0000","last_clean":"2024-09-10T22:45:19.965982+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:45:19.965982+0000","last_undegraded":"2024-09-10T22:45:19.965982+0000","last_fullsized":"2024-09-10T22:45:19.965982+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:42.073825+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:45:42.073825+0000","last_peered":"2024-09-10T22:45:42.073825+0000","last_clean":"2024-09-10T22:45:42.073825+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:45:42.073825+0000","last_undegraded":"2024-09-10T22:45:42.073825+0000","last_fullsized":"2024-09-10T22:45:42.073825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965438+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:45:19.965438+0000","last_peered":"2024-09-10T22:45:19.965438+0000","last_clean":"2024-09-10T22:45:19.965438+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:45:19.965438+0000","last_undegraded":"2024-09-10T22:45:19.965438+0000","last_fullsized":"2024-09-10T22:45:19.965438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052098+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:30.052098+0000","last_peered":"2024-09-10T22:45:30.052098+0000","last_clean":"2024-09-10T22:45:30.052098+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:30.052098+0000","last_undegraded":"2024-09-10T22:45:30.052098+0000","last_fullsized":"2024-09-10T22:45:30.052098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971123+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:45:19.971123+0000","last_peered":"2024-09-10T22:45:19.971123+0000","last_clean":"2024-09-10T22:45:19.971123+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:45:19.971123+0000","last_undegraded":"2024-09-10T22:45:19.971123+0000","last_fullsized":"2024-09-10T22:45:19.971123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.025586+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:30.025586+0000","last_peered":"2024-09-10T22:45:30.025586+0000","last_clean":"2024-09-10T22:45:30.025586+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:30.025586+0000","last_undegraded":"2024-09-10T22:45:30.025586+0000","last_fullsized":"2024-09-10T22:45:30.025586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962244+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:45:19.962244+0000","last_peered":"2024-09-10T22:45:19.962244+0000","last_clean":"2024-09-10T22:45:19.962244+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:45:19.962244+0000","last_undegraded":"2024-09-10T22:45:19.962244+0000","last_fullsized":"2024-09-10T22:45:19.962244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.051784+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:30.051784+0000","last_peered":"2024-09-10T22:45:30.051784+0000","last_clean":"2024-09-10T22:45:30.051784+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:30.051784+0000","last_undegraded":"2024-09-10T22:45:30.051784+0000","last_fullsized":"2024-09-10T22:45:30.051784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966650+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:45:19.966650+0000","last_peered":"2024-09-10T22:45:19.966650+0000","last_clean":"2024-09-10T22:45:19.966650+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:45:19.966650+0000","last_undegraded":"2024-09-10T22:45:19.966650+0000","last_fullsized":"2024-09-10T22:45:19.966650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054625+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:30.054625+0000","last_peered":"2024-09-10T22:45:30.054625+0000","last_clean":"2024-09-10T22:45:30.054625+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:30.054625+0000","last_undegraded":"2024-09-10T22:45:30.054625+0000","last_fullsized":"2024-09-10T22:45:30.054625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966333+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:45:19.966333+0000","last_peered":"2024-09-10T22:45:19.966333+0000","last_clean":"2024-09-10T22:45:19.966333+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:45:19.966333+0000","last_undegraded":"2024-09-10T22:45:19.966333+0000","last_fullsized":"2024-09-10T22:45:19.966333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.072867+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:45:41.072867+0000","last_peered":"2024-09-10T22:45:41.072867+0000","last_clean":"2024-09-10T22:45:41.072867+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:45:41.072867+0000","last_undegraded":"2024-09-10T22:45:41.072867+0000","last_fullsized":"2024-09-10T22:45:41.072867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967248+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:45:19.967248+0000","last_peered":"2024-09-10T22:45:19.967248+0000","last_clean":"2024-09-10T22:45:19.967248+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:45:19.967248+0000","last_undegraded":"2024-09-10T22:45:19.967248+0000","last_fullsized":"2024-09-10T22:45:19.967248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965696+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:19.965696+0000","last_peered":"2024-09-10T22:45:19.965696+0000","last_clean":"2024-09-10T22:45:19.965696+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:19.965696+0000","last_undegraded":"2024-09-10T22:45:19.965696+0000","last_fullsized":"2024-09-10T22:45:19.965696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.055797+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:31.055797+0000","last_peered":"2024-09-10T22:45:31.055797+0000","last_clean":"2024-09-10T22:45:31.055797+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:31.055797+0000","last_undegraded":"2024-09-10T22:45:31.055797+0000","last_fullsized":"2024-09-10T22:45:31.055797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965000+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:45:19.965000+0000","last_peered":"2024-09-10T22:45:19.965000+0000","last_clean":"2024-09-10T22:45:19.965000+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:45:19.965000+0000","last_undegraded":"2024-09-10T22:45:19.965000+0000","last_fullsized":"2024-09-10T22:45:19.965000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:28.029817+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:28.029817+0000","last_peered":"2024-09-10T22:45:28.029817+0000","last_clean":"2024-09-10T22:45:28.029817+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:28.029817+0000","last_undegraded":"2024-09-10T22:45:28.029817+0000","last_fullsized":"2024-09-10T22:45:28.029817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964385+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:45:19.964385+0000","last_peered":"2024-09-10T22:45:19.964385+0000","last_clean":"2024-09-10T22:45:19.964385+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:45:19.964385+0000","last_undegraded":"2024-09-10T22:45:19.964385+0000","last_fullsized":"2024-09-10T22:45:19.964385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:43.074476+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:45:43.074476+0000","last_peered":"2024-09-10T22:45:43.074476+0000","last_clean":"2024-09-10T22:45:43.074476+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:45:43.074476+0000","last_undegraded":"2024-09-10T22:45:43.074476+0000","last_fullsized":"2024-09-10T22:45:43.074476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971192+0000","last_change":"2024-09-10T22:45:18.951851+0000","last_active":"2024-09-10T22:45:19.971192+0000","last_peered":"2024-09-10T22:45:19.971192+0000","last_clean":"2024-09-10T22:45:19.971192+0000","last_became_active":"2024-09-10T22:45:18.951516+0000","last_became_peered":"2024-09-10T22:45:18.951516+0000","last_unstale":"2024-09-10T22:45:19.971192+0000","last_undegraded":"2024-09-10T22:45:19.971192+0000","last_fullsized":"2024-09-10T22:45:19.971192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:00.873675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:43.075113+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:45:43.075113+0000","last_peered":"2024-09-10T22:45:43.075113+0000","last_clean":"2024-09-10T22:45:43.075113+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:45:43.075113+0000","last_undegraded":"2024-09-10T22:45:43.075113+0000","last_fullsized":"2024-09-10T22:45:43.075113+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958650+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:45:19.958650+0000","last_peered":"2024-09-10T22:45:19.958650+0000","last_clean":"2024-09-10T22:45:19.958650+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:45:19.958650+0000","last_undegraded":"2024-09-10T22:45:19.958650+0000","last_fullsized":"2024-09-10T22:45:19.958650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.959826+0000","last_change":"2024-09-10T22:45:16.990874+0000","last_active":"2024-09-10T22:45:19.959826+0000","last_peered":"2024-09-10T22:45:19.959826+0000","last_clean":"2024-09-10T22:45:19.959826+0000","last_became_active":"2024-09-10T22:45:16.990510+0000","last_became_peered":"2024-09-10T22:45:16.990510+0000","last_unstale":"2024-09-10T22:45:19.959826+0000","last_undegraded":"2024-09-10T22:45:19.959826+0000","last_fullsized":"2024-09-10T22:45:19.959826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:37.436782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.958947+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:45:19.958947+0000","last_peered":"2024-09-10T22:45:19.958947+0000","last_clean":"2024-09-10T22:45:19.958947+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:45:19.958947+0000","last_undegraded":"2024-09-10T22:45:19.958947+0000","last_fullsized":"2024-09-10T22:45:19.958947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052501+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:30.052501+0000","last_peered":"2024-09-10T22:45:30.052501+0000","last_clean":"2024-09-10T22:45:30.052501+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:30.052501+0000","last_undegraded":"2024-09-10T22:45:30.052501+0000","last_fullsized":"2024-09-10T22:45:30.052501+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965079+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:45:19.965079+0000","last_peered":"2024-09-10T22:45:19.965079+0000","last_clean":"2024-09-10T22:45:19.965079+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:45:19.965079+0000","last_undegraded":"2024-09-10T22:45:19.965079+0000","last_fullsized":"2024-09-10T22:45:19.965079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'12","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:39.965464+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:39.965464+0000","last_peered":"2024-09-10T22:45:39.965464+0000","last_clean":"2024-09-10T22:45:39.965464+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:39.965464+0000","last_undegraded":"2024-09-10T22:45:39.965464+0000","last_fullsized":"2024-09-10T22:45:39.965464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965399+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:45:19.965399+0000","last_peered":"2024-09-10T22:45:19.965399+0000","last_clean":"2024-09-10T22:45:19.965399+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:45:19.965399+0000","last_undegraded":"2024-09-10T22:45:19.965399+0000","last_fullsized":"2024-09-10T22:45:19.965399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.052806+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:30.052806+0000","last_peered":"2024-09-10T22:45:30.052806+0000","last_clean":"2024-09-10T22:45:30.052806+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:30.052806+0000","last_undegraded":"2024-09-10T22:45:30.052806+0000","last_fullsized":"2024-09-10T22:45:30.052806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.969837+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:45:19.969837+0000","last_peered":"2024-09-10T22:45:19.969837+0000","last_clean":"2024-09-10T22:45:19.969837+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:45:19.969837+0000","last_undegraded":"2024-09-10T22:45:19.969837+0000","last_fullsized":"2024-09-10T22:45:19.969837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:27.021098+0000","last_change":"2024-09-10T22:45:16.990623+0000","last_active":"2024-09-10T22:45:27.021098+0000","last_peered":"2024-09-10T22:45:27.021098+0000","last_clean":"2024-09-10T22:45:27.021098+0000","last_became_active":"2024-09-10T22:45:16.990342+0000","last_became_peered":"2024-09-10T22:45:16.990342+0000","last_unstale":"2024-09-10T22:45:27.021098+0000","last_undegraded":"2024-09-10T22:45:27.021098+0000","last_fullsized":"2024-09-10T22:45:27.021098+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:12.470779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.966362+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:45:19.966362+0000","last_peered":"2024-09-10T22:45:19.966362+0000","last_clean":"2024-09-10T22:45:19.966362+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:45:19.966362+0000","last_undegraded":"2024-09-10T22:45:19.966362+0000","last_fullsized":"2024-09-10T22:45:19.966362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.054187+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:30.054187+0000","last_peered":"2024-09-10T22:45:30.054187+0000","last_clean":"2024-09-10T22:45:30.054187+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:30.054187+0000","last_undegraded":"2024-09-10T22:45:30.054187+0000","last_fullsized":"2024-09-10T22:45:30.054187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.962197+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:45:19.962197+0000","last_peered":"2024-09-10T22:45:19.962197+0000","last_clean":"2024-09-10T22:45:19.962197+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:45:19.962197+0000","last_undegraded":"2024-09-10T22:45:19.962197+0000","last_fullsized":"2024-09-10T22:45:19.962197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:29.038048+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:29.038048+0000","last_peered":"2024-09-10T22:45:29.038048+0000","last_clean":"2024-09-10T22:45:29.038048+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:29.038048+0000","last_undegraded":"2024-09-10T22:45:29.038048+0000","last_fullsized":"2024-09-10T22:45:29.038048+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965043+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:45:19.965043+0000","last_peered":"2024-09-10T22:45:19.965043+0000","last_clean":"2024-09-10T22:45:19.965043+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:45:19.965043+0000","last_undegraded":"2024-09-10T22:45:19.965043+0000","last_fullsized":"2024-09-10T22:45:19.965043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:27.020435+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:27.020435+0000","last_peered":"2024-09-10T22:45:27.020435+0000","last_clean":"2024-09-10T22:45:27.020435+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:27.020435+0000","last_undegraded":"2024-09-10T22:45:27.020435+0000","last_fullsized":"2024-09-10T22:45:27.020435+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.965221+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:45:19.965221+0000","last_peered":"2024-09-10T22:45:19.965221+0000","last_clean":"2024-09-10T22:45:19.965221+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:45:19.965221+0000","last_undegraded":"2024-09-10T22:45:19.965221+0000","last_fullsized":"2024-09-10T22:45:19.965221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.971942+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:45:19.971942+0000","last_peered":"2024-09-10T22:45:19.971942+0000","last_clean":"2024-09-10T22:45:19.971942+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:45:19.971942+0000","last_undegraded":"2024-09-10T22:45:19.971942+0000","last_fullsized":"2024-09-10T22:45:19.971942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:30.053034+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:30.053034+0000","last_peered":"2024-09-10T22:45:30.053034+0000","last_clean":"2024-09-10T22:45:30.053034+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:30.053034+0000","last_undegraded":"2024-09-10T22:45:30.053034+0000","last_fullsized":"2024-09-10T22:45:30.053034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.967833+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:45:19.967833+0000","last_peered":"2024-09-10T22:45:19.967833+0000","last_clean":"2024-09-10T22:45:19.967833+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:45:19.967833+0000","last_undegraded":"2024-09-10T22:45:19.967833+0000","last_fullsized":"2024-09-10T22:45:19.967833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:31.048510+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:31.048510+0000","last_peered":"2024-09-10T22:45:31.048510+0000","last_clean":"2024-09-10T22:45:31.048510+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:31.048510+0000","last_undegraded":"2024-09-10T22:45:31.048510+0000","last_fullsized":"2024-09-10T22:45:31.048510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.970070+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:45:19.970070+0000","last_peered":"2024-09-10T22:45:19.970070+0000","last_clean":"2024-09-10T22:45:19.970070+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:45:19.970070+0000","last_undegraded":"2024-09-10T22:45:19.970070+0000","last_fullsized":"2024-09-10T22:45:19.970070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.972695+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:19.972695+0000","last_peered":"2024-09-10T22:45:19.972695+0000","last_clean":"2024-09-10T22:45:19.972695+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:19.972695+0000","last_undegraded":"2024-09-10T22:45:19.972695+0000","last_fullsized":"2024-09-10T22:45:19.972695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:19.964827+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:45:19.964827+0000","last_peered":"2024-09-10T22:45:19.964827+0000","last_clean":"2024-09-10T22:45:19.964827+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:45:19.964827+0000","last_undegraded":"2024-09-10T22:45:19.964827+0000","last_fullsized":"2024-09-10T22:45:19.964827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T22:45:41.071902+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:45:41.071902+0000","last_peered":"2024-09-10T22:45:41.071902+0000","last_clean":"2024-09-10T22:45:41.071902+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:45:41.071902+0000","last_undegraded":"2024-09-10T22:45:41.071902+0000","last_fullsized":"2024-09-10T22:45:41.071902+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53222,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":230,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":159666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":389066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":1026895,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103196,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":388996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":389374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495654,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":981501,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":388713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":981179,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":388574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":388538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":436646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":436426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":390052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:45:46.852 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:45:46.852 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T22:45:46.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T22:45:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:46 smithi117 ceph-mon[35081]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:47.102 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:47.222 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:46 smithi096 ceph-mon[30869]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:46 smithi121 ceph-mon[35119]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:45:47.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 -- 172.21.15.96:0/231178953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 msgr2=0x7f14e41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 --2- 172.21.15.96:0/231178953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41069f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009ca0 tx=0x7f14dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 -- 172.21.15.96:0/231178953 shutdown_connections 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 --2- 172.21.15.96:0/231178953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e4105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 --2- 172.21.15.96:0/231178953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41069f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 --2- 172.21.15.96:0/231178953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 0x7f14e4103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 -- 172.21.15.96:0/231178953 >> 172.21.15.96:0/231178953 conn(0x7f14e40fd220 msgr2=0x7f14e40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 -- 172.21.15.96:0/231178953 shutdown_connections 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.583+0000 7f14ebfec640 1 -- 172.21.15.96:0/231178953 wait complete. 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 Processor -- start 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 -- start start 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 0x7f14e419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114d00 con 0x7f14e4106540 2024-09-10T22:45:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114b80 con 0x7f14e4103990 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ebfec640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114e80 con 0x7f14e406c0e0 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52586/0 (socket says 172.21.15.96:52586) 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 -- 172.21.15.96:0/3446454577 learned_addr learned my addr 172.21.15.96:0/3446454577 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14e9587640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 -- 172.21.15.96:0/3446454577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 msgr2=0x7f14e419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14e9d88640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 0x7f14e419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 0x7f14e419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 -- 172.21.15.96:0/3446454577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 msgr2=0x7f14e419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14ea589640 1 -- 172.21.15.96:0/3446454577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e41a0f60 con 0x7f14e4106540 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.584+0000 7f14e9587640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14ea589640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc003e10 con 0x7f14e4106540 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc0044f0 con 0x7f14e4106540 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e41a11f0 con 0x7f14e4106540 2024-09-10T22:45:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc02fe50 con 0x7f14e4106540 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.585+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e41a7930 con 0x7f14e4106540 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.586+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e40fed10 con 0x7f14e4106540 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.587+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f14dc004040 con 0x7f14e4106540 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.588+0000 7f14daffd640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 0x7f14d00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.588+0000 7f14e9d88640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 0x7f14d00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.588+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f14dc107410 con 0x7f14e4106540 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.588+0000 7f14e9d88640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 0x7f14d00b7830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14e000a810 tx=0x7f14e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.591+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f14dc0ce320 con 0x7f14e4106540 2024-09-10T22:45:47.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.839+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f14e402e5c0 con 0x7f14e4106540 2024-09-10T22:45:47.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.841+0000 7f14daffd640 1 -- 172.21.15.96:0/3446454577 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f14dc0cdc20 con 0x7f14e4106540 2024-09-10T22:45:47.843 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-10T22:45:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAbzOBmCOsiMhAAlCO+v7tlA56qhLb/sz5OWA== 2024-09-10T22:45:47.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 msgr2=0x7f14d00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:47.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 0x7f14d00b7830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f14e000a810 tx=0x7f14e0008040 comp rx=0 tx=0).stop 2024-09-10T22:45:47.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 msgr2=0x7f14e41a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:47.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc003c20 comp rx=0 tx=0).stop 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 shutdown_connections 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e4103990 0x7f14e419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f14d00b5370 0x7f14d00b7830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14e4106540 0x7f14e41a06c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 --2- 172.21.15.96:0/3446454577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e406c0e0 0x7f14e419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 >> 172.21.15.96:0/3446454577 conn(0x7f14e40fd220 msgr2=0x7f14e4101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 shutdown_connections 2024-09-10T22:45:47.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:47.844+0000 7f14ebfec640 1 -- 172.21.15.96:0/3446454577 wait complete. 2024-09-10T22:45:48.237 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:47 smithi096 ceph-mon[30869]: pgmap v300: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T22:45:48.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:47 smithi121 ceph-mon[35119]: pgmap v300: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T22:45:48.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:47 smithi117 ceph-mon[35081]: pgmap v300: 97 pgs: 97 active+clean; 629 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T22:45:48.615 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:45:48.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T22:45:48.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T22:45:48.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T22:45:48.934 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:48 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:45:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:48 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:45:49.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:48 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:45:49.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:48 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:45:49.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:48 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:45:49.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:48 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3446454577' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 -- 172.21.15.96:0/1484956735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce41076a0 msgr2=0x7f8ce4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 --2- 172.21.15.96:0/1484956735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce41076a0 0x7f8ce4107ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009900 tx=0x7f8cd802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 -- 172.21.15.96:0/1484956735 shutdown_connections 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 --2- 172.21.15.96:0/1484956735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4108020 0x7f8ce410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 --2- 172.21.15.96:0/1484956735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce41076a0 0x7f8ce4107ae0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 --2- 172.21.15.96:0/1484956735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce4101c90 0x7f8ce4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 -- 172.21.15.96:0/1484956735 >> 172.21.15.96:0/1484956735 conn(0x7f8ce40fd700 msgr2=0x7f8ce40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 -- 172.21.15.96:0/1484956735 shutdown_connections 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 -- 172.21.15.96:0/1484956735 wait complete. 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.419+0000 7f8cebafb640 1 Processor -- start 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 -- start start 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4101c90 0x7f8ce41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 0x7f8ce41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111ce0 con 0x7f8ce4108020 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111b60 con 0x7f8ce4101c90 2024-09-10T22:45:49.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cebafb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111e60 con 0x7f8ce41076a0 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52594/0 (socket says 172.21.15.96:52594) 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 -- 172.21.15.96:0/3472234239 learned_addr learned my addr 172.21.15.96:0/3472234239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8ce9897640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4101c90 0x7f8ce41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8ce9096640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 0x7f8ce41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 -- 172.21.15.96:0/3472234239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 msgr2=0x7f8ce41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 0x7f8ce41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 -- 172.21.15.96:0/3472234239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4101c90 msgr2=0x7f8ce41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4101c90 0x7f8ce41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 -- 172.21.15.96:0/3472234239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce419e250 con 0x7f8ce4108020 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8ce9096640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 0x7f8ce41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.420+0000 7f8cea098640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eaa0 tx=0x7f8cdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:49.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.421+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00cdb0 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.421+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce419e540 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.421+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc005020 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.421+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0116d0 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.421+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a4c00 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.422+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce4102070 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.423+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8cdc00f040 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.424+0000 7f8ce2ffd640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 0x7f8ccc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.424+0000 7f8ce9897640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 0x7f8ccc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.424+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f8cdc0e5e30 con 0x7f8ce4108020 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.424+0000 7f8ce9897640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 0x7f8ccc0b7830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009bd0 tx=0x7f8cd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:49.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.427+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8cdc016030 con 0x7f8ce4108020 2024-09-10T22:45:49.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.604+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f8ce402e5c0 con 0x7f8ce4108020 2024-09-10T22:45:49.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.604+0000 7f8ce2ffd640 1 -- 172.21.15.96:0/3472234239 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v49) ==== 86+0+133 (secure 0 0 0) 0x7f8cdc0acd40 con 0x7f8ce4108020 2024-09-10T22:45:49.606 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:49.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 msgr2=0x7f8ccc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 0x7f8ccc0b7830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009bd0 tx=0x7f8cd4005f90 comp rx=0 tx=0).stop 2024-09-10T22:45:49.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 msgr2=0x7f8ce419da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eaa0 tx=0x7f8cdc00ef60 comp rx=0 tx=0).stop 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 shutdown_connections 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8ccc0b5370 0x7f8ccc0b7830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ce4108020 0x7f8ce419da80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ce41076a0 0x7f8ce41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 --2- 172.21.15.96:0/3472234239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce4101c90 0x7f8ce41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 >> 172.21.15.96:0/3472234239 conn(0x7f8ce40fd700 msgr2=0x7f8ce40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 shutdown_connections 2024-09-10T22:45:49.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:49.607+0000 7f8cebafb640 1 -- 172.21.15.96:0/3472234239 wait complete. 2024-09-10T22:45:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:49 smithi096 ceph-mon[30869]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T22:45:49.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3472234239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:45:50.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:49 smithi121 ceph-mon[35119]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T22:45:50.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:49 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3472234239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:45:50.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:49 smithi117 ceph-mon[35081]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-09-10T22:45:50.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:49 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3472234239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:45:50.411 INFO:teuthology.orchestra.run.smithi096.stdout:[{"version":49,"timestamp":"2024-09-10T22:45:09.399174+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-10T22:45:50.412 INFO:tasks.ceph_manager:config epoch is 49 2024-09-10T22:45:50.412 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:45:50.412 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:45:50.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T22:45:50.662 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:51.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- 172.21.15.96:0/1353978950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4108020 msgr2=0x7fc6f410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:51.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/1353978950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4108020 0x7fc6f410e820 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009ca0 tx=0x7fc6ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- 172.21.15.96:0/1353978950 shutdown_connections 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/1353978950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 0x7fc6f4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/1353978950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4108020 0x7fc6f410e820 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/1353978950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4101c90 0x7fc6f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- 172.21.15.96:0/1353978950 >> 172.21.15.96:0/1353978950 conn(0x7fc6f40fd700 msgr2=0x7fc6f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- 172.21.15.96:0/1353978950 shutdown_connections 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- 172.21.15.96:0/1353978950 wait complete. 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 Processor -- start 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.144+0000 7fc6f9dea640 1 -- start start 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 0x7fc6f41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4108020 0x7fc6f4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111ce0 con 0x7fc6f4101c90 2024-09-10T22:45:51.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111b60 con 0x7fc6f41076a0 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111e60 con 0x7fc6f4108020 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52610/0 (socket says 172.21.15.96:52610) 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 -- 172.21.15.96:0/962897836 learned_addr learned my addr 172.21.15.96:0/962897836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f2ffd640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 0x7fc6f41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f3fff640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4108020 0x7fc6f4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 -- 172.21.15.96:0/962897836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4108020 msgr2=0x7fc6f4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4108020 0x7fc6f4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 -- 172.21.15.96:0/962897836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 msgr2=0x7fc6f41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 0x7fc6f41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 -- 172.21.15.96:0/962897836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f41a7680 con 0x7fc6f4101c90 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f37fe640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b4a0 tx=0x7fc6e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4014070 con 0x7fc6f4101c90 2024-09-10T22:45:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41a7970 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4010ac0 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.145+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4017c30 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.146+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41a7f30 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.146+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4102070 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.148+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc6e4010610 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.149+0000 7fc6f0ff9640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 0x7fc6dc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.149+0000 7fc6f2ffd640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 0x7fc6dc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.149+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc6e40e5510 con 0x7fc6f4101c90 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.149+0000 7fc6f2ffd640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 0x7fc6dc0b7830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4103d70 tx=0x7fc6e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.152+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6e40ac420 con 0x7fc6f4101c90 2024-09-10T22:45:51.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.374+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc6f406aec0 con 0x7fc6f4101c90 2024-09-10T22:45:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.379+0000 7fc6f0ff9640 1 -- 172.21.15.96:0/962897836 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+294212 (secure 0 0 0) 0x7fc6f406aec0 con 0x7fc6f4101c90 2024-09-10T22:45:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 msgr2=0x7fc6dc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 0x7fc6dc0b7830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4103d70 tx=0x7fc6e800b390 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 msgr2=0x7fc6f4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b4a0 tx=0x7fc6e400b960 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 shutdown_connections 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f41076a0 0x7fc6f41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc6dc0b5370 0x7fc6dc0b7830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f4108020 0x7fc6f4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 --2- 172.21.15.96:0/962897836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6f4101c90 0x7fc6f4197ea0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 >> 172.21.15.96:0/962897836 conn(0x7fc6f40fd700 msgr2=0x7fc6f40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:51.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 shutdown_connections 2024-09-10T22:45:51.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:51.382+0000 7fc6f9dea640 1 -- 172.21.15.96:0/962897836 wait complete. 2024-09-10T22:45:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:51 smithi096 ceph-mon[30869]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-10T22:45:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:51 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/962897836' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:45:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:51 smithi121 ceph-mon[35119]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-10T22:45:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:51 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/962897836' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:45:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:51 smithi117 ceph-mon[35081]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-10T22:45:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:51 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/962897836' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:45:52.084 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3353555679},{"type":"v1","addr":"172.21.15.96:6801","nonce":3353555679}]},"active_addr":"172.21.15.96:6801/3353555679","active_change":"2024-09-10T22:37:57.209985+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24125,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.96:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3362455818}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2741133105}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":7310156}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2068503473}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":350305554}]}]} 2024-09-10T22:45:52.090 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T22:45:52.090 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T22:45:52.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:45:52.336 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:52.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/3915467192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b8101cd0 msgr2=0x7fb0b81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:52.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/3915467192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b8101cd0 0x7fb0b81020b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009900 tx=0x7fb0a802a410 comp rx=0 tx=0).stop 2024-09-10T22:45:52.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/3915467192 shutdown_connections 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/3915467192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 0x7fb0b810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/3915467192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b81076e0 0x7fb0b8107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/3915467192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b8101cd0 0x7fb0b81020b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/3915467192 >> 172.21.15.96:0/3915467192 conn(0x7fb0b80fd700 msgr2=0x7fb0b80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/3915467192 shutdown_connections 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.820+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/3915467192 wait complete. 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 Processor -- start 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 -- start start 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b8101cd0 0x7fb0b81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:52.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 0x7fb0b819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8111d20 con 0x7fb0b81076e0 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8111ba0 con 0x7fb0b8108060 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0bd2a0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8111ea0 con 0x7fb0b8101cd0 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52630/0 (socket says 172.21.15.96:52630) 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 -- 172.21.15.96:0/1907880449 learned_addr learned my addr 172.21.15.96:0/1907880449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b77fe640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 0x7fb0b819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 -- 172.21.15.96:0/1907880449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b8101cd0 msgr2=0x7fb0b81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b8101cd0 0x7fb0b81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 -- 172.21.15.96:0/1907880449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 msgr2=0x7fb0b819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 0x7fb0b819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:52.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.821+0000 7fb0b67fc640 1 -- 172.21.15.96:0/1907880449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b819f350 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb0b67fc640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d770 tx=0x7fb0ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac017070 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b819f640 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac0054d0 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac004c30 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.822+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a5d00 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.823+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b81020b0 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.824+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb0ac004dd0 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.825+0000 7fb09ffff640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 0x7fb0a00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.825+0000 7fb0b6ffd640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 0x7fb0a00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.825+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fb0ac0e5b40 con 0x7fb0b81076e0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.825+0000 7fb0b6ffd640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 0x7fb0a00b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80096c0 tx=0x7fb0a8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:52.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:52.828+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb0ac0aca50 con 0x7fb0b81076e0 2024-09-10T22:45:53.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.006+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b81a6110 con 0x7fb0b81076e0 2024-09-10T22:45:53.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.007+0000 7fb09ffff640 1 -- 172.21.15.96:0/1907880449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22291 (secure 0 0 0) 0x7fb0ac0b5030 con 0x7fb0b81076e0 2024-09-10T22:45:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:53.009 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:19.551740+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.009+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 msgr2=0x7fb0a00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.009+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 0x7fb0a00b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80096c0 tx=0x7fb0a8031400 comp rx=0 tx=0).stop 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.009+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 msgr2=0x7fb0b81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.009+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d770 tx=0x7fb0ac00dc30 comp rx=0 tx=0).stop 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 shutdown_connections 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b8108060 0x7fb0b819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:53.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb0a00b5370 0x7fb0a00b7830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:53.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0b81076e0 0x7fb0b81997e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:53.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 --2- 172.21.15.96:0/1907880449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b8101cd0 0x7fb0b81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:53.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 >> 172.21.15.96:0/1907880449 conn(0x7fb0b80fd700 msgr2=0x7fb0b806b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:53.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 shutdown_connections 2024-09-10T22:45:53.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:53.010+0000 7fb0bd2a0640 1 -- 172.21.15.96:0/1907880449 wait complete. 2024-09-10T22:45:53.487 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:53 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1907880449' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:53.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:53 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1907880449' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:53 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1907880449' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:53.729 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T22:45:53.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:45:53.976 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:54.377 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:54 smithi096 ceph-mon[30869]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:45:54.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 -- 172.21.15.96:0/1100510171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 msgr2=0x7f63d010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 --2- 172.21.15.96:0/1100510171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d010f950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009ca0 tx=0x7f63c802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 -- 172.21.15.96:0/1100510171 shutdown_connections 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 --2- 172.21.15.96:0/1100510171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d010f950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 --2- 172.21.15.96:0/1100510171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 0x7f63d0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 --2- 172.21.15.96:0/1100510171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 0x7f63d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 -- 172.21.15.96:0/1100510171 >> 172.21.15.96:0/1100510171 conn(0x7f63d00fe830 msgr2=0x7f63d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 -- 172.21.15.96:0/1100510171 shutdown_connections 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.455+0000 7f63d64c5640 1 -- 172.21.15.96:0/1100510171 wait complete. 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 Processor -- start 2024-09-10T22:45:54.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 -- start start 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 0x7f63d01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 0x7f63d0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0112e10 con 0x7f63d0109150 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0112c90 con 0x7f63d0102dc0 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d64c5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0112f90 con 0x7f63d01087d0 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.456+0000 7f63d4a62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52652/0 (socket says 172.21.15.96:52652) 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 -- 172.21.15.96:0/3825340318 learned_addr learned my addr 172.21.15.96:0/3825340318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63cffff640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 0x7f63d01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63cf7fe640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 0x7f63d0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 -- 172.21.15.96:0/3825340318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 msgr2=0x7f63d0199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 0x7f63d0199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 -- 172.21.15.96:0/3825340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 msgr2=0x7f63d01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 0x7f63d01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 -- 172.21.15.96:0/3825340318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d019f370 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d4a62640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005cc0 tx=0x7f63c802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8003bf0 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d019f600 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63c8003d90 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c802fb20 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.457+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d01a5da0 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.458+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d01031a0 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.459+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f63c8004580 con 0x7f63d0109150 2024-09-10T22:45:54.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.460+0000 7f63cd7fa640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 0x7f63b80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:54.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.461+0000 7f63cffff640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 0x7f63b80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:54.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.461+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f63c8107120 con 0x7f63d0109150 2024-09-10T22:45:54.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.461+0000 7f63cffff640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 0x7f63b80b7830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63c000bbe0 tx=0x7f63c000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:54.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.464+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f63c80cdf30 con 0x7f63d0109150 2024-09-10T22:45:54.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:54 smithi121 ceph-mon[35119]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:45:54.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:54 smithi117 ceph-mon[35081]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:45:54.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.642+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63d019a3f0 con 0x7f63d0109150 2024-09-10T22:45:54.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.643+0000 7f63cd7fa640 1 -- 172.21.15.96:0/3825340318 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22291 (secure 0 0 0) 0x7f63c80cd830 con 0x7f63d0109150 2024-09-10T22:45:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:45:54.646 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:19.551740+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:45:54.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 msgr2=0x7f63b80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 0x7f63b80b7830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63c000bbe0 tx=0x7f63c000b390 comp rx=0 tx=0).stop 2024-09-10T22:45:54.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 msgr2=0x7f63d019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:54.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f63c8005cc0 tx=0x7f63c802f380 comp rx=0 tx=0).stop 2024-09-10T22:45:54.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 shutdown_connections 2024-09-10T22:45:54.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63b80b5370 0x7f63b80b7830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63d0109150 0x7f63d019eba0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63d01087d0 0x7f63d0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 --2- 172.21.15.96:0/3825340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d0102dc0 0x7f63d01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 >> 172.21.15.96:0/3825340318 conn(0x7f63d00fe830 msgr2=0x7f63d0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 shutdown_connections 2024-09-10T22:45:54.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:54.646+0000 7f63d64c5640 1 -- 172.21.15.96:0/3825340318 wait complete. 2024-09-10T22:45:55.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:55 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3825340318' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:55.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T22:45:55.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T22:45:55.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T22:45:55.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T22:45:55.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T22:45:55.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T22:45:55.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T22:45:55.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T22:45:55.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T22:45:55.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T22:45:55.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T22:45:55.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T22:45:55.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:55 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3825340318' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:55 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3825340318' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:45:56.021 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.155 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.158 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.159 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.159 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.160 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.160 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.369 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:56 smithi096 ceph-mon[30869]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T22:45:56.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:56 smithi121 ceph-mon[35119]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T22:45:56.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:56 smithi117 ceph-mon[35081]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T22:45:56.590 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:56.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 -- 172.21.15.96:0/2018169466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8108060 msgr2=0x7fefd810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 --2- 172.21.15.96:0/2018169466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8108060 0x7fefd810e860 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009ca0 tx=0x7fefd002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 -- 172.21.15.96:0/2018169466 shutdown_connections 2024-09-10T22:45:56.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 --2- 172.21.15.96:0/2018169466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8108060 0x7fefd810e860 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 --2- 172.21.15.96:0/2018169466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd81076e0 0x7fefd8107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 --2- 172.21.15.96:0/2018169466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8101cd0 0x7fefd81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 -- 172.21.15.96:0/2018169466 >> 172.21.15.96:0/2018169466 conn(0x7fefd80fd740 msgr2=0x7fefd80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 -- 172.21.15.96:0/2018169466 shutdown_connections 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.781+0000 7fefdf937640 1 -- 172.21.15.96:0/2018169466 wait complete. 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 Processor -- start 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 -- start start 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 0x7fefd81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd81076e0 0x7fefd81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8108060 0x7fefd819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111d60 con 0x7fefd8101cd0 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111be0 con 0x7fefd8108060 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdf937640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111ee0 con 0x7fefd81076e0 2024-09-10T22:45:56.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdd6d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 0x7fefd81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdd6d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 0x7fefd81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41556/0 (socket says 172.21.15.96:41556) 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdd6d3640 1 -- 172.21.15.96:0/646483023 learned_addr learned my addr 172.21.15.96:0/646483023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.782+0000 7fefdced2640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd81076e0 0x7fefd81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdded4640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8108060 0x7fefd819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 -- 172.21.15.96:0/646483023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd81076e0 msgr2=0x7fefd81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefd81076e0 0x7fefd81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 -- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8108060 msgr2=0x7fefd819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8108060 0x7fefd819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd819e360 con 0x7fefd8101cd0 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdd6d3640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 0x7fefd81982b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d990 tx=0x7fefd400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:56.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd401b070 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd819e650 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd400edf0 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4014800 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.783+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd806b520 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.784+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefd806bb30 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.786+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fefd400e040 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.786+0000 7fefce7fc640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fefc40b5370 0x7fefc40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.786+0000 7fefdced2640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fefc40b5370 0x7fefc40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.786+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fefd40e7d90 con 0x7fefd8101cd0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefce7fc640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] conn(0x7fefc40beab0 0x7fefc40c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefd4010e50 con 0x7fefc40beab0 2024-09-10T22:45:56.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fefd40e8270 con 0x7fefd8101cd0 2024-09-10T22:45:56.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefdced2640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fefc40b5370 0x7fefc40b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fefd81042f0 tx=0x7fefc8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:56.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefdded4640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] conn(0x7fefc40beab0 0x7fefc40c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:56.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.787+0000 7fefdded4640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] conn(0x7fefc40beab0 0x7fefc40c0ed0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:56.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.788+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== osd.4 v2:172.21.15.117:6808/2758163149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefd4010e50 con 0x7fefc40beab0 2024-09-10T22:45:56.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.802+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 --> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefd806bf30 con 0x7fefc40beab0 2024-09-10T22:45:56.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.802+0000 7fefce7fc640 1 -- 172.21.15.96:0/646483023 <== osd.4 v2:172.21.15.117:6808/2758163149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fefd806bf30 con 0x7fefc40beab0 2024-09-10T22:45:56.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] conn(0x7fefc40beab0 msgr2=0x7fefc40c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] conn(0x7fefc40beab0 0x7fefc40c0ed0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fefc40b5370 msgr2=0x7fefc40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fefc40b5370 0x7fefc40b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fefd81042f0 tx=0x7fefc8008040 comp rx=0 tx=0).stop 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 msgr2=0x7fefd81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 --2- 172.21.15.96:0/646483023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefd8101cd0 0x7fefd81982b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d990 tx=0x7fefd400de50 comp rx=0 tx=0).stop 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdded4640 1 -- 172.21.15.96:0/646483023 reap_dead start 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 shutdown_connections 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 >> 172.21.15.96:0/646483023 conn(0x7fefd80fd740 msgr2=0x7fefd80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 shutdown_connections 2024-09-10T22:45:56.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:56.803+0000 7fefdf937640 1 -- 172.21.15.96:0/646483023 wait complete. 2024-09-10T22:45:57.092 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:57.094 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:45:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-10T22:45:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:45:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:57 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-10T22:45:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:45:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:57 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:57 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-10T22:45:57.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.819+0000 7f860839e640 1 -- 172.21.15.96:0/2013959100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860010d140 msgr2=0x7f860010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.819+0000 7f860839e640 1 --2- 172.21.15.96:0/2013959100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860010d140 0x7f860010f5a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009ca0 tx=0x7f85f802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- 172.21.15.96:0/2013959100 shutdown_connections 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- 172.21.15.96:0/2013959100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860010d140 0x7f860010f5a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- 172.21.15.96:0/2013959100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8600104600 0x7f860010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- 172.21.15.96:0/2013959100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 0x7f86001040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- 172.21.15.96:0/2013959100 >> 172.21.15.96:0/2013959100 conn(0x7f86000fd700 msgr2=0x7f86000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- 172.21.15.96:0/2013959100 shutdown_connections 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- 172.21.15.96:0/2013959100 wait complete. 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 Processor -- start 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- start start 2024-09-10T22:45:57.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 0x7f86001992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 0x7f8600199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f860010d140 0x7f860019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8600112e10 con 0x7f8600104600 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8600112c90 con 0x7f8600103ce0 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f860839e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8600112f90 con 0x7f860010d140 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f8605939640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 0x7f8600199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f8605939640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 0x7f8600199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41578/0 (socket says 172.21.15.96:41578) 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.820+0000 7f8605939640 1 -- 172.21.15.96:0/2286306940 learned_addr learned my addr 172.21.15.96:0/2286306940 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f860693b640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f860010d140 0x7f860019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f860613a640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 0x7f86001992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:57.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 -- 172.21.15.96:0/2286306940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f860010d140 msgr2=0x7f860019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f860010d140 0x7f860019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 -- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 msgr2=0x7f86001992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 0x7f86001992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860019f370 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f860613a640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8600103ce0 0x7f86001992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f8605939640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 0x7f8600199800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b3c0 tx=0x7f85f000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0011830 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860019f660 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0016070 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0010de0 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.821+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001a5da0 con 0x7f8600104600 2024-09-10T22:45:57.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.822+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86000ffd30 con 0x7f8600104600 2024-09-10T22:45:57.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.824+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f85f00119d0 con 0x7f8600104600 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.824+0000 7f85f77fe640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85ec0b5370 0x7f85ec0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f860613a640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85ec0b5370 0x7f85ec0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8153+0+0 (secure 0 0 0) 0x7f85f00e5d00 con 0x7f8600104600 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f85f77fe640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] conn(0x7f85ec0bed10 0x7f85ec0c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85ec0c1880 con 0x7f85ec0bed10 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f85f00e61e0 con 0x7f8600104600 2024-09-10T22:45:57.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f860613a640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85ec0b5370 0x7f85ec0b7830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009b40 tx=0x7f85fc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:57.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f860693b640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] conn(0x7f85ec0bed10 0x7f85ec0c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:57.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.825+0000 7f860693b640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] conn(0x7f85ec0bed10 0x7f85ec0c11b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:57.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.826+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== osd.1 v2:172.21.15.117:6800/3283899162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85ec0c1880 con 0x7f85ec0bed10 2024-09-10T22:45:57.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.839+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 --> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f860006bb90 con 0x7f85ec0bed10 2024-09-10T22:45:57.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f85f77fe640 1 -- 172.21.15.96:0/2286306940 <== osd.1 v2:172.21.15.117:6800/3283899162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f860006bb90 con 0x7f85ec0bed10 2024-09-10T22:45:57.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] conn(0x7f85ec0bed10 msgr2=0x7f85ec0c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] conn(0x7f85ec0bed10 0x7f85ec0c11b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:57.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85ec0b5370 msgr2=0x7f85ec0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85ec0b5370 0x7f85ec0b7830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009b40 tx=0x7f85fc005e50 comp rx=0 tx=0).stop 2024-09-10T22:45:57.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 msgr2=0x7f8600199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 --2- 172.21.15.96:0/2286306940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8600104600 0x7f8600199800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b3c0 tx=0x7f85f000b880 comp rx=0 tx=0).stop 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860693b640 1 -- 172.21.15.96:0/2286306940 reap_dead start 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.840+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 shutdown_connections 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.841+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 >> 172.21.15.96:0/2286306940 conn(0x7f86000fd700 msgr2=0x7f86000fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.841+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 shutdown_connections 2024-09-10T22:45:57.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:57.841+0000 7f860839e640 1 -- 172.21.15.96:0/2286306940 wait complete. 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T22:45:58.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-10T22:45:58.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:45:58.461 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:58 smithi096 ceph-mon[30869]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T22:45:58.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-10T22:45:58.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T22:45:58.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T22:45:58.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:45:58.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T22:45:58.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-10T22:45:58.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:45:58.765 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:58 smithi121 ceph-mon[35119]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:45:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:58 smithi117 ceph-mon[35081]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T22:45:59.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.167+0000 7fac92c84640 1 -- 172.21.15.96:0/310941299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c101d30 msgr2=0x7fac8c102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.167+0000 7fac92c84640 1 --2- 172.21.15.96:0/310941299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c101d30 0x7fac8c102110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009ca0 tx=0x7fac7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- 172.21.15.96:0/310941299 shutdown_connections 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- 172.21.15.96:0/310941299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c107740 0x7fac8c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- 172.21.15.96:0/310941299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8c1080c0 0x7fac8c10e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- 172.21.15.96:0/310941299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c101d30 0x7fac8c102110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- 172.21.15.96:0/310941299 >> 172.21.15.96:0/310941299 conn(0x7fac8c0fd7e0 msgr2=0x7fac8c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- 172.21.15.96:0/310941299 shutdown_connections 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- 172.21.15.96:0/310941299 wait complete. 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 Processor -- start 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- start start 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c101d30 0x7fac8c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8c107740 0x7fac8c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 0x7fac8c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c111d80 con 0x7fac8c1080c0 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c111c00 con 0x7fac8c101d30 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac92c84640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c111f00 con 0x7fac8c107740 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.168+0000 7fac91221640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 0x7fac8c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 0x7fac8c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41630/0 (socket says 172.21.15.96:41630) 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 -- 172.21.15.96:0/3418295705 learned_addr learned my addr 172.21.15.96:0/3418295705 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac8bfff640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8c107740 0x7fac8c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac90a20640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c101d30 0x7fac8c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 -- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8c107740 msgr2=0x7fac8c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac8c107740 0x7fac8c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 -- 172.21.15.96:0/3418295705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c101d30 msgr2=0x7fac8c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c101d30 0x7fac8c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c19e300 con 0x7fac8c1080c0 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac90a20640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac8c101d30 0x7fac8c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:45:59.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac91221640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 0x7fac8c19db30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac8400d750 tx=0x7fac8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84017070 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c19e5f0 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac840054d0 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84020060 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.169+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c1a4e80 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.170+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac8c063820 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.172+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fac84004c00 con 0x7fac8c1080c0 2024-09-10T22:45:59.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.172+0000 7fac89ffb640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac780b5370 0x7fac780b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.172+0000 7fac90a20640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac780b5370 0x7fac780b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.172+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac840e5c70 con 0x7fac8c1080c0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac89ffb640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] conn(0x7fac780bed10 0x7fac780c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac84015e80 con 0x7fac780bed10 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fac840e6150 con 0x7fac8c1080c0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac90a20640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac780b5370 0x7fac780b7830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac7c000f80 tx=0x7fac7c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac8bfff640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] conn(0x7fac780bed10 0x7fac780c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.173+0000 7fac8bfff640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] conn(0x7fac780bed10 0x7fac780c11b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.174+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== osd.11 v2:172.21.15.121:6824/2582866969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac84015e80 con 0x7fac780bed10 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.188+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 --> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac8c063c50 con 0x7fac780bed10 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.188+0000 7fac89ffb640 1 -- 172.21.15.96:0/3418295705 <== osd.11 v2:172.21.15.121:6824/2582866969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac8c063c50 con 0x7fac780bed10 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] conn(0x7fac780bed10 msgr2=0x7fac780c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] conn(0x7fac780bed10 0x7fac780c11b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac780b5370 msgr2=0x7fac780b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac780b5370 0x7fac780b7830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac7c000f80 tx=0x7fac7c009360 comp rx=0 tx=0).stop 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 msgr2=0x7fac8c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 --2- 172.21.15.96:0/3418295705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac8c1080c0 0x7fac8c19db30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac8400d750 tx=0x7fac8400dc10 comp rx=0 tx=0).stop 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac91221640 1 -- 172.21.15.96:0/3418295705 reap_dead start 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 shutdown_connections 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 >> 172.21.15.96:0/3418295705 conn(0x7fac8c0fd7e0 msgr2=0x7fac8c0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 shutdown_connections 2024-09-10T22:45:59.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:59.189+0000 7fac92c84640 1 -- 172.21.15.96:0/3418295705 wait complete. 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3841830317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f01087d0 msgr2=0x7f49f0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3841830317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f01087d0 0x7f49f0108c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009900 tx=0x7f49dc02a410 comp rx=0 tx=0).stop 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3841830317 shutdown_connections 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3841830317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49f0109150 0x7f49f010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3841830317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f01087d0 0x7f49f0108c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3841830317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f0102dc0 0x7f49f01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3841830317 >> 172.21.15.96:0/3841830317 conn(0x7f49f00fe830 msgr2=0x7f49f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3841830317 shutdown_connections 2024-09-10T22:45:59.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3841830317 wait complete. 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.439+0000 7f49f6fcf640 1 Processor -- start 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 -- start start 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49f0102dc0 0x7f49f01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f01087d0 0x7f49f0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 0x7f49f019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0112da0 con 0x7f49f0109150 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0112c20 con 0x7f49f0102dc0 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f6fcf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0112f20 con 0x7f49f01087d0 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 0x7f49f019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 0x7f49f019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41602/0 (socket says 172.21.15.96:41602) 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 -- 172.21.15.96:0/3115116303 learned_addr learned my addr 172.21.15.96:0/3115116303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f4d6b640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49f0102dc0 0x7f49f01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49effff640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f01087d0 0x7f49f0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 -- 172.21.15.96:0/3115116303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f01087d0 msgr2=0x7f49f0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f01087d0 0x7f49f0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 -- 172.21.15.96:0/3115116303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49f0102dc0 msgr2=0x7f49f01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49f0102dc0 0x7f49f01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f019f290 con 0x7f49f0109150 2024-09-10T22:45:59.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.440+0000 7f49f556c640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 0x7f49f019eac0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49e800eaa0 tx=0x7f49e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.441+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8004d20 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.441+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e8005020 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.441+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80116b0 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.441+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f019f580 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.441+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f01a5c40 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.442+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49f00638e0 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.443+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f49e800f040 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49edffb640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f49e00b53f0 0x7f49e00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49f4d6b640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f49e00b53f0 0x7f49e00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f49e80e5ef0 con 0x7f49f0109150 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49edffb640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] conn(0x7f49e00bed90 0x7f49e00c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49e8014ec0 con 0x7f49e00bed90 2024-09-10T22:45:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f49e80e63d0 con 0x7f49f0109150 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49effff640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] conn(0x7f49e00bed90 0x7f49e00c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49f4d6b640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f49e00b53f0 0x7f49e00b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49e400abe0 tx=0x7f49e400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.444+0000 7f49effff640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] conn(0x7f49e00bed90 0x7f49e00c1230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.445+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== osd.9 v2:172.21.15.96:6826/1287082888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49e8014ec0 con 0x7f49e00bed90 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.458+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 --> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49f0063d60 con 0x7f49e00bed90 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.458+0000 7f49edffb640 1 -- 172.21.15.96:0/3115116303 <== osd.9 v2:172.21.15.96:6826/1287082888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49f0063d60 con 0x7f49e00bed90 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] conn(0x7f49e00bed90 msgr2=0x7f49e00c1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] conn(0x7f49e00bed90 0x7f49e00c1230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f49e00b53f0 msgr2=0x7f49e00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f49e00b53f0 0x7f49e00b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49e400abe0 tx=0x7f49e400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 msgr2=0x7f49f019eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 --2- 172.21.15.96:0/3115116303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f0109150 0x7f49f019eac0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49e800eaa0 tx=0x7f49e800eea0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f556c640 1 -- 172.21.15.96:0/3115116303 reap_dead start 2024-09-10T22:45:59.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 shutdown_connections 2024-09-10T22:45:59.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 >> 172.21.15.96:0/3115116303 conn(0x7f49f00fe830 msgr2=0x7f49f0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 shutdown_connections 2024-09-10T22:45:59.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.459+0000 7f49f6fcf640 1 -- 172.21.15.96:0/3115116303 wait complete. 2024-09-10T22:45:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:45:59 smithi096 ceph-mon[30869]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:45:59.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 -- 172.21.15.96:0/3103036739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 msgr2=0x7f64d410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/3103036739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d410e860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009900 tx=0x7f64c402a410 comp rx=0 tx=0).stop 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 -- 172.21.15.96:0/3103036739 shutdown_connections 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/3103036739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d41076e0 0x7f64d4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/3103036739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d410e860 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/3103036739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d4101cd0 0x7f64d41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 -- 172.21.15.96:0/3103036739 >> 172.21.15.96:0/3103036739 conn(0x7f64d40fd700 msgr2=0x7f64d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 -- 172.21.15.96:0/3103036739 shutdown_connections 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 -- 172.21.15.96:0/3103036739 wait complete. 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.556+0000 7f64dbf1c640 1 Processor -- start 2024-09-10T22:45:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 -- start start 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4101cd0 0x7f64d419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d41076e0 0x7f64d419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111d20 con 0x7f64d4108060 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111ba0 con 0x7f64d4101cd0 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111ea0 con 0x7f64d41076e0 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41616/0 (socket says 172.21.15.96:41616) 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 -- 172.21.15.96:0/2483335573 learned_addr learned my addr 172.21.15.96:0/2483335573 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64d9cb8640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4101cd0 0x7f64d419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64d94b7640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d41076e0 0x7f64d419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 -- 172.21.15.96:0/2483335573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d41076e0 msgr2=0x7f64d419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d41076e0 0x7f64d419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 -- 172.21.15.96:0/2483335573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4101cd0 msgr2=0x7f64d419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4101cd0 0x7f64d419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d41a54b0 con 0x7f64d4108060 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64d94b7640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64d41076e0 0x7f64d419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64da4b9640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d419ab70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004770 tx=0x7f64c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c403f030 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c4003aa0 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.557+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d41a5740 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.558+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4035070 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.558+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d41a5ca0 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.558+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64d40638e0 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.560+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f64c40035f0 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.560+0000 7f64d2ffd640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64bc0b5370 0x7f64bc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d9cb8640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64bc0b5370 0x7f64bc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f64c4106e50 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d2ffd640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] conn(0x7f64bc0bed10 0x7f64bc0c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64bc0c1880 con 0x7f64bc0bed10 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f64c4107330 con 0x7f64d4108060 2024-09-10T22:45:59.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d9cb8640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64bc0b5370 0x7f64bc0b7830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005fd0 tx=0x7f64cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d94b7640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] conn(0x7f64bc0bed10 0x7f64bc0c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.561+0000 7f64d94b7640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] conn(0x7f64bc0bed10 0x7f64bc0c11b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.562+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== osd.3 v2:172.21.15.96:6810/2944383556 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64bc0c1880 con 0x7f64bc0bed10 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.575+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 --> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64d4063d60 con 0x7f64bc0bed10 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.575+0000 7f64d2ffd640 1 -- 172.21.15.96:0/2483335573 <== osd.3 v2:172.21.15.96:6810/2944383556 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f64d4063d60 con 0x7f64bc0bed10 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] conn(0x7f64bc0bed10 msgr2=0x7f64bc0c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] conn(0x7f64bc0bed10 0x7f64bc0c11b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64bc0b5370 msgr2=0x7f64bc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64bc0b5370 0x7f64bc0b7830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005fd0 tx=0x7f64cc005850 comp rx=0 tx=0).stop 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 msgr2=0x7f64d419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 --2- 172.21.15.96:0/2483335573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64d4108060 0x7f64d419ab70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004770 tx=0x7f64c4004270 comp rx=0 tx=0).stop 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64da4b9640 1 -- 172.21.15.96:0/2483335573 reap_dead start 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 shutdown_connections 2024-09-10T22:45:59.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 >> 172.21.15.96:0/2483335573 conn(0x7f64d40fd700 msgr2=0x7f64d40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:45:59.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 shutdown_connections 2024-09-10T22:45:59.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:45:58.576+0000 7f64dbf1c640 1 -- 172.21.15.96:0/2483335573 wait complete. 2024-09-10T22:45:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:45:59 smithi121 ceph-mon[35119]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:45:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:45:59 smithi117 ceph-mon[35081]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:46:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:00 smithi096 ceph-mon[30869]: pgmap v308: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:00 smithi096 ceph-mon[30869]: Health check failed: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded (PG_DEGRADED) 2024-09-10T22:46:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:00 smithi121 ceph-mon[35119]: pgmap v308: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:00.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:00 smithi121 ceph-mon[35119]: Health check failed: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded (PG_DEGRADED) 2024-09-10T22:46:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:00 smithi117 ceph-mon[35081]: pgmap v308: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:00 smithi117 ceph-mon[35081]: Health check failed: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded (PG_DEGRADED) 2024-09-10T22:46:01.059 INFO:teuthology.orchestra.run.smithi096.stdout:296352743445 2024-09-10T22:46:01.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T22:46:01.196 INFO:teuthology.orchestra.run.smithi096.stdout:103079215166 2024-09-10T22:46:01.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 -- 172.21.15.96:0/503572741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc1087f0 msgr2=0x7f83bc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 --2- 172.21.15.96:0/503572741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc1087f0 0x7f83bc108c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009900 tx=0x7f83b002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 -- 172.21.15.96:0/503572741 shutdown_connections 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 --2- 172.21.15.96:0/503572741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 0x7f83bc10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 --2- 172.21.15.96:0/503572741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc1087f0 0x7f83bc108c30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 --2- 172.21.15.96:0/503572741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc102de0 0x7f83bc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 -- 172.21.15.96:0/503572741 >> 172.21.15.96:0/503572741 conn(0x7f83bc0fe830 msgr2=0x7f83bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 -- 172.21.15.96:0/503572741 shutdown_connections 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.350+0000 7f83c0a85640 1 -- 172.21.15.96:0/503572741 wait complete. 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 Processor -- start 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 -- start start 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 0x7f83bc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1087f0 0x7f83bc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 0x7f83bc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc112e30 con 0x7f83bc102de0 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc112cb0 con 0x7f83bc109170 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83c0a85640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc112fb0 con 0x7f83bc1087f0 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83ba59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 0x7f83bc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83ba59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 0x7f83bc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41644/0 (socket says 172.21.15.96:41644) 2024-09-10T22:46:01.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.351+0000 7f83ba59c640 1 -- 172.21.15.96:0/2983009047 learned_addr learned my addr 172.21.15.96:0/2983009047 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83b9d9b640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1087f0 0x7f83bc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83bad9d640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 0x7f83bc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 -- 172.21.15.96:0/2983009047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1087f0 msgr2=0x7f83bc199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1087f0 0x7f83bc199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 -- 172.21.15.96:0/2983009047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 msgr2=0x7f83bc19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 0x7f83bc19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc19f390 con 0x7f83bc102de0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83bad9d640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc109170 0x7f83bc19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ba59c640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 0x7f83bc1992e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c920 tx=0x7f83ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac007c70 con 0x7f83bc102de0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ac007e10 con 0x7f83bc102de0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc19f680 con 0x7f83bc102de0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac018820 con 0x7f83bc102de0 2024-09-10T22:46:01.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.352+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc075550 con 0x7f83bc102de0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.353+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83bc0638e0 con 0x7f83bc102de0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.355+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f83ac00f450 con 0x7f83bc102de0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.355+0000 7f83ab7fe640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83a40b53f0 0x7f83a40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.355+0000 7f83b9d9b640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83a40b53f0 0x7f83a40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f83ac0e6170 con 0x7f83bc102de0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83ab7fe640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] conn(0x7f83a40bed90 0x7f83a40c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83a40c1900 con 0x7f83a40bed90 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f83ac0e6650 con 0x7f83bc102de0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83bad9d640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] conn(0x7f83a40bed90 0x7f83a40c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:01.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83b9d9b640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83a40b53f0 0x7f83a40b78b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83b0031440 tx=0x7f83b0005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:01.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.356+0000 7f83bad9d640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] conn(0x7f83a40bed90 0x7f83a40c1230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:01.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.357+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== osd.0 v2:172.21.15.96:6802/2443850477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83a40c1900 con 0x7f83a40bed90 2024-09-10T22:46:01.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 --> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83bc063d60 con 0x7f83a40bed90 2024-09-10T22:46:01.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83ab7fe640 1 -- 172.21.15.96:0/2983009047 <== osd.0 v2:172.21.15.96:6802/2443850477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83bc063d60 con 0x7f83a40bed90 2024-09-10T22:46:01.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] conn(0x7f83a40bed90 msgr2=0x7f83a40c1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] conn(0x7f83a40bed90 0x7f83a40c1230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83a40b53f0 msgr2=0x7f83a40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83a40b53f0 0x7f83a40b78b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83b0031440 tx=0x7f83b0005e80 comp rx=0 tx=0).stop 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 msgr2=0x7f83bc1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 --2- 172.21.15.96:0/2983009047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83bc102de0 0x7f83bc1992e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c920 tx=0x7f83ac00cde0 comp rx=0 tx=0).stop 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83bad9d640 1 -- 172.21.15.96:0/2983009047 reap_dead start 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 shutdown_connections 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.371+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 >> 172.21.15.96:0/2983009047 conn(0x7f83bc0fe830 msgr2=0x7f83bc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.372+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 shutdown_connections 2024-09-10T22:46:01.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:01.372+0000 7f83c0a85640 1 -- 172.21.15.96:0/2983009047 wait complete. 2024-09-10T22:46:02.354 INFO:teuthology.orchestra.run.smithi096.stdout:128849018937 2024-09-10T22:46:02.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T22:46:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:02 smithi096 ceph-mon[30869]: pgmap v309: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:02.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:02 smithi121 ceph-mon[35119]: pgmap v309: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:02 smithi117 ceph-mon[35081]: pgmap v309: 97 pgs: 1 activating+degraded, 1 activating, 95 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 11 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:02.864 INFO:teuthology.orchestra.run.smithi096.stdout:34359738446 2024-09-10T22:46:02.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T22:46:03.511 INFO:teuthology.orchestra.run.smithi096.stdout:55834574920 2024-09-10T22:46:03.511 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T22:46:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 -- 172.21.15.96:0/4137641449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 msgr2=0x7fadd010cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 --2- 172.21.15.96:0/4137641449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd010cb10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009900 tx=0x7fadb802a410 comp rx=0 tx=0).stop 2024-09-10T22:46:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 -- 172.21.15.96:0/4137641449 shutdown_connections 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 --2- 172.21.15.96:0/4137641449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadd010d0e0 0x7fadd010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 --2- 172.21.15.96:0/4137641449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd010cb10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 --2- 172.21.15.96:0/4137641449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd0103b30 0x7fadd0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 -- 172.21.15.96:0/4137641449 >> 172.21.15.96:0/4137641449 conn(0x7fadd00fd700 msgr2=0x7fadd00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 -- 172.21.15.96:0/4137641449 shutdown_connections 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.734+0000 7fadd6387640 1 -- 172.21.15.96:0/4137641449 wait complete. 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 Processor -- start 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 -- start start 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadd0103b30 0x7fadd01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd010d0e0 0x7fadd019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:03.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0112dc0 con 0x7fadd0104530 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0112c40 con 0x7fadd010d0e0 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadd6387640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0112f40 con 0x7fadd0103b30 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41654/0 (socket says 172.21.15.96:41654) 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadc7fff640 1 -- 172.21.15.96:0/2527046777 learned_addr learned my addr 172.21.15.96:0/2527046777 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.735+0000 7fadcffff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadd0103b30 0x7fadd01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadd4924640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd010d0e0 0x7fadd019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 -- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadd0103b30 msgr2=0x7fadd01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadd0103b30 0x7fadd01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 -- 172.21.15.96:0/2527046777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd010d0e0 msgr2=0x7fadd019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd010d0e0 0x7fadd019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd019f3a0 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadd4924640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd010d0e0 0x7fadd019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadc7fff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd0199830 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005940 tx=0x7fadb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb803f030 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb8003cb0 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd019f630 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8035070 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.736+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01a5d50 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.737+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadd00638e0 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.738+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fadb8003800 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcdffb640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadbc0b5370 0x7fadbc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcffff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadbc0b5370 0x7fadbc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fadb8106b10 con 0x7fadd0104530 2024-09-10T22:46:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcdffb640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] conn(0x7fadbc0bed10 0x7fadbc0c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadb8030e80 con 0x7fadbc0bed10 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.739+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fadb8106ff0 con 0x7fadd0104530 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.740+0000 7fadcffff640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadbc0b5370 0x7fadbc0b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009990 tx=0x7fadc0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.740+0000 7fadd4924640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] conn(0x7fadbc0bed10 0x7fadbc0c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.740+0000 7fadd4924640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] conn(0x7fadbc0bed10 0x7fadbc0c11b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.741+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== osd.5 v2:172.21.15.121:6808/1183691363 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadb8030e80 con 0x7fadbc0bed10 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.754+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 --> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadd0063d60 con 0x7fadbc0bed10 2024-09-10T22:46:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadcdffb640 1 -- 172.21.15.96:0/2527046777 <== osd.5 v2:172.21.15.121:6808/1183691363 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fadd0063d60 con 0x7fadbc0bed10 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] conn(0x7fadbc0bed10 msgr2=0x7fadbc0c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] conn(0x7fadbc0bed10 0x7fadbc0c11b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadbc0b5370 msgr2=0x7fadbc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadbc0b5370 0x7fadbc0b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009990 tx=0x7fadc0009110 comp rx=0 tx=0).stop 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 msgr2=0x7fadd0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 --2- 172.21.15.96:0/2527046777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd0104530 0x7fadd0199830 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005940 tx=0x7fadb80042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd4924640 1 -- 172.21.15.96:0/2527046777 reap_dead start 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 shutdown_connections 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.755+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 >> 172.21.15.96:0/2527046777 conn(0x7fadd00fd700 msgr2=0x7fadd00fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.756+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 shutdown_connections 2024-09-10T22:46:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:03.756+0000 7fadd6387640 1 -- 172.21.15.96:0/2527046777 wait complete. 2024-09-10T22:46:04.394 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:04.395 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:04.409 INFO:teuthology.orchestra.run.smithi096.stdout:150323855412 2024-09-10T22:46:04.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T22:46:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:04 smithi096 ceph-mon[30869]: pgmap v310: 97 pgs: 1 activating+degraded, 96 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:04 smithi121 ceph-mon[35119]: pgmap v310: 97 pgs: 1 activating+degraded, 96 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:04 smithi117 ceph-mon[35081]: pgmap v310: 97 pgs: 1 activating+degraded, 96 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s; 1/306 objects degraded (0.327%); 0 B/s, 0 objects/s recovering 2024-09-10T22:46:04.828 INFO:teuthology.orchestra.run.smithi096.stdout:249108103200 2024-09-10T22:46:04.829 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T22:46:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- 172.21.15.96:0/1974933636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 msgr2=0x7fe7cc106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/1974933636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc106890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009900 tx=0x7fe7bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- 172.21.15.96:0/1974933636 shutdown_connections 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/1974933636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7cc106ee0 0x7fe7cc107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/1974933636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc106890 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/1974933636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc101aa0 0x7fe7cc101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- 172.21.15.96:0/1974933636 >> 172.21.15.96:0/1974933636 conn(0x7fe7cc0fd6d0 msgr2=0x7fe7cc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:05.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- 172.21.15.96:0/1974933636 shutdown_connections 2024-09-10T22:46:05.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- 172.21.15.96:0/1974933636 wait complete. 2024-09-10T22:46:05.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 Processor -- start 2024-09-10T22:46:05.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.245+0000 7fe7d40f0640 1 -- start start 2024-09-10T22:46:05.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7cc101aa0 0x7fe7cc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc106ee0 0x7fe7cc1a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1193c0 con 0x7fe7cc1023c0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc119240 con 0x7fe7cc106ee0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d40f0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc119540 con 0x7fe7cc101aa0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41666/0 (socket says 172.21.15.96:41666) 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 -- 172.21.15.96:0/2264201999 learned_addr learned my addr 172.21.15.96:0/2264201999 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d268d640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc106ee0 0x7fe7cc1a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d1e8c640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7cc101aa0 0x7fe7cc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:05.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 -- 172.21.15.96:0/2264201999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7cc101aa0 msgr2=0x7fe7cc19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7cc101aa0 0x7fe7cc19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 -- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc106ee0 msgr2=0x7fe7cc1a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc106ee0 0x7fe7cc1a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc1a59a0 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d268d640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7cc106ee0 0x7fe7cc1a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.246+0000 7fe7d168b640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc19fe30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc031440 tx=0x7fe7bc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.247+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc03f070 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.247+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc1a5c30 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.247+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7bc031d60 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.247+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc1ac350 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.247+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc035030 con 0x7fe7cc1023c0 2024-09-10T22:46:05.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.248+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7cc106240 con 0x7fe7cc1023c0 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.249+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fe7bc04c060 con 0x7fe7cc1023c0 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7c2ffd640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7b80b5370 0x7fe7b80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7d1e8c640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7b80b5370 0x7fe7b80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe7bc106750 con 0x7fe7cc1023c0 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7c2ffd640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] conn(0x7fe7b80bed10 0x7fe7b80c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7b80c1880 con 0x7fe7b80bed10 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe7bc106c30 con 0x7fe7cc1023c0 2024-09-10T22:46:05.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7d1e8c640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7b80b5370 0x7fe7b80b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80058c0 tx=0x7fe7c800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:05.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.250+0000 7fe7d268d640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] conn(0x7fe7b80bed10 0x7fe7b80c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:05.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.251+0000 7fe7d268d640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] conn(0x7fe7b80bed10 0x7fe7b80c11b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:05.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.252+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== osd.10 v2:172.21.15.117:6824/3915077352 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7b80c1880 con 0x7fe7b80bed10 2024-09-10T22:46:05.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.265+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 --> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7cc06bbd0 con 0x7fe7b80bed10 2024-09-10T22:46:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.265+0000 7fe7c2ffd640 1 -- 172.21.15.96:0/2264201999 <== osd.10 v2:172.21.15.117:6824/3915077352 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe7cc06bbd0 con 0x7fe7b80bed10 2024-09-10T22:46:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] conn(0x7fe7b80bed10 msgr2=0x7fe7b80c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] conn(0x7fe7b80bed10 0x7fe7b80c11b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7b80b5370 msgr2=0x7fe7b80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7b80b5370 0x7fe7b80b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80058c0 tx=0x7fe7c800a4c0 comp rx=0 tx=0).stop 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 msgr2=0x7fe7cc19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 --2- 172.21.15.96:0/2264201999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7cc1023c0 0x7fe7cc19fe30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc031440 tx=0x7fe7bc003c80 comp rx=0 tx=0).stop 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d268d640 1 -- 172.21.15.96:0/2264201999 reap_dead start 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 shutdown_connections 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 >> 172.21.15.96:0/2264201999 conn(0x7fe7cc0fd6d0 msgr2=0x7fe7cc1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 shutdown_connections 2024-09-10T22:46:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.266+0000 7fe7d40f0640 1 -- 172.21.15.96:0/2264201999 wait complete. 2024-09-10T22:46:05.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:05 smithi096 ceph-mon[30869]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 383 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:05.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:05 smithi096 ceph-mon[30869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded) 2024-09-10T22:46:05.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:05 smithi096 ceph-mon[30869]: Cluster is now healthy 2024-09-10T22:46:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:05 smithi121 ceph-mon[35119]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 383 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:05 smithi121 ceph-mon[35119]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded) 2024-09-10T22:46:05.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:05 smithi121 ceph-mon[35119]: Cluster is now healthy 2024-09-10T22:46:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:05 smithi117 ceph-mon[35081]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 383 B/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:05 smithi117 ceph-mon[35081]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/306 objects degraded (0.327%), 1 pg degraded) 2024-09-10T22:46:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:05 smithi117 ceph-mon[35081]: Cluster is now healthy 2024-09-10T22:46:06.325 INFO:teuthology.orchestra.run.smithi096.stdout:270582939675 2024-09-10T22:46:06.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 -- 172.21.15.96:0/883588264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14101cb0 msgr2=0x7f8f14102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 --2- 172.21.15.96:0/883588264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14101cb0 0x7f8f14102090 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009900 tx=0x7f8f0402a410 comp rx=0 tx=0).stop 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 -- 172.21.15.96:0/883588264 shutdown_connections 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 --2- 172.21.15.96:0/883588264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f141076c0 0x7f8f14107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 --2- 172.21.15.96:0/883588264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14108040 0x7f8f1410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 --2- 172.21.15.96:0/883588264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14101cb0 0x7f8f14102090 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 -- 172.21.15.96:0/883588264 >> 172.21.15.96:0/883588264 conn(0x7f8f140fd700 msgr2=0x7f8f140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 -- 172.21.15.96:0/883588264 shutdown_connections 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.433+0000 7f8f197b1640 1 -- 172.21.15.96:0/883588264 wait complete. 2024-09-10T22:46:06.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 Processor -- start 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 -- start start 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14101cb0 0x7f8f14199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f141076c0 0x7f8f141997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 0x7f8f1419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14112e20 con 0x7f8f14108040 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14112ca0 con 0x7f8f141076c0 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f197b1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14112fa0 con 0x7f8f14101cb0 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 0x7f8f1419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 0x7f8f1419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41684/0 (socket says 172.21.15.96:41684) 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 -- 172.21.15.96:0/277977735 learned_addr learned my addr 172.21.15.96:0/277977735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f0bfff640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f141076c0 0x7f8f141997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:06.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f12ffd640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14101cb0 0x7f8f14199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 -- 172.21.15.96:0/277977735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14101cb0 msgr2=0x7f8f14199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14101cb0 0x7f8f14199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 -- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f141076c0 msgr2=0x7f8f141997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f141076c0 0x7f8f141997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f137fe640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1419f340 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.434+0000 7f8f12ffd640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f14101cb0 0x7f8f14199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f137fe640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 0x7f8f1419eb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00cad0 tx=0x7f8f0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00ae20 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f0c005020 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1419f630 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00f6b0 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.435+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141a5ec0 con 0x7f8f14108040 2024-09-10T22:46:06.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.436+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f140638e0 con 0x7f8f14108040 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.437+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8f0c0054f0 con 0x7f8f14108040 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f10ff9640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f000b53f0 0x7f8f000b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f12ffd640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f000b53f0 0x7f8f000b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8f0c0e6020 con 0x7f8f14108040 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f10ff9640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] conn(0x7f8f000bed90 0x7f8f000c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f000c1900 con 0x7f8f000bed90 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f8f0c0e6500 con 0x7f8f14108040 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f12ffd640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f000b53f0 0x7f8f000b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f04031440 tx=0x7f8f040095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:06.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.438+0000 7f8f0bfff640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] conn(0x7f8f000bed90 0x7f8f000c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.439+0000 7f8f0bfff640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] conn(0x7f8f000bed90 0x7f8f000c1230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.439+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== osd.7 v2:172.21.15.117:6816/918709723 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f000c1900 con 0x7f8f000bed90 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.453+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 --> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f14063ce0 con 0x7f8f000bed90 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f10ff9640 1 -- 172.21.15.96:0/277977735 <== osd.7 v2:172.21.15.117:6816/918709723 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f14063ce0 con 0x7f8f000bed90 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] conn(0x7f8f000bed90 msgr2=0x7f8f000c1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] conn(0x7f8f000bed90 0x7f8f000c1230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f000b53f0 msgr2=0x7f8f000b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f000b53f0 0x7f8f000b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f04031440 tx=0x7f8f040095f0 comp rx=0 tx=0).stop 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 msgr2=0x7f8f1419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 --2- 172.21.15.96:0/277977735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f14108040 0x7f8f1419eb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00cad0 tx=0x7f8f0c00cf90 comp rx=0 tx=0).stop 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f137fe640 1 -- 172.21.15.96:0/277977735 reap_dead start 2024-09-10T22:46:06.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 shutdown_connections 2024-09-10T22:46:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 >> 172.21.15.96:0/277977735 conn(0x7f8f140fd700 msgr2=0x7f8f140ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 shutdown_connections 2024-09-10T22:46:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:05.454+0000 7f8f197b1640 1 -- 172.21.15.96:0/277977735 wait complete. 2024-09-10T22:46:07.325 INFO:teuthology.orchestra.run.smithi096.stdout:197568495659 2024-09-10T22:46:07.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T22:46:07.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 -- 172.21.15.96:0/1042627903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd0103ba0 msgr2=0x7f4cd0103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/1042627903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd0103ba0 0x7f4cd0103f80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009900 tx=0x7f4ccc02a410 comp rx=0 tx=0).stop 2024-09-10T22:46:07.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 -- 172.21.15.96:0/1042627903 shutdown_connections 2024-09-10T22:46:07.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/1042627903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd01045a0 0x7f4cd010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/1042627903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd010d150 0x7f4cd010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/1042627903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd0103ba0 0x7f4cd0103f80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.510+0000 7f4cd7f16640 1 -- 172.21.15.96:0/1042627903 >> 172.21.15.96:0/1042627903 conn(0x7f4cd00fd770 msgr2=0x7f4cd00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:07.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.511+0000 7f4cd7f16640 1 -- 172.21.15.96:0/1042627903 shutdown_connections 2024-09-10T22:46:07.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.511+0000 7f4cd7f16640 1 -- 172.21.15.96:0/1042627903 wait complete. 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.511+0000 7f4cd7f16640 1 Processor -- start 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.511+0000 7f4cd7f16640 1 -- start start 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.511+0000 7f4cd7f16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd0103ba0 0x7f4cd0101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd7f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 0x7f4cd0101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd7f16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd010d150 0x7f4cd01024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:07.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 0x7f4cd0101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd7f16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112dd0 con 0x7f4cd01045a0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd7f16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112c50 con 0x7f4cd0103ba0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd7f16640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112f50 con 0x7f4cd010d150 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 0x7f4cd0101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45832/0 (socket says 172.21.15.96:45832) 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 -- 172.21.15.96:0/90655854 learned_addr learned my addr 172.21.15.96:0/90655854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd5cb2640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd0103ba0 0x7f4cd0101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd64b3640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd010d150 0x7f4cd01024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 -- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd010d150 msgr2=0x7f4cd01024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd010d150 0x7f4cd01024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 -- 172.21.15.96:0/90655854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd0103ba0 msgr2=0x7f4cd0101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd0103ba0 0x7f4cd0101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd01a76c0 con 0x7f4cd01045a0 2024-09-10T22:46:07.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.512+0000 7f4cd54b1640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 0x7f4cd0101f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c890 tx=0x7f4cc000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.513+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0004220 con 0x7f4cd01045a0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.513+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc00043c0 con 0x7f4cd01045a0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.513+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0004a00 con 0x7f4cd01045a0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.513+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd01a79b0 con 0x7f4cd01045a0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.513+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd01a7f20 con 0x7f4cd01045a0 2024-09-10T22:46:07.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.514+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cd00ffda0 con 0x7f4cd01045a0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.515+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f4cc00079b0 con 0x7f4cd01045a0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cc6ffd640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4cbc0b54c0 0x7f4cbc0b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cd5cb2640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4cbc0b54c0 0x7f4cbc0b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4cc00e5cc0 con 0x7f4cd01045a0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cc6ffd640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] conn(0x7f4cbc0bee60 0x7f4cbc0c1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cbc0c19d0 con 0x7f4cbc0bee60 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.516+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4cc00e61a0 con 0x7f4cd01045a0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.517+0000 7f4cd5cb2640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4cbc0b54c0 0x7f4cbc0b7980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc031440 tx=0x7f4ccc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:07.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.517+0000 7f4cd64b3640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] conn(0x7f4cbc0bee60 0x7f4cbc0c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:07.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.517+0000 7f4cd64b3640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] conn(0x7f4cbc0bee60 0x7f4cbc0c1300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:07.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.518+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== osd.2 v2:172.21.15.121:6800/203762338 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cbc0c19d0 con 0x7f4cbc0bee60 2024-09-10T22:46:07.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.532+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 --> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4cd006b930 con 0x7f4cbc0bee60 2024-09-10T22:46:07.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.532+0000 7f4cc6ffd640 1 -- 172.21.15.96:0/90655854 <== osd.2 v2:172.21.15.121:6800/203762338 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4cd006b930 con 0x7f4cbc0bee60 2024-09-10T22:46:07.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] conn(0x7f4cbc0bee60 msgr2=0x7f4cbc0c1300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] conn(0x7f4cbc0bee60 0x7f4cbc0c1300 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4cbc0b54c0 msgr2=0x7f4cbc0b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4cbc0b54c0 0x7f4cbc0b7980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc031440 tx=0x7f4ccc005e80 comp rx=0 tx=0).stop 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 msgr2=0x7f4cd0101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 --2- 172.21.15.96:0/90655854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd01045a0 0x7f4cd0101f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c890 tx=0x7f4cc000cd50 comp rx=0 tx=0).stop 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd64b3640 1 -- 172.21.15.96:0/90655854 reap_dead start 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 shutdown_connections 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 >> 172.21.15.96:0/90655854 conn(0x7f4cd00fd770 msgr2=0x7f4cd00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 shutdown_connections 2024-09-10T22:46:07.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:07.533+0000 7f4cd7f16640 1 -- 172.21.15.96:0/90655854 wait complete. 2024-09-10T22:46:07.545 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:07 smithi096 ceph-mon[30869]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 307 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:07 smithi121 ceph-mon[35119]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 307 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:07 smithi117 ceph-mon[35081]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 307 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:08.522 INFO:teuthology.orchestra.run.smithi096.stdout:77309411397 2024-09-10T22:46:08.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T22:46:08.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.691+0000 7f3e08ae0640 1 -- 172.21.15.96:0/341146381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04109150 msgr2=0x7f3e0410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.691+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/341146381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04109150 0x7f3e0410f950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009ca0 tx=0x7f3dfc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 -- 172.21.15.96:0/341146381 shutdown_connections 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/341146381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04109150 0x7f3e0410f950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/341146381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e041087d0 0x7f3e04108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/341146381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04102dc0 0x7f3e041031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 -- 172.21.15.96:0/341146381 >> 172.21.15.96:0/341146381 conn(0x7f3e040fe870 msgr2=0x7f3e04100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 -- 172.21.15.96:0/341146381 shutdown_connections 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 -- 172.21.15.96:0/341146381 wait complete. 2024-09-10T22:46:08.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 Processor -- start 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.692+0000 7f3e08ae0640 1 -- start start 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 0x7f3e04199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e041087d0 0x7f3e041997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04109150 0x7f3e0419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04112e10 con 0x7f3e04102dc0 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04112c90 con 0x7f3e041087d0 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e08ae0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04112f90 con 0x7f3e04109150 2024-09-10T22:46:08.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 0x7f3e04199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 0x7f3e04199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45860/0 (socket says 172.21.15.96:45860) 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 -- 172.21.15.96:0/1235063674 learned_addr learned my addr 172.21.15.96:0/1235063674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e02d9d640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04109150 0x7f3e0419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3dfbfff640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e041087d0 0x7f3e041997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 -- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04109150 msgr2=0x7f3e0419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04109150 0x7f3e0419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 -- 172.21.15.96:0/1235063674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e041087d0 msgr2=0x7f3e041997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e041087d0 0x7f3e041997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0419f310 con 0x7f3e04102dc0 2024-09-10T22:46:08.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3e0259c640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 0x7f3e04199260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c890 tx=0x7f3df400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:08.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.693+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4004220 con 0x7f3e04102dc0 2024-09-10T22:46:08.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.694+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0419f600 con 0x7f3e04102dc0 2024-09-10T22:46:08.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.694+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df40043c0 con 0x7f3e04102dc0 2024-09-10T22:46:08.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.694+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a5e90 con 0x7f3e04102dc0 2024-09-10T22:46:08.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.694+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4004a00 con 0x7f3e04102dc0 2024-09-10T22:46:08.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.695+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e040638e0 con 0x7f3e04102dc0 2024-09-10T22:46:08.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.696+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f3df4004ba0 con 0x7f3e04102dc0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfb7fe640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3df00b5370 0x7f3df00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfbfff640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3df00b5370 0x7f3df00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3df40e5fa0 con 0x7f3e04102dc0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfb7fe640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] conn(0x7f3df00bed10 0x7f3df00c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3df4012ec0 con 0x7f3df00bed10 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.697+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3df40e6350 con 0x7f3e04102dc0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.698+0000 7f3dfbfff640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3df00b5370 0x7f3df00b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e04102470 tx=0x7f3dec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.698+0000 7f3e02d9d640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] conn(0x7f3df00bed10 0x7f3df00c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.698+0000 7f3e02d9d640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] conn(0x7f3df00bed10 0x7f3df00c11b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:08.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.699+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== osd.8 v2:172.21.15.121:6816/4251994917 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3df4012ec0 con 0x7f3df00bed10 2024-09-10T22:46:08.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.714+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 --> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e04063ce0 con 0x7f3df00bed10 2024-09-10T22:46:08.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.714+0000 7f3dfb7fe640 1 -- 172.21.15.96:0/1235063674 <== osd.8 v2:172.21.15.121:6816/4251994917 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3e04063ce0 con 0x7f3df00bed10 2024-09-10T22:46:08.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] conn(0x7f3df00bed10 msgr2=0x7f3df00c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] conn(0x7f3df00bed10 0x7f3df00c11b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3df00b5370 msgr2=0x7f3df00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3df00b5370 0x7f3df00b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e04102470 tx=0x7f3dec00b390 comp rx=0 tx=0).stop 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 msgr2=0x7f3e04199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 --2- 172.21.15.96:0/1235063674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e04102dc0 0x7f3e04199260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c890 tx=0x7f3df400cd50 comp rx=0 tx=0).stop 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e02d9d640 1 -- 172.21.15.96:0/1235063674 reap_dead start 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 shutdown_connections 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 >> 172.21.15.96:0/1235063674 conn(0x7f3e040fe870 msgr2=0x7f3e04100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 shutdown_connections 2024-09-10T22:46:08.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:08.715+0000 7f3e08ae0640 1 -- 172.21.15.96:0/1235063674 wait complete. 2024-09-10T22:46:09.353 INFO:teuthology.orchestra.run.smithi096.stdout:223338299430 2024-09-10T22:46:09.353 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 -- 172.21.15.96:0/1517170135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc104590 msgr2=0x7f55bc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 --2- 172.21.15.96:0/1517170135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc104590 0x7f55bc10cb70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009900 tx=0x7f55b002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 -- 172.21.15.96:0/1517170135 shutdown_connections 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 --2- 172.21.15.96:0/1517170135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc10d140 0x7f55bc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 --2- 172.21.15.96:0/1517170135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc104590 0x7f55bc10cb70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 --2- 172.21.15.96:0/1517170135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc103b90 0x7f55bc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 -- 172.21.15.96:0/1517170135 >> 172.21.15.96:0/1517170135 conn(0x7f55bc0fd700 msgr2=0x7f55bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 -- 172.21.15.96:0/1517170135 shutdown_connections 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.555+0000 7f55c1867640 1 -- 172.21.15.96:0/1517170135 wait complete. 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 Processor -- start 2024-09-10T22:46:09.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 -- start start 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc103b90 0x7f55bc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc104590 0x7f55bc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 0x7f55bc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc112dc0 con 0x7f55bc10d140 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc112c40 con 0x7f55bc104590 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55c1867640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc112f40 con 0x7f55bc103b90 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 0x7f55bc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 0x7f55bc19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45870/0 (socket says 172.21.15.96:45870) 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 -- 172.21.15.96:0/2464720938 learned_addr learned my addr 172.21.15.96:0/2464720938 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55baffd640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc103b90 0x7f55bc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55ba7fc640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc104590 0x7f55bc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:09.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 -- 172.21.15.96:0/2464720938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc103b90 msgr2=0x7f55bc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55bc103b90 0x7f55bc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.556+0000 7f55bb7fe640 1 -- 172.21.15.96:0/2464720938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc104590 msgr2=0x7f55bc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55bb7fe640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55bc104590 0x7f55bc199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55bb7fe640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc19f3e0 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55bb7fe640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 0x7f55bc19ec10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55b400ccb0 tx=0x7f55b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400f450 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc19f6d0 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b400faa0 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a5d90 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.557+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40189d0 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.558+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55a8001060 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.559+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f55b4020030 con 0x7f55bc10d140 2024-09-10T22:46:09.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55c0865640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55a40b5370 0x7f55a40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55baffd640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55a40b5370 0x7f55a40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55b40e6050 con 0x7f55bc10d140 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55c0865640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] conn(0x7f55a40bed10 0x7f55a40c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55b4012ec0 con 0x7f55a40bed10 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f55b40e6530 con 0x7f55bc10d140 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55ba7fc640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] conn(0x7f55a40bed10 0x7f55a40c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.560+0000 7f55baffd640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55a40b5370 0x7f55a40b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.561+0000 7f55ba7fc640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] conn(0x7f55a40bed10 0x7f55a40c11b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:09.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.561+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== osd.6 v2:172.21.15.96:6818/201788565 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55b4012ec0 con 0x7f55a40bed10 2024-09-10T22:46:09.580 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:09 smithi096 ceph-mon[30869]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 278 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:09.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.576+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 --> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55a8002e00 con 0x7f55a40bed10 2024-09-10T22:46:09.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.577+0000 7f55c0865640 1 -- 172.21.15.96:0/2464720938 <== osd.6 v2:172.21.15.96:6818/201788565 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f55a8002e00 con 0x7f55a40bed10 2024-09-10T22:46:09.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.577+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] conn(0x7f55a40bed10 msgr2=0x7f55a40c11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.577+0000 7f55c1867640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] conn(0x7f55a40bed10 0x7f55a40c11b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:09.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.580+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55a40b5370 msgr2=0x7f55a40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.580+0000 7f55c1867640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55a40b5370 0x7f55a40b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac009110 comp rx=0 tx=0).stop 2024-09-10T22:46:09.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.580+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 msgr2=0x7f55bc19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:09.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.580+0000 7f55c1867640 1 --2- 172.21.15.96:0/2464720938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55bc10d140 0x7f55bc19ec10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55b400ccb0 tx=0x7f55b400a620 comp rx=0 tx=0).stop 2024-09-10T22:46:09.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.580+0000 7f55bb7fe640 1 -- 172.21.15.96:0/2464720938 reap_dead start 2024-09-10T22:46:09.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.581+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 shutdown_connections 2024-09-10T22:46:09.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.581+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 >> 172.21.15.96:0/2464720938 conn(0x7f55bc0fd700 msgr2=0x7f55bc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:09.589 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:09.589 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:09.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.584+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 shutdown_connections 2024-09-10T22:46:09.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:09.584+0000 7f55c1867640 1 -- 172.21.15.96:0/2464720938 wait complete. 2024-09-10T22:46:09.590 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:09.592 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:09.593 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:09 smithi121 ceph-mon[35119]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 278 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:09 smithi117 ceph-mon[35081]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 278 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:10.073 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:10.075 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:11.405 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:11 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/982483990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:46:11.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:11 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/982483990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 -- 172.21.15.96:0/989342049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 msgr2=0x7f75a410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 --2- 172.21.15.96:0/989342049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a410f550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f759c009ca0 tx=0x7f759c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 -- 172.21.15.96:0/989342049 shutdown_connections 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 --2- 172.21.15.96:0/989342049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4104570 0x7f75a410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 --2- 172.21.15.96:0/989342049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a410f550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 --2- 172.21.15.96:0/989342049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4103b70 0x7f75a4103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 -- 172.21.15.96:0/989342049 >> 172.21.15.96:0/989342049 conn(0x7f75a40fd700 msgr2=0x7f75a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:11.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 -- 172.21.15.96:0/989342049 shutdown_connections 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.148+0000 7f75ab171640 1 -- 172.21.15.96:0/989342049 wait complete. 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 Processor -- start 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 -- start start 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4103b70 0x7f75a4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4104570 0x7f75a41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112d60 con 0x7f75a410d120 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112be0 con 0x7f75a4103b70 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75ab171640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4112ee0 con 0x7f75a4104570 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75a970e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75a970e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45902/0 (socket says 172.21.15.96:45902) 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75a970e640 1 -- 172.21.15.96:0/2972290901 learned_addr learned my addr 172.21.15.96:0/2972290901 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:11.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75a3fff640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4104570 0x7f75a41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.149+0000 7f75a8f0d640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4103b70 0x7f75a4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 -- 172.21.15.96:0/2972290901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4104570 msgr2=0x7f75a41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4104570 0x7f75a41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 -- 172.21.15.96:0/2972290901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4103b70 msgr2=0x7f75a4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4103b70 0x7f75a4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 -- 172.21.15.96:0/2972290901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a419f240 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a970e640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f759c005cc0 tx=0x7f759c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c003bf0 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a419f4d0 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c003d90 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c02fb20 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.150+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a41a5bf0 con 0x7f75a410d120 2024-09-10T22:46:11.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.151+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a4103f50 con 0x7f75a410d120 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.153+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f759c004580 con 0x7f75a410d120 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.153+0000 7f75a1ffb640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 0x7f75940b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.153+0000 7f75a8f0d640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 0x7f75940b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.153+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f759c1070d0 con 0x7f75a410d120 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.154+0000 7f75a8f0d640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 0x7f75940b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7598005fd0 tx=0x7f759800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.156+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f759c0cdf30 con 0x7f75a410d120 2024-09-10T22:46:11.546 INFO:teuthology.orchestra.run.smithi096.stdout:55834574923 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.357+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f75a419a2c0 con 0x7f75a410d120 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.358+0000 7f75a1ffb640 1 -- 172.21.15.96:0/2972290901 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f759c0098b0 con 0x7f75a410d120 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 msgr2=0x7f75940b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 0x7f75940b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7598005fd0 tx=0x7f759800b390 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 msgr2=0x7f75a419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f759c005cc0 tx=0x7f759c02f380 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 shutdown_connections 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75940b5370 0x7f75940b7830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75a410d120 0x7f75a419ea70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75a4104570 0x7f75a41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 --2- 172.21.15.96:0/2972290901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75a4103b70 0x7f75a4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:11.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 >> 172.21.15.96:0/2972290901 conn(0x7f75a40fd700 msgr2=0x7f75a410f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:11.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 shutdown_connections 2024-09-10T22:46:11.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.360+0000 7f75ab171640 1 -- 172.21.15.96:0/2972290901 wait complete. 2024-09-10T22:46:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:11 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/982483990' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:46:12.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-10T22:46:12.195 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:12.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 -- 172.21.15.96:0/2570164155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5128102de0 msgr2=0x7f51281031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 --2- 172.21.15.96:0/2570164155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5128102de0 0x7f51281031c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5118009900 tx=0x7f511802a410 comp rx=0 tx=0).stop 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 -- 172.21.15.96:0/2570164155 shutdown_connections 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 --2- 172.21.15.96:0/2570164155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51281087f0 0x7f5128108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 --2- 172.21.15.96:0/2570164155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 0x7f512810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 --2- 172.21.15.96:0/2570164155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5128102de0 0x7f51281031c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 -- 172.21.15.96:0/2570164155 >> 172.21.15.96:0/2570164155 conn(0x7f51280fe830 msgr2=0x7f5128100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 -- 172.21.15.96:0/2570164155 shutdown_connections 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.252+0000 7f512e77e640 1 -- 172.21.15.96:0/2570164155 wait complete. 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.253+0000 7f512e77e640 1 Processor -- start 2024-09-10T22:46:12.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.253+0000 7f512e77e640 1 -- start start 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 0x7f5128199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 0x7f512819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112e30 con 0x7f51281087f0 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112cb0 con 0x7f5128102de0 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512e77e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5128112fb0 con 0x7f5128109170 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45926/0 (socket says 172.21.15.96:45926) 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 -- 172.21.15.96:0/2407848391 learned_addr learned my addr 172.21.15.96:0/2407848391 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f512cd1b640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 0x7f512819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f5127fff640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 0x7f5128199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stdout:197568495659 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 -- 172.21.15.96:0/2407848391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 msgr2=0x7f512819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 0x7f512819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 -- 172.21.15.96:0/2407848391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 msgr2=0x7f5128199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 0x7f5128199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.254+0000 7f51277fe640 1 -- 172.21.15.96:0/2407848391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512819f330 con 0x7f51281087f0 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f5127fff640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 0x7f5128199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f51277fe640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d950 tx=0x7f511c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c017030 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512819f620 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f511c004020 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0047a0 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.255+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281a5eb0 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.256+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51281031c0 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.257+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f511c009a30 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.259+0000 7f51257fa640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 0x7f51100b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.259+0000 7f5127fff640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 0x7f51100b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.259+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f511c0e5840 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.259+0000 7f5127fff640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 0x7f51100b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5118031440 tx=0x7f5118005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.262+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f511c0ac5a0 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.470+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f512810af50 con 0x7f51281087f0 2024-09-10T22:46:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.471+0000 7f51257fa640 1 -- 172.21.15.96:0/2407848391 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f511c0abea0 con 0x7f51281087f0 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 msgr2=0x7f51100b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 0x7f51100b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5118031440 tx=0x7f5118005a20 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 msgr2=0x7f51281997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d950 tx=0x7f511c00de10 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 shutdown_connections 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f51100b5370 0x7f51100b7830 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5128109170 0x7f512819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51281087f0 0x7f51281997c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 --2- 172.21.15.96:0/2407848391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5128102de0 0x7f5128199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 >> 172.21.15.96:0/2407848391 conn(0x7f51280fe830 msgr2=0x7f5128100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 shutdown_connections 2024-09-10T22:46:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.473+0000 7f512e77e640 1 -- 172.21.15.96:0/2407848391 wait complete. 2024-09-10T22:46:12.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:12 smithi096 ceph-mon[30869]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:12.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2972290901' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:46:12.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2407848391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:46:12.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:12 smithi121 ceph-mon[35119]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:12.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2972290901' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:46:12.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2407848391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:46:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:12 smithi117 ceph-mon[35081]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2972290901' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:46:12.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2407848391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 -- 172.21.15.96:0/729670833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c101cb0 msgr2=0x7fcb0c102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 --2- 172.21.15.96:0/729670833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c102090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802a410 comp rx=0 tx=0).stop 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 -- 172.21.15.96:0/729670833 shutdown_connections 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 --2- 172.21.15.96:0/729670833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c108040 0x7fcb0c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 --2- 172.21.15.96:0/729670833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 0x7fcb0c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 --2- 172.21.15.96:0/729670833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c102090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 -- 172.21.15.96:0/729670833 >> 172.21.15.96:0/729670833 conn(0x7fcb0c0fd700 msgr2=0x7fcb0c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 -- 172.21.15.96:0/729670833 shutdown_connections 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.844+0000 7fcb14587640 1 -- 172.21.15.96:0/729670833 wait complete. 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 Processor -- start 2024-09-10T22:46:12.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 -- start start 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 0x7fcb0c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c111d00 con 0x7fcb0c108040 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c111b80 con 0x7fcb0c101cb0 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb14587640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c111e80 con 0x7fcb0c1076c0 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45946/0 (socket says 172.21.15.96:45946) 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 -- 172.21.15.96:0/664066501 learned_addr learned my addr 172.21.15.96:0/664066501 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb11b22640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 0x7fcb0c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12323640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 -- 172.21.15.96:0/664066501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 msgr2=0x7fcb0c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 0x7fcb0c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.845+0000 7fcb12b24640 1 -- 172.21.15.96:0/664066501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c101cb0 msgr2=0x7fcb0c19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb12b24640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb12b24640 1 -- 172.21.15.96:0/664066501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a54f0 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb12b24640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009fd0 tx=0x7fcb0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04017070 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb04003e20 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a57e0 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04010460 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.846+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1a5d20 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.847+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c102090 con 0x7fcb0c108040 2024-09-10T22:46:12.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.848+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fcb04003970 con 0x7fcb0c108040 2024-09-10T22:46:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.849+0000 7fcb037fe640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 0x7fcaf80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.849+0000 7fcb12323640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 0x7fcaf80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.849+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcb040e58c0 con 0x7fcb0c108040 2024-09-10T22:46:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.849+0000 7fcb12323640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 0x7fcaf80b78b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb08031440 tx=0x7fcb08005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:12.852+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb040ac150 con 0x7fcb0c108040 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 -- 172.21.15.96:0/4212307108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0348108020 msgr2=0x7f034810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 --2- 172.21.15.96:0/4212307108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0348108020 0x7f034810e820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0340009ca0 tx=0x7f034002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 -- 172.21.15.96:0/4212307108 shutdown_connections 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 --2- 172.21.15.96:0/4212307108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0348108020 0x7f034810e820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 --2- 172.21.15.96:0/4212307108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03481076a0 0x7f0348107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 --2- 172.21.15.96:0/4212307108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348101c90 0x7f0348102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 -- 172.21.15.96:0/4212307108 >> 172.21.15.96:0/4212307108 conn(0x7f03480fd700 msgr2=0x7f03480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 -- 172.21.15.96:0/4212307108 shutdown_connections 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.024+0000 7f034f216640 1 -- 172.21.15.96:0/4212307108 wait complete. 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 Processor -- start 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 -- start start 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 0x7f03481992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348108020 0x7f034819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0348111ce0 con 0x7f03481076a0 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0348111b60 con 0x7f0348108020 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034f216640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0348111e60 con 0x7f0348101c90 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45888/0 (socket says 172.21.15.96:45888) 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 -- 172.21.15.96:0/982483990 learned_addr learned my addr 172.21.15.96:0/982483990 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034d7b3640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348108020 0x7f034819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f034cfb2640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 0x7f03481992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stdout:34359738448 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 -- 172.21.15.96:0/982483990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 msgr2=0x7f03481992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 0x7f03481992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 -- 172.21.15.96:0/982483990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348108020 msgr2=0x7f034819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348108020 0x7f034819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.025+0000 7f0347fff640 1 -- 172.21.15.96:0/982483990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034819f390 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f034cfb2640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 0x7f03481992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f0347fff640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f033400b4a0 tx=0x7f033400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334014070 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0334010ac0 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034819f680 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334017c30 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.026+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a5dc0 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.027+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0348102070 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.028+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f0334010610 con 0x7f03481076a0 2024-09-10T22:46:12.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.029+0000 7f0345ffb640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 0x7f03380b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.029+0000 7f034cfb2640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 0x7f03380b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.029+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f03340e5520 con 0x7f03481076a0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.029+0000 7f034cfb2640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 0x7f03380b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f033c009c20 tx=0x7f033c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.033+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03340ac380 con 0x7f03481076a0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.221+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f03481a60f0 con 0x7f03481076a0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.222+0000 7f0345ffb640 1 -- 172.21.15.96:0/982483990 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03340abc80 con 0x7f03481076a0 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 msgr2=0x7f03380b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 0x7f03380b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f033c009c20 tx=0x7f033c005f90 comp rx=0 tx=0).stop 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 msgr2=0x7f0348199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f033400b4a0 tx=0x7f033400b960 comp rx=0 tx=0).stop 2024-09-10T22:46:12.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 shutdown_connections 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0348108020 0x7f034819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f03380b5370 0x7f03380b7830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03481076a0 0x7f0348199820 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 --2- 172.21.15.96:0/982483990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0348101c90 0x7f03481992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 >> 172.21.15.96:0/982483990 conn(0x7f03480fd700 msgr2=0x7f0348104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 shutdown_connections 2024-09-10T22:46:12.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:11.225+0000 7f034f216640 1 -- 172.21.15.96:0/982483990 wait complete. 2024-09-10T22:46:13.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.036+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcb0c02e5c0 con 0x7fcb0c108040 2024-09-10T22:46:13.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.036+0000 7fcb037fe640 1 -- 172.21.15.96:0/664066501 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb040b0b80 con 0x7fcb0c108040 2024-09-10T22:46:13.038 INFO:teuthology.orchestra.run.smithi096.stdout:128849018940 2024-09-10T22:46:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 msgr2=0x7fcaf80b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 0x7fcaf80b78b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb08031440 tx=0x7fcb08005b50 comp rx=0 tx=0).stop 2024-09-10T22:46:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 msgr2=0x7fcb0c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009fd0 tx=0x7fcb0400b260 comp rx=0 tx=0).stop 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 shutdown_connections 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcaf80b53f0 0x7fcaf80b78b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb0c108040 0x7fcb0c19abb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb0c1076c0 0x7fcb0c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 --2- 172.21.15.96:0/664066501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb0c101cb0 0x7fcb0c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 >> 172.21.15.96:0/664066501 conn(0x7fcb0c0fd700 msgr2=0x7fcb0c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 shutdown_connections 2024-09-10T22:46:13.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:13.039+0000 7fcb14587640 1 -- 172.21.15.96:0/664066501 wait complete. 2024-09-10T22:46:13.288 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/664066501' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:46:13.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738448 for osd.0 2024-09-10T22:46:13.635 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:13.665 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/664066501' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:46:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/664066501' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 -- 172.21.15.96:0/4060977072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 msgr2=0x7f5d5c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 --2- 172.21.15.96:0/4060977072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c107b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009900 tx=0x7f5d4c02a410 comp rx=0 tx=0).stop 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 -- 172.21.15.96:0/4060977072 shutdown_connections 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 --2- 172.21.15.96:0/4060977072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c108040 0x7f5d5c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 --2- 172.21.15.96:0/4060977072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c107b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 --2- 172.21.15.96:0/4060977072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c101cb0 0x7f5d5c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 -- 172.21.15.96:0/4060977072 >> 172.21.15.96:0/4060977072 conn(0x7f5d5c0fd740 msgr2=0x7f5d5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 -- 172.21.15.96:0/4060977072 shutdown_connections 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 -- 172.21.15.96:0/4060977072 wait complete. 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.341+0000 7f5d63c28640 1 Processor -- start 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 -- start start 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c101cb0 0x7f5d5c0fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 0x7f5d5c0fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111d00 con 0x7f5d5c1076c0 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111b80 con 0x7f5d5c101cb0 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d63c28640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111e80 con 0x7f5d5c108040 2024-09-10T22:46:14.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45968/0 (socket says 172.21.15.96:45968) 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 -- 172.21.15.96:0/1296101100 learned_addr learned my addr 172.21.15.96:0/1296101100 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d619c4640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c101cb0 0x7f5d5c0fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d621c5640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 0x7f5d5c0fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 -- 172.21.15.96:0/1296101100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 msgr2=0x7f5d5c0fe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 0x7f5d5c0fe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 -- 172.21.15.96:0/1296101100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c101cb0 msgr2=0x7f5d5c0fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c101cb0 0x7f5d5c0fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 -- 172.21.15.96:0/1296101100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c1a7700 con 0x7f5d5c1076c0 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d621c5640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 0x7f5d5c0fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.342+0000 7f5d611c3640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005940 tx=0x7f5d4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.343+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c03f030 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.343+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c1a7990 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.343+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4c003cb0 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.343+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c035070 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.343+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c1a7e70 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.344+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c102f60 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.345+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f5d4c03bbf0 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.346+0000 7f5d52ffd640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 0x7f5d480b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.346+0000 7f5d619c4640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 0x7f5d480b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.346+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5d4c106030 con 0x7f5d5c1076c0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.346+0000 7f5d619c4640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 0x7f5d480b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005fd0 tx=0x7f5d5800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.350+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5d4c0cce90 con 0x7f5d5c1076c0 2024-09-10T22:46:14.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.532+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5d5c0fef40 con 0x7f5d5c1076c0 2024-09-10T22:46:14.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.533+0000 7f5d52ffd640 1 -- 172.21.15.96:0/1296101100 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d5c0fef40 con 0x7f5d5c1076c0 2024-09-10T22:46:14.534 INFO:teuthology.orchestra.run.smithi096.stdout:103079215169 2024-09-10T22:46:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 msgr2=0x7f5d480b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 0x7f5d480b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d58005fd0 tx=0x7f5d5800b390 comp rx=0 tx=0).stop 2024-09-10T22:46:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 msgr2=0x7f5d5c0fe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005940 tx=0x7f5d4c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 shutdown_connections 2024-09-10T22:46:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5d480b5370 0x7f5d480b7830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c108040 0x7f5d5c0fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.537 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:14 smithi096 ceph-mon[30869]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d5c1076c0 0x7f5d5c0fe180 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 --2- 172.21.15.96:0/1296101100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d5c101cb0 0x7f5d5c0fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 >> 172.21.15.96:0/1296101100 conn(0x7f5d5c0fd740 msgr2=0x7f5d5c06b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 shutdown_connections 2024-09-10T22:46:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:14.535+0000 7f5d63c28640 1 -- 172.21.15.96:0/1296101100 wait complete. 2024-09-10T22:46:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:14 smithi121 ceph-mon[35119]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:14.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:14 smithi117 ceph-mon[35081]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 -- 172.21.15.96:0/3453954860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c109170 msgr2=0x7f459c10f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 --2- 172.21.15.96:0/3453954860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c109170 0x7f459c10f970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ca0 tx=0x7f459402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 -- 172.21.15.96:0/3453954860 shutdown_connections 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 --2- 172.21.15.96:0/3453954860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 --2- 172.21.15.96:0/3453954860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c109170 0x7f459c10f970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 --2- 172.21.15.96:0/3453954860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c102de0 0x7f459c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 -- 172.21.15.96:0/3453954860 >> 172.21.15.96:0/3453954860 conn(0x7f459c0fe830 msgr2=0x7f459c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 -- 172.21.15.96:0/3453954860 shutdown_connections 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 -- 172.21.15.96:0/3453954860 wait complete. 2024-09-10T22:46:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.433+0000 7f45a39da640 1 Processor -- start 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 -- start start 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c109170 0x7f459c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f459c112e30 con 0x7f459c102de0 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f459c112cb0 con 0x7f459c1087f0 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a39da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f459c112fb0 con 0x7f459c109170 2024-09-10T22:46:15.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45988/0 (socket says 172.21.15.96:45988) 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 -- 172.21.15.96:0/1080873730 learned_addr learned my addr 172.21.15.96:0/1080873730 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a0f75640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1f77640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c109170 0x7f459c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 -- 172.21.15.96:0/1080873730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c109170 msgr2=0x7f459c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c109170 0x7f459c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 -- 172.21.15.96:0/1080873730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 msgr2=0x7f459c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 -- 172.21.15.96:0/1080873730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c19f290 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a0f75640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45a1776640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f459800c890 tx=0x7f459800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.434+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598004220 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.435+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c19f580 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.435+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45980043c0 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.435+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598004a00 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.435+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c1a5e10 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.435+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c1031c0 con 0x7f459c102de0 2024-09-10T22:46:15.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.437+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f45980079b0 con 0x7f459c102de0 2024-09-10T22:46:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.437+0000 7f45927fc640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 0x7f45880b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.438+0000 7f45a0f75640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 0x7f45880b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.438+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f45980e5de0 con 0x7f459c102de0 2024-09-10T22:46:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.438+0000 7f45a0f75640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 0x7f45880b78b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f459c100fd0 tx=0x7f458c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.440+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4598014030 con 0x7f459c102de0 2024-09-10T22:46:15.564 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1296101100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:46:15.564 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:15 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:15.596 INFO:teuthology.orchestra.run.smithi096.stdout:176093659184 2024-09-10T22:46:15.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T22:46:15.640 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1296101100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:46:15.640 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:15 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:15.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.643+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f459c02e5c0 con 0x7f459c102de0 2024-09-10T22:46:15.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.643+0000 7f45927fc640 1 -- 172.21.15.96:0/1080873730 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f45980accc0 con 0x7f459c102de0 2024-09-10T22:46:15.645 INFO:teuthology.orchestra.run.smithi096.stdout:296352743447 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.645+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 msgr2=0x7f45880b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.645+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 0x7f45880b78b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f459c100fd0 tx=0x7f458c00a5c0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 msgr2=0x7f459c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f459800c890 tx=0x7f459800cd50 comp rx=0 tx=0).stop 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 shutdown_connections 2024-09-10T22:46:15.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f459c1087f0 0x7f459c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f45880b53f0 0x7f45880b78b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f459c109170 0x7f459c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 --2- 172.21.15.96:0/1080873730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f459c102de0 0x7f459c1991e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 >> 172.21.15.96:0/1080873730 conn(0x7f459c0fe830 msgr2=0x7f459c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 shutdown_connections 2024-09-10T22:46:15.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:15.646+0000 7f45a39da640 1 -- 172.21.15.96:0/1080873730 wait complete. 2024-09-10T22:46:15.658 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1296101100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:46:15.658 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:15 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:46:15.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495659 got 197568495659 for osd.7 2024-09-10T22:46:15.897 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:16.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018937 got 128849018940 for osd.4 2024-09-10T22:46:16.028 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:16.465 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:16 smithi096 ceph-mon[30869]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:16.465 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:16 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1080873730' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:46:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:16 smithi121 ceph-mon[35119]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:16 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1080873730' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:46:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:16 smithi117 ceph-mon[35081]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:46:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:16 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1080873730' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:46:17.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.11 2024-09-10T22:46:17.049 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:17.454 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:17.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215166 got 103079215169 for osd.3 2024-09-10T22:46:17.486 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:17.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.727 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.727 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.727 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/856941553' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:46:17.727 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:17 smithi096 ceph-mon[30869]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:46:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/856941553' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:46:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:17 smithi121 ceph-mon[35119]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/856941553' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:46:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:17 smithi117 ceph-mon[35081]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T22:46:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 -- 172.21.15.96:0/1428691107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c1045c0 msgr2=0x7f476c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 --2- 172.21.15.96:0/1428691107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c1045c0 0x7f476c10cab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4760009900 tx=0x7f476002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:17.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 -- 172.21.15.96:0/1428691107 shutdown_connections 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 --2- 172.21.15.96:0/1428691107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c10d100 0x7f476c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 --2- 172.21.15.96:0/1428691107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c1045c0 0x7f476c10cab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 --2- 172.21.15.96:0/1428691107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c103ca0 0x7f476c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 -- 172.21.15.96:0/1428691107 >> 172.21.15.96:0/1428691107 conn(0x7f476c0fd6e0 msgr2=0x7f476c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 -- 172.21.15.96:0/1428691107 shutdown_connections 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.463+0000 7f477247c640 1 -- 172.21.15.96:0/1428691107 wait complete. 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 Processor -- start 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 -- start start 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c1045c0 0x7f476c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c10d100 0x7f476c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f476c112d70 con 0x7f476c103ca0 2024-09-10T22:46:17.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f476c112bf0 con 0x7f476c1045c0 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.464+0000 7f477247c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f476c112ef0 con 0x7f476c10d100 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41554/0 (socket says 172.21.15.96:41554) 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 -- 172.21.15.96:0/856941553 learned_addr learned my addr 172.21.15.96:0/856941553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f4770a19640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c10d100 0x7f476c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476b7fe640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c1045c0 0x7f476c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 -- 172.21.15.96:0/856941553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c10d100 msgr2=0x7f476c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c10d100 0x7f476c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 -- 172.21.15.96:0/856941553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c1045c0 msgr2=0x7f476c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c1045c0 0x7f476c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 -- 172.21.15.96:0/856941553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c19f3f0 con 0x7f476c103ca0 2024-09-10T22:46:17.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f476bfff640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f475c00e9f0 tx=0x7f475c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00cd80 con 0x7f476c103ca0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c19f6e0 con 0x7f476c103ca0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c0108e0 con 0x7f476c103ca0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c018980 con 0x7f476c103ca0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stdout:249108103203 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.465+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a5da0 con 0x7f476c103ca0 2024-09-10T22:46:17.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.466+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c104080 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.468+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f475c010430 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.468+0000 7f47697fa640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 0x7f47540b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.468+0000 7f476b7fe640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 0x7f47540b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.468+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f475c0e63f0 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.469+0000 7f476b7fe640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 0x7f47540b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4760000c00 tx=0x7f4760031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.471+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f475c0acc00 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.690+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f476c19a470 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.690+0000 7f47697fa640 1 -- 172.21.15.96:0/856941553 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f475c0b1630 con 0x7f476c103ca0 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.692+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 msgr2=0x7f47540b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.692+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 0x7f47540b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4760000c00 tx=0x7f4760031400 comp rx=0 tx=0).stop 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 msgr2=0x7f476c199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:17.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f475c00e9f0 tx=0x7f475c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 shutdown_connections 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f476c1045c0 0x7f476c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f47540b5370 0x7f47540b7830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f476c10d100 0x7f476c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 --2- 172.21.15.96:0/856941553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f476c103ca0 0x7f476c199340 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 >> 172.21.15.96:0/856941553 conn(0x7f476c0fd6e0 msgr2=0x7f476c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 shutdown_connections 2024-09-10T22:46:17.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:16.693+0000 7f477247c640 1 -- 172.21.15.96:0/856941553 wait complete. 2024-09-10T22:46:18.504 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103203 for osd.9 2024-09-10T22:46:18.505 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:18.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 -- 172.21.15.96:0/3596843263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 msgr2=0x7f55c410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:18.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 --2- 172.21.15.96:0/3596843263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c410f560 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009ca0 tx=0x7f55bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 -- 172.21.15.96:0/3596843263 shutdown_connections 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 --2- 172.21.15.96:0/3596843263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 0x7f55c410cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 --2- 172.21.15.96:0/3596843263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c410f560 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 --2- 172.21.15.96:0/3596843263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 0x7f55c4103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 -- 172.21.15.96:0/3596843263 >> 172.21.15.96:0/3596843263 conn(0x7f55c40fd6d0 msgr2=0x7f55c40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 -- 172.21.15.96:0/3596843263 shutdown_connections 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.789+0000 7f55c99ea640 1 -- 172.21.15.96:0/3596843263 wait complete. 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 Processor -- start 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 -- start start 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 0x7f55c4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 0x7f55c4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:18.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112db0 con 0x7f55c410d130 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112c30 con 0x7f55c4104580 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c99ea640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112f30 con 0x7f55c4103b80 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41572/0 (socket says 172.21.15.96:41572) 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c37fe640 1 -- 172.21.15.96:0/2770403814 learned_addr learned my addr 172.21.15.96:0/2770403814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.790+0000 7f55c2ffd640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 0x7f55c4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c27fc640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 0x7f55c4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 -- 172.21.15.96:0/2770403814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 msgr2=0x7f55c4101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 0x7f55c4101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 -- 172.21.15.96:0/2770403814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 msgr2=0x7f55c4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 0x7f55c4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 -- 172.21.15.96:0/2770403814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a76e0 con 0x7f55c410d130 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c37fe640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005cc0 tx=0x7f55bc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:18.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003bf0 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a7970 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55bc003d90 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc02fbc0 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.791+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a7f00 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.792+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c4103f60 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.793+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f55bc004610 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.794+0000 7f55c89e8640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 0x7f55ac0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.794+0000 7f55c2ffd640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 0x7f55ac0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.794+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55bc107370 con 0x7f55c410d130 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.794+0000 7f55c2ffd640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 0x7f55ac0b7830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009890 tx=0x7f55b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:18.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.797+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f55bc0ce1d0 con 0x7f55c410d130 2024-09-10T22:46:18.831 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:18.832 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:18.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.998+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f55c4102d00 con 0x7f55c410d130 2024-09-10T22:46:19.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:18.998+0000 7f55c89e8640 1 -- 172.21.15.96:0/2770403814 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f55c4102d00 con 0x7f55c410d130 2024-09-10T22:46:19.000 INFO:teuthology.orchestra.run.smithi096.stdout:270582939677 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 msgr2=0x7f55ac0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 0x7f55ac0b7830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009890 tx=0x7f55b8008040 comp rx=0 tx=0).stop 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 msgr2=0x7f55c4102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005cc0 tx=0x7f55bc02f380 comp rx=0 tx=0).stop 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 shutdown_connections 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4104580 0x7f55c4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55ac0b5370 0x7f55ac0b7830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55c410d130 0x7f55c4102480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 --2- 172.21.15.96:0/2770403814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c4103b80 0x7f55c4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.000+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 >> 172.21.15.96:0/2770403814 conn(0x7f55c40fd6d0 msgr2=0x7f55c410f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.001+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 shutdown_connections 2024-09-10T22:46:19.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.001+0000 7f55c99ea640 1 -- 172.21.15.96:0/2770403814 wait complete. 2024-09-10T22:46:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:19 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2770403814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:46:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:19 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2770403814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:46:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:19 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2770403814' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:46:19.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 -- 172.21.15.96:0/3012772668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 msgr2=0x7f1d8010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 --2- 172.21.15.96:0/3012772668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8010f960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009ca0 tx=0x7f1d7802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 -- 172.21.15.96:0/3012772668 shutdown_connections 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 --2- 172.21.15.96:0/3012772668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d801087e0 0x7f1d80108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 --2- 172.21.15.96:0/3012772668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8010f960 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 --2- 172.21.15.96:0/3012772668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d80102dd0 0x7f1d801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 -- 172.21.15.96:0/3012772668 >> 172.21.15.96:0/3012772668 conn(0x7f1d800fe800 msgr2=0x7f1d80100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 -- 172.21.15.96:0/3012772668 shutdown_connections 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.733+0000 7f1d87077640 1 -- 172.21.15.96:0/3012772668 wait complete. 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 Processor -- start 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 -- start start 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d80102dd0 0x7f1d801992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d801087e0 0x7f1d80199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80112e70 con 0x7f1d80109160 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80112cf0 con 0x7f1d80102dd0 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d87077640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80112ff0 con 0x7f1d801087e0 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41600/0 (socket says 172.21.15.96:41600) 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 -- 172.21.15.96:0/510486735 learned_addr learned my addr 172.21.15.96:0/510486735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d7ffff640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d801087e0 0x7f1d80199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d84e13640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d80102dd0 0x7f1d801992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 -- 172.21.15.96:0/510486735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d801087e0 msgr2=0x7f1d80199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d801087e0 0x7f1d80199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 -- 172.21.15.96:0/510486735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d80102dd0 msgr2=0x7f1d801992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d80102dd0 0x7f1d801992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 -- 172.21.15.96:0/510486735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8019f390 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.734+0000 7f1d85614640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009430 tx=0x7f1d780317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.735+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78003b70 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.735+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d78003d10 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.735+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8019f620 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.735+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7802fa30 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.735+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d801a5d40 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.736+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d801031b0 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.737+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f1d7802fbd0 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.738+0000 7f1d7dffb640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 0x7f1d700b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.738+0000 7f1d84e13640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 0x7f1d700b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.738+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1d78106e90 con 0x7f1d80109160 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.738+0000 7f1d84e13640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 0x7f1d700b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009bd0 tx=0x7f1d74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.742+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1d780cdcf0 con 0x7f1d80109160 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.621+0000 7ff7514fe640 1 -- 172.21.15.96:0/2440831863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 msgr2=0x7ff74c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.621+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2440831863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c10cab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff740009ca0 tx=0x7ff74002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 -- 172.21.15.96:0/2440831863 shutdown_connections 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2440831863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 0x7ff74c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2440831863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c10cab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2440831863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 -- 172.21.15.96:0/2440831863 >> 172.21.15.96:0/2440831863 conn(0x7ff74c0fd700 msgr2=0x7ff74c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 -- 172.21.15.96:0/2440831863 shutdown_connections 2024-09-10T22:46:19.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 -- 172.21.15.96:0/2440831863 wait complete. 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 Processor -- start 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 -- start start 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.622+0000 7ff7514fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 0x7ff74c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c112d70 con 0x7ff74c1045c0 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c112bf0 con 0x7ff74c103ca0 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c112ef0 con 0x7ff74c10d100 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41582/0 (socket says 172.21.15.96:41582) 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 -- 172.21.15.96:0/2001322421 learned_addr learned my addr 172.21.15.96:0/2001322421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:19.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74affd640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74b7fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 0x7ff74c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 -- 172.21.15.96:0/2001322421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 msgr2=0x7ff74c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 0x7ff74c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 -- 172.21.15.96:0/2001322421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 msgr2=0x7ff74c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 -- 172.21.15.96:0/2001322421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c1a76c0 con 0x7ff74c1045c0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74affd640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff74a7fc640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff740033040 tx=0x7ff7400316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740003b70 con 0x7ff74c1045c0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff740003d10 con 0x7ff74c1045c0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c1a7950 con 0x7ff74c1045c0 2024-09-10T22:46:19.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74002fa80 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.623+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a7e90 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.624+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c104080 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.626+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff74002fc20 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.626+0000 7ff733fff640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 0x7ff7340b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.627+0000 7ff74affd640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 0x7ff7340b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.627+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff740107080 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.627+0000 7ff74affd640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 0x7ff7340b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0058c0 tx=0x7ff73c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.629+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff7400cdee0 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.826+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff74c02e5c0 con 0x7ff74c1045c0 2024-09-10T22:46:19.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.827+0000 7ff733fff640 1 -- 172.21.15.96:0/2001322421 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff7400cd7e0 con 0x7ff74c1045c0 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stdout:150323855414 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.829+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 msgr2=0x7ff7340b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.829+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 0x7ff7340b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0058c0 tx=0x7ff73c005ec0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.829+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 msgr2=0x7ff74c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff740033040 tx=0x7ff7400316f0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 shutdown_connections 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7340b5370 0x7ff7340b7830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff74c10d100 0x7ff74c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff74c1045c0 0x7ff74c101e90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 --2- 172.21.15.96:0/2001322421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff74c103ca0 0x7ff74c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 >> 172.21.15.96:0/2001322421 conn(0x7ff74c0fd700 msgr2=0x7ff74c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 shutdown_connections 2024-09-10T22:46:19.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.830+0000 7ff7514fe640 1 -- 172.21.15.96:0/2001322421 wait complete. 2024-09-10T22:46:19.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.938+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1d8010af70 con 0x7f1d80109160 2024-09-10T22:46:19.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.938+0000 7f1d7dffb640 1 -- 172.21.15.96:0/510486735 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d780cd5f0 con 0x7f1d80109160 2024-09-10T22:46:19.940 INFO:teuthology.orchestra.run.smithi096.stdout:77309411399 2024-09-10T22:46:19.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 msgr2=0x7f1d700b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 0x7f1d700b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009bd0 tx=0x7f1d74005f90 comp rx=0 tx=0).stop 2024-09-10T22:46:19.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 msgr2=0x7f1d8019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:19.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009430 tx=0x7f1d780317c0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 shutdown_connections 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1d700b5370 0x7f1d700b7830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d80109160 0x7f1d8019ebc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d801087e0 0x7f1d80199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 --2- 172.21.15.96:0/510486735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d80102dd0 0x7f1d801992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 >> 172.21.15.96:0/510486735 conn(0x7f1d800fe800 msgr2=0x7f1d80100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 shutdown_connections 2024-09-10T22:46:19.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:19.941+0000 7f1d87077640 1 -- 172.21.15.96:0/510486735 wait complete. 2024-09-10T22:46:20.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:20 smithi096 ceph-mon[30869]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:46:20.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2001322421' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:46:20.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/510486735' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:46:20.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:20 smithi121 ceph-mon[35119]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:46:20.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:20 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2001322421' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:46:20.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:20 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/510486735' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:46:20.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:20 smithi117 ceph-mon[35081]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:46:20.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:20 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2001322421' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:46:20.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:20 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/510486735' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:46:20.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855412 got 150323855414 for osd.5 2024-09-10T22:46:20.834 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:21.053 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:21.056 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:21.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939675 got 270582939677 for osd.10 2024-09-10T22:46:21.497 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:21.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:21 smithi096 ceph-mon[30869]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:21 smithi121 ceph-mon[35119]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:21 smithi117 ceph-mon[35081]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:21.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411399 for osd.2 2024-09-10T22:46:21.959 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- 172.21.15.96:0/774101331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c4108060 msgr2=0x7f94c410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- 172.21.15.96:0/774101331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c4108060 0x7f94c410e860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009ca0 tx=0x7f94bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- 172.21.15.96:0/774101331 shutdown_connections 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- 172.21.15.96:0/774101331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c41076e0 0x7f94c4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- 172.21.15.96:0/774101331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c4108060 0x7f94c410e860 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- 172.21.15.96:0/774101331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4101cd0 0x7f94c41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- 172.21.15.96:0/774101331 >> 172.21.15.96:0/774101331 conn(0x7f94c40fd700 msgr2=0x7f94c40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- 172.21.15.96:0/774101331 shutdown_connections 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- 172.21.15.96:0/774101331 wait complete. 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 Processor -- start 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 -- start start 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c4101cd0 0x7f94c419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.438+0000 7f94ca391640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4108060 0x7f94c41a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4111d60 con 0x7f94c41076e0 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4111be0 con 0x7f94c4101cd0 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4111ee0 con 0x7f94c4108060 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:22.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41620/0 (socket says 172.21.15.96:41620) 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 -- 172.21.15.96:0/141316969 learned_addr learned my addr 172.21.15.96:0/141316969 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94c892e640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4108060 0x7f94c41a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94c3fff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c4101cd0 0x7f94c419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 -- 172.21.15.96:0/141316969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4108060 msgr2=0x7f94c41a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4108060 0x7f94c41a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 -- 172.21.15.96:0/141316969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c4101cd0 msgr2=0x7f94c419b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c4101cd0 0x7f94c419b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 -- 172.21.15.96:0/141316969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41a1660 con 0x7f94c41076e0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94bbfff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00b4a0 tx=0x7f94ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac014070 con 0x7f94c41076e0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c41a1950 con 0x7f94c41076e0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94ac010ac0 con 0x7f94c41076e0 2024-09-10T22:46:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac017c30 con 0x7f94c41076e0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.439+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c41a8090 con 0x7f94c41076e0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.440+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c4102130 con 0x7f94c41076e0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.442+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f94ac010610 con 0x7f94c41076e0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.442+0000 7f94c1ffb640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 0x7f94b00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.443+0000 7f94c3fff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 0x7f94b00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.443+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f94ac0e5530 con 0x7f94c41076e0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.443+0000 7f94c3fff640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 0x7f94b00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009890 tx=0x7f94b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.445+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f94ac0ac390 con 0x7f94c41076e0 2024-09-10T22:46:22.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.624+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f94c41a83b0 con 0x7f94c41076e0 2024-09-10T22:46:22.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.624+0000 7f94c1ffb640 1 -- 172.21.15.96:0/141316969 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94ac0abc90 con 0x7f94c41076e0 2024-09-10T22:46:22.626 INFO:teuthology.orchestra.run.smithi096.stdout:223338299433 2024-09-10T22:46:22.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 msgr2=0x7f94b00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:22.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 0x7f94b00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009890 tx=0x7f94b4008040 comp rx=0 tx=0).stop 2024-09-10T22:46:22.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 msgr2=0x7f94c419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:22.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00b4a0 tx=0x7f94ac00b960 comp rx=0 tx=0).stop 2024-09-10T22:46:22.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 shutdown_connections 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94b00b5370 0x7f94b00b7830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94c4108060 0x7f94c41a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c41076e0 0x7f94c419baf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 --2- 172.21.15.96:0/141316969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94c4101cd0 0x7f94c419b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.626+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 >> 172.21.15.96:0/141316969 conn(0x7f94c40fd700 msgr2=0x7f94c4104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.627+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 shutdown_connections 2024-09-10T22:46:22.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:22.627+0000 7f94ca391640 1 -- 172.21.15.96:0/141316969 wait complete. 2024-09-10T22:46:22.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:22 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/141316969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:46:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:22 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/141316969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:46:23.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:22 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/141316969' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:46:23.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 -- 172.21.15.96:0/3948948585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e010d100 msgr2=0x7f25e010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 --2- 172.21.15.96:0/3948948585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e010d100 0x7f25e010f560 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009ca0 tx=0x7f25d802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 -- 172.21.15.96:0/3948948585 shutdown_connections 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 --2- 172.21.15.96:0/3948948585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e010d100 0x7f25e010f560 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 --2- 172.21.15.96:0/3948948585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25e01045c0 0x7f25e010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 --2- 172.21.15.96:0/3948948585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e0103ca0 0x7f25e0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 -- 172.21.15.96:0/3948948585 >> 172.21.15.96:0/3948948585 conn(0x7f25e00fd700 msgr2=0x7f25e00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 -- 172.21.15.96:0/3948948585 shutdown_connections 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.261+0000 7f25e4dae640 1 -- 172.21.15.96:0/3948948585 wait complete. 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 Processor -- start 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 -- start start 2024-09-10T22:46:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e01045c0 0x7f25e0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25e010d100 0x7f25e019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112dd0 con 0x7f25e0103ca0 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112c50 con 0x7f25e01045c0 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25e4dae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112f50 con 0x7f25e010d100 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41636/0 (socket says 172.21.15.96:41636) 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 -- 172.21.15.96:0/1789458806 learned_addr learned my addr 172.21.15.96:0/1789458806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 -- 172.21.15.96:0/1789458806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25e010d100 msgr2=0x7f25e019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25e010d100 0x7f25e019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 -- 172.21.15.96:0/1789458806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e01045c0 msgr2=0x7f25e0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e01045c0 0x7f25e0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 -- 172.21.15.96:0/1789458806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e019f3f0 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25de59c640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d950 tx=0x7f25d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.262+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0017070 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.263+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0009c30 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.263+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e019f6e0 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.263+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d000f620 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.263+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e01a5da0 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.264+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e0104080 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.265+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f25d0010460 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.265+0000 7f25d77fe640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 0x7f25cc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.266+0000 7f25d7fff640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 0x7f25cc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.266+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f25d00e5950 con 0x7f25e0103ca0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.266+0000 7f25d7fff640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 0x7f25cc0b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25e00ffe00 tx=0x7f25c8006d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.268+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f25d00ac7b0 con 0x7f25e0103ca0 2024-09-10T22:46:23.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.452+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f25e019a470 con 0x7f25e0103ca0 2024-09-10T22:46:23.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.453+0000 7f25d77fe640 1 -- 172.21.15.96:0/1789458806 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25d0009de0 con 0x7f25e0103ca0 2024-09-10T22:46:23.455 INFO:teuthology.orchestra.run.smithi096.stdout:176093659187 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 msgr2=0x7f25cc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 0x7f25cc0b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25e00ffe00 tx=0x7f25c8006d30 comp rx=0 tx=0).stop 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 msgr2=0x7f25e0199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d950 tx=0x7f25d000de10 comp rx=0 tx=0).stop 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 shutdown_connections 2024-09-10T22:46:23.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e01045c0 0x7f25e0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f25cc0b5370 0x7f25cc0b7830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25e010d100 0x7f25e019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 --2- 172.21.15.96:0/1789458806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25e0103ca0 0x7f25e0199340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 >> 172.21.15.96:0/1789458806 conn(0x7f25e00fd700 msgr2=0x7f25e010f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 shutdown_connections 2024-09-10T22:46:23.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:23.455+0000 7f25e4dae640 1 -- 172.21.15.96:0/1789458806 wait complete. 2024-09-10T22:46:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:23 smithi096 ceph-mon[30869]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:23 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1789458806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:46:23.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299430 got 223338299433 for osd.8 2024-09-10T22:46:23.807 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:23 smithi121 ceph-mon[35119]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:24.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:23 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1789458806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:46:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:23 smithi117 ceph-mon[35081]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:24.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:23 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1789458806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:46:24.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659184 got 176093659187 for osd.6 2024-09-10T22:46:24.464 DEBUG:teuthology.parallel:result is None 2024-09-10T22:46:24.465 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:46:24.465 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:46:25.177 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:25.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:25 smithi096 ceph-mon[30869]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:25.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 -- 172.21.15.96:0/3910606980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac44104590 msgr2=0x7fac4410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 --2- 172.21.15.96:0/3910606980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac44104590 0x7fac4410cb70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 -- 172.21.15.96:0/3910606980 shutdown_connections 2024-09-10T22:46:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 --2- 172.21.15.96:0/3910606980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac4410d140 0x7fac4410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 --2- 172.21.15.96:0/3910606980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac44104590 0x7fac4410cb70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 --2- 172.21.15.96:0/3910606980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44103b90 0x7fac44103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 -- 172.21.15.96:0/3910606980 >> 172.21.15.96:0/3910606980 conn(0x7fac440fd700 msgr2=0x7fac440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 -- 172.21.15.96:0/3910606980 shutdown_connections 2024-09-10T22:46:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.665+0000 7fac4b014640 1 -- 172.21.15.96:0/3910606980 wait complete. 2024-09-10T22:46:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 Processor -- start 2024-09-10T22:46:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 -- start start 2024-09-10T22:46:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 0x7fac4419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44104590 0x7fac4419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac44112dc0 con 0x7fac4410d140 2024-09-10T22:46:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac44112c40 con 0x7fac44103b90 2024-09-10T22:46:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac4b014640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac44112f40 con 0x7fac44104590 2024-09-10T22:46:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac495b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac495b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41662/0 (socket says 172.21.15.96:41662) 2024-09-10T22:46:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac495b1640 1 -- 172.21.15.96:0/3457395918 learned_addr learned my addr 172.21.15.96:0/3457395918 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac43fff640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44104590 0x7fac4419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac48db0640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 0x7fac4419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac495b1640 1 -- 172.21.15.96:0/3457395918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44104590 msgr2=0x7fac4419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.666+0000 7fac495b1640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44104590 0x7fac4419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac495b1640 1 -- 172.21.15.96:0/3457395918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 msgr2=0x7fac4419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac495b1640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 0x7fac4419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac495b1640 1 -- 172.21.15.96:0/3457395918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac441a1530 con 0x7fac4410d140 2024-09-10T22:46:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac48db0640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 0x7fac4419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac495b1640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00ccb0 tx=0x7fac3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c00f450 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c00faa0 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac441a1820 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c018b00 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.667+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac441a7ee0 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.668+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac44103f70 con 0x7fac4410d140 2024-09-10T22:46:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.669+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fac3c00f5f0 con 0x7fac4410d140 2024-09-10T22:46:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.670+0000 7fac41ffb640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 0x7fac340b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.670+0000 7fac48db0640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 0x7fac340b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.670+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac3c0e6400 con 0x7fac4410d140 2024-09-10T22:46:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.670+0000 7fac48db0640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 0x7fac340b7830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.674+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac3c014030 con 0x7fac4410d140 2024-09-10T22:46:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:25 smithi121 ceph-mon[35119]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:25 smithi117 ceph-mon[35081]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:25.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.876+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac441030d0 con 0x7fac340b5370 2024-09-10T22:46:25.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.881+0000 7fac41ffb640 1 -- 172.21.15.96:0/3457395918 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258881 (secure 0 0 0) 0x7fac441030d0 con 0x7fac340b5370 2024-09-10T22:46:25.887 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:25.889 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 msgr2=0x7fac340b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 0x7fac340b7830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38008040 comp rx=0 tx=0).stop 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 msgr2=0x7fac441a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00ccb0 tx=0x7fac3c00a620 comp rx=0 tx=0).stop 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 shutdown_connections 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac340b5370 0x7fac340b7830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac4410d140 0x7fac441a0d60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac44104590 0x7fac4419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 --2- 172.21.15.96:0/3457395918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac44103b90 0x7fac4419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:25.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 >> 172.21.15.96:0/3457395918 conn(0x7fac440fd700 msgr2=0x7fac4410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:25.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 shutdown_connections 2024-09-10T22:46:25.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:25.885+0000 7fac4b014640 1 -- 172.21.15.96:0/3457395918 wait complete. 2024-09-10T22:46:26.591 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:26 smithi096 ceph-mon[30869]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:26.619 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T22:46:25.313440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656786,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":421,"num_write":365,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355592,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586232,"statfs":{"total":1151940427776,"available":1151576301568,"internally_reserved":0,"allocated":10665984,"data_stored":6827790,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003106"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598491+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:45:58.598491+0000","last_peered":"2024-09-10T22:45:58.598491+0000","last_clean":"2024-09-10T22:45:58.598491+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:45:58.598491+0000","last_undegraded":"2024-09-10T22:45:58.598491+0000","last_fullsized":"2024-09-10T22:45:58.598491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299573+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:45:58.299573+0000","last_peered":"2024-09-10T22:45:58.299573+0000","last_clean":"2024-09-10T22:45:58.299573+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:45:58.299573+0000","last_undegraded":"2024-09-10T22:45:58.299573+0000","last_fullsized":"2024-09-10T22:45:58.299573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299520+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:45:58.299520+0000","last_peered":"2024-09-10T22:45:58.299520+0000","last_clean":"2024-09-10T22:45:58.299520+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:45:58.299520+0000","last_undegraded":"2024-09-10T22:45:58.299520+0000","last_fullsized":"2024-09-10T22:45:58.299520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296446+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:45:58.296446+0000","last_peered":"2024-09-10T22:45:58.296446+0000","last_clean":"2024-09-10T22:45:58.296446+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:45:58.296446+0000","last_undegraded":"2024-09-10T22:45:58.296446+0000","last_fullsized":"2024-09-10T22:45:58.296446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298692+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:45:58.298692+0000","last_peered":"2024-09-10T22:45:58.298692+0000","last_clean":"2024-09-10T22:45:58.298692+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:45:58.298692+0000","last_undegraded":"2024-09-10T22:45:58.298692+0000","last_fullsized":"2024-09-10T22:45:58.298692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298140+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:45:58.298140+0000","last_peered":"2024-09-10T22:45:58.298140+0000","last_clean":"2024-09-10T22:45:58.298140+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:45:58.298140+0000","last_undegraded":"2024-09-10T22:45:58.298140+0000","last_fullsized":"2024-09-10T22:45:58.298140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680391+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:45:58.680391+0000","last_peered":"2024-09-10T22:45:58.680391+0000","last_clean":"2024-09-10T22:45:58.680391+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:45:58.680391+0000","last_undegraded":"2024-09-10T22:45:58.680391+0000","last_fullsized":"2024-09-10T22:45:58.680391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680156+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:45:58.680156+0000","last_peered":"2024-09-10T22:45:58.680156+0000","last_clean":"2024-09-10T22:45:58.680156+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:45:58.680156+0000","last_undegraded":"2024-09-10T22:45:58.680156+0000","last_fullsized":"2024-09-10T22:45:58.680156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301757+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:45:58.301757+0000","last_peered":"2024-09-10T22:45:58.301757+0000","last_clean":"2024-09-10T22:45:58.301757+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:45:58.301757+0000","last_undegraded":"2024-09-10T22:45:58.301757+0000","last_fullsized":"2024-09-10T22:45:58.301757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.305030+0000","last_change":"2024-09-10T22:45:58.304922+0000","last_active":"2024-09-10T22:45:58.305030+0000","last_peered":"2024-09-10T22:45:58.305030+0000","last_clean":"2024-09-10T22:45:58.305030+0000","last_became_active":"2024-09-10T22:45:58.304541+0000","last_became_peered":"2024-09-10T22:45:58.304541+0000","last_unstale":"2024-09-10T22:45:58.305030+0000","last_undegraded":"2024-09-10T22:45:58.305030+0000","last_fullsized":"2024-09-10T22:45:58.305030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:09.043873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680739+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:45:58.680739+0000","last_peered":"2024-09-10T22:45:58.680739+0000","last_clean":"2024-09-10T22:45:58.680739+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:45:58.680739+0000","last_undegraded":"2024-09-10T22:45:58.680739+0000","last_fullsized":"2024-09-10T22:45:58.680739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296574+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:45:58.296574+0000","last_peered":"2024-09-10T22:45:58.296574+0000","last_clean":"2024-09-10T22:45:58.296574+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:45:58.296574+0000","last_undegraded":"2024-09-10T22:45:58.296574+0000","last_fullsized":"2024-09-10T22:45:58.296574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298285+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:45:58.298285+0000","last_peered":"2024-09-10T22:45:58.298285+0000","last_clean":"2024-09-10T22:45:58.298285+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:45:58.298285+0000","last_undegraded":"2024-09-10T22:45:58.298285+0000","last_fullsized":"2024-09-10T22:45:58.298285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299518+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:45:58.299518+0000","last_peered":"2024-09-10T22:45:58.299518+0000","last_clean":"2024-09-10T22:45:58.299518+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:45:58.299518+0000","last_undegraded":"2024-09-10T22:45:58.299518+0000","last_fullsized":"2024-09-10T22:45:58.299518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299311+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:45:58.299311+0000","last_peered":"2024-09-10T22:45:58.299311+0000","last_clean":"2024-09-10T22:45:58.299311+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:45:58.299311+0000","last_undegraded":"2024-09-10T22:45:58.299311+0000","last_fullsized":"2024-09-10T22:45:58.299311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301179+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:45:58.301179+0000","last_peered":"2024-09-10T22:45:58.301179+0000","last_clean":"2024-09-10T22:45:58.301179+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:45:58.301179+0000","last_undegraded":"2024-09-10T22:45:58.301179+0000","last_fullsized":"2024-09-10T22:45:58.301179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598540+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:45:58.598540+0000","last_peered":"2024-09-10T22:45:58.598540+0000","last_clean":"2024-09-10T22:45:58.598540+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:45:58.598540+0000","last_undegraded":"2024-09-10T22:45:58.598540+0000","last_fullsized":"2024-09-10T22:45:58.598540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297599+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:45:58.297599+0000","last_peered":"2024-09-10T22:45:58.297599+0000","last_clean":"2024-09-10T22:45:58.297599+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:45:58.297599+0000","last_undegraded":"2024-09-10T22:45:58.297599+0000","last_fullsized":"2024-09-10T22:45:58.297599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303833+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:45:58.303833+0000","last_peered":"2024-09-10T22:45:58.303833+0000","last_clean":"2024-09-10T22:45:58.303833+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:45:58.303833+0000","last_undegraded":"2024-09-10T22:45:58.303833+0000","last_fullsized":"2024-09-10T22:45:58.303833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598358+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:45:58.598358+0000","last_peered":"2024-09-10T22:45:58.598358+0000","last_clean":"2024-09-10T22:45:58.598358+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:45:58.598358+0000","last_undegraded":"2024-09-10T22:45:58.598358+0000","last_fullsized":"2024-09-10T22:45:58.598358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298333+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:45:58.298333+0000","last_peered":"2024-09-10T22:45:58.298333+0000","last_clean":"2024-09-10T22:45:58.298333+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:45:58.298333+0000","last_undegraded":"2024-09-10T22:45:58.298333+0000","last_fullsized":"2024-09-10T22:45:58.298333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301369+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:45:58.301369+0000","last_peered":"2024-09-10T22:45:58.301369+0000","last_clean":"2024-09-10T22:45:58.301369+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:45:58.301369+0000","last_undegraded":"2024-09-10T22:45:58.301369+0000","last_fullsized":"2024-09-10T22:45:58.301369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680216+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:45:58.680216+0000","last_peered":"2024-09-10T22:45:58.680216+0000","last_clean":"2024-09-10T22:45:58.680216+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:45:58.680216+0000","last_undegraded":"2024-09-10T22:45:58.680216+0000","last_fullsized":"2024-09-10T22:45:58.680216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672102+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:45:58.672102+0000","last_peered":"2024-09-10T22:45:58.672102+0000","last_clean":"2024-09-10T22:45:58.672102+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:45:58.672102+0000","last_undegraded":"2024-09-10T22:45:58.672102+0000","last_fullsized":"2024-09-10T22:45:58.672102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680490+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:45:58.680490+0000","last_peered":"2024-09-10T22:45:58.680490+0000","last_clean":"2024-09-10T22:45:58.680490+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:45:58.680490+0000","last_undegraded":"2024-09-10T22:45:58.680490+0000","last_fullsized":"2024-09-10T22:45:58.680490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303849+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:45:58.303849+0000","last_peered":"2024-09-10T22:45:58.303849+0000","last_clean":"2024-09-10T22:45:58.303849+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:45:58.303849+0000","last_undegraded":"2024-09-10T22:45:58.303849+0000","last_fullsized":"2024-09-10T22:45:58.303849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598214+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:45:58.598214+0000","last_peered":"2024-09-10T22:45:58.598214+0000","last_clean":"2024-09-10T22:45:58.598214+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:45:58.598214+0000","last_undegraded":"2024-09-10T22:45:58.598214+0000","last_fullsized":"2024-09-10T22:45:58.598214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301320+0000","last_change":"2024-09-10T22:45:58.301222+0000","last_active":"2024-09-10T22:45:58.301320+0000","last_peered":"2024-09-10T22:45:58.301320+0000","last_clean":"2024-09-10T22:45:58.301320+0000","last_became_active":"2024-09-10T22:45:58.300835+0000","last_became_peered":"2024-09-10T22:45:58.300835+0000","last_unstale":"2024-09-10T22:45:58.301320+0000","last_undegraded":"2024-09-10T22:45:58.301320+0000","last_fullsized":"2024-09-10T22:45:58.301320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.772354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299680+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:45:58.299680+0000","last_peered":"2024-09-10T22:45:58.299680+0000","last_clean":"2024-09-10T22:45:58.299680+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:45:58.299680+0000","last_undegraded":"2024-09-10T22:45:58.299680+0000","last_fullsized":"2024-09-10T22:45:58.299680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303384+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:45:58.303384+0000","last_peered":"2024-09-10T22:45:58.303384+0000","last_clean":"2024-09-10T22:45:58.303384+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:45:58.303384+0000","last_undegraded":"2024-09-10T22:45:58.303384+0000","last_fullsized":"2024-09-10T22:45:58.303384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303157+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:45:58.303157+0000","last_peered":"2024-09-10T22:45:58.303157+0000","last_clean":"2024-09-10T22:45:58.303157+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:45:58.303157+0000","last_undegraded":"2024-09-10T22:45:58.303157+0000","last_fullsized":"2024-09-10T22:45:58.303157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598494+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:45:58.598494+0000","last_peered":"2024-09-10T22:45:58.598494+0000","last_clean":"2024-09-10T22:45:58.598494+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:45:58.598494+0000","last_undegraded":"2024-09-10T22:45:58.598494+0000","last_fullsized":"2024-09-10T22:45:58.598494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298625+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:45:58.298625+0000","last_peered":"2024-09-10T22:45:58.298625+0000","last_clean":"2024-09-10T22:45:58.298625+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:45:58.298625+0000","last_undegraded":"2024-09-10T22:45:58.298625+0000","last_fullsized":"2024-09-10T22:45:58.298625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.126955+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:46:21.126955+0000","last_peered":"2024-09-10T22:46:21.126955+0000","last_clean":"2024-09-10T22:46:21.126955+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:46:21.126955+0000","last_undegraded":"2024-09-10T22:46:21.126955+0000","last_fullsized":"2024-09-10T22:46:21.126955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672197+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:45:58.672197+0000","last_peered":"2024-09-10T22:45:58.672197+0000","last_clean":"2024-09-10T22:45:58.672197+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:45:58.672197+0000","last_undegraded":"2024-09-10T22:45:58.672197+0000","last_fullsized":"2024-09-10T22:45:58.672197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.306152+0000","last_change":"2024-09-10T22:45:58.306070+0000","last_active":"2024-09-10T22:45:58.306152+0000","last_peered":"2024-09-10T22:45:58.306152+0000","last_clean":"2024-09-10T22:45:58.306152+0000","last_became_active":"2024-09-10T22:45:58.300281+0000","last_became_peered":"2024-09-10T22:45:58.300281+0000","last_unstale":"2024-09-10T22:45:58.306152+0000","last_undegraded":"2024-09-10T22:45:58.306152+0000","last_fullsized":"2024-09-10T22:45:58.306152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:01.832411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297848+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:58.297848+0000","last_peered":"2024-09-10T22:45:58.297848+0000","last_clean":"2024-09-10T22:45:58.297848+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:58.297848+0000","last_undegraded":"2024-09-10T22:45:58.297848+0000","last_fullsized":"2024-09-10T22:45:58.297848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680216+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:45:58.680216+0000","last_peered":"2024-09-10T22:45:58.680216+0000","last_clean":"2024-09-10T22:45:58.680216+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:45:58.680216+0000","last_undegraded":"2024-09-10T22:45:58.680216+0000","last_fullsized":"2024-09-10T22:45:58.680216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297330+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:45:58.297330+0000","last_peered":"2024-09-10T22:45:58.297330+0000","last_clean":"2024-09-10T22:45:58.297330+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:45:58.297330+0000","last_undegraded":"2024-09-10T22:45:58.297330+0000","last_fullsized":"2024-09-10T22:45:58.297330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.128945+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:46:23.128945+0000","last_peered":"2024-09-10T22:46:23.128945+0000","last_clean":"2024-09-10T22:46:23.128945+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:46:23.128945+0000","last_undegraded":"2024-09-10T22:46:23.128945+0000","last_fullsized":"2024-09-10T22:46:23.128945+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301637+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:45:58.301637+0000","last_peered":"2024-09-10T22:45:58.301637+0000","last_clean":"2024-09-10T22:45:58.301637+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:45:58.301637+0000","last_undegraded":"2024-09-10T22:45:58.301637+0000","last_fullsized":"2024-09-10T22:45:58.301637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680757+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:58.680757+0000","last_peered":"2024-09-10T22:45:58.680757+0000","last_clean":"2024-09-10T22:45:58.680757+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:58.680757+0000","last_undegraded":"2024-09-10T22:45:58.680757+0000","last_fullsized":"2024-09-10T22:45:58.680757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299932+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:45:58.299932+0000","last_peered":"2024-09-10T22:45:58.299932+0000","last_clean":"2024-09-10T22:45:58.299932+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:45:58.299932+0000","last_undegraded":"2024-09-10T22:45:58.299932+0000","last_fullsized":"2024-09-10T22:45:58.299932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672340+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:58.672340+0000","last_peered":"2024-09-10T22:45:58.672340+0000","last_clean":"2024-09-10T22:45:58.672340+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:58.672340+0000","last_undegraded":"2024-09-10T22:45:58.672340+0000","last_fullsized":"2024-09-10T22:45:58.672340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301345+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:45:58.301345+0000","last_peered":"2024-09-10T22:45:58.301345+0000","last_clean":"2024-09-10T22:45:58.301345+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:45:58.301345+0000","last_undegraded":"2024-09-10T22:45:58.301345+0000","last_fullsized":"2024-09-10T22:45:58.301345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'30","reported_seq":168,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.131424+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:46:23.131424+0000","last_peered":"2024-09-10T22:46:23.131424+0000","last_clean":"2024-09-10T22:46:23.131424+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:46:23.131424+0000","last_undegraded":"2024-09-10T22:46:23.131424+0000","last_fullsized":"2024-09-10T22:46:23.131424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298966+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:45:58.298966+0000","last_peered":"2024-09-10T22:45:58.298966+0000","last_clean":"2024-09-10T22:45:58.298966+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:45:58.298966+0000","last_undegraded":"2024-09-10T22:45:58.298966+0000","last_fullsized":"2024-09-10T22:45:58.298966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303539+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:58.303539+0000","last_peered":"2024-09-10T22:45:58.303539+0000","last_clean":"2024-09-10T22:45:58.303539+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:58.303539+0000","last_undegraded":"2024-09-10T22:45:58.303539+0000","last_fullsized":"2024-09-10T22:45:58.303539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298508+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:45:58.298508+0000","last_peered":"2024-09-10T22:45:58.298508+0000","last_clean":"2024-09-10T22:45:58.298508+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:45:58.298508+0000","last_undegraded":"2024-09-10T22:45:58.298508+0000","last_fullsized":"2024-09-10T22:45:58.298508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680545+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:45:58.680545+0000","last_peered":"2024-09-10T22:45:58.680545+0000","last_clean":"2024-09-10T22:45:58.680545+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:45:58.680545+0000","last_undegraded":"2024-09-10T22:45:58.680545+0000","last_fullsized":"2024-09-10T22:45:58.680545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:22.144222+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:46:22.144222+0000","last_peered":"2024-09-10T22:46:22.144222+0000","last_clean":"2024-09-10T22:46:22.144222+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:46:22.144222+0000","last_undegraded":"2024-09-10T22:46:22.144222+0000","last_fullsized":"2024-09-10T22:46:22.144222+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680351+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:45:58.680351+0000","last_peered":"2024-09-10T22:45:58.680351+0000","last_clean":"2024-09-10T22:45:58.680351+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:45:58.680351+0000","last_undegraded":"2024-09-10T22:45:58.680351+0000","last_fullsized":"2024-09-10T22:45:58.680351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298597+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:58.298597+0000","last_peered":"2024-09-10T22:45:58.298597+0000","last_clean":"2024-09-10T22:45:58.298597+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:58.298597+0000","last_undegraded":"2024-09-10T22:45:58.298597+0000","last_fullsized":"2024-09-10T22:45:58.298597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672094+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:45:58.672094+0000","last_peered":"2024-09-10T22:45:58.672094+0000","last_clean":"2024-09-10T22:45:58.672094+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:45:58.672094+0000","last_undegraded":"2024-09-10T22:45:58.672094+0000","last_fullsized":"2024-09-10T22:45:58.672094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299235+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:58.299235+0000","last_peered":"2024-09-10T22:45:58.299235+0000","last_clean":"2024-09-10T22:45:58.299235+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:58.299235+0000","last_undegraded":"2024-09-10T22:45:58.299235+0000","last_fullsized":"2024-09-10T22:45:58.299235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598312+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:45:58.598312+0000","last_peered":"2024-09-10T22:45:58.598312+0000","last_clean":"2024-09-10T22:45:58.598312+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:45:58.598312+0000","last_undegraded":"2024-09-10T22:45:58.598312+0000","last_fullsized":"2024-09-10T22:45:58.598312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298492+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:58.298492+0000","last_peered":"2024-09-10T22:45:58.298492+0000","last_clean":"2024-09-10T22:45:58.298492+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:58.298492+0000","last_undegraded":"2024-09-10T22:45:58.298492+0000","last_fullsized":"2024-09-10T22:45:58.298492+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680736+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:45:58.680736+0000","last_peered":"2024-09-10T22:45:58.680736+0000","last_clean":"2024-09-10T22:45:58.680736+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:45:58.680736+0000","last_undegraded":"2024-09-10T22:45:58.680736+0000","last_fullsized":"2024-09-10T22:45:58.680736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672395+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:58.672395+0000","last_peered":"2024-09-10T22:45:58.672395+0000","last_clean":"2024-09-10T22:45:58.672395+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:58.672395+0000","last_undegraded":"2024-09-10T22:45:58.672395+0000","last_fullsized":"2024-09-10T22:45:58.672395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680546+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:45:58.680546+0000","last_peered":"2024-09-10T22:45:58.680546+0000","last_clean":"2024-09-10T22:45:58.680546+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:45:58.680546+0000","last_undegraded":"2024-09-10T22:45:58.680546+0000","last_fullsized":"2024-09-10T22:45:58.680546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.126737+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:46:21.126737+0000","last_peered":"2024-09-10T22:46:21.126737+0000","last_clean":"2024-09-10T22:46:21.126737+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:46:21.126737+0000","last_undegraded":"2024-09-10T22:46:21.126737+0000","last_fullsized":"2024-09-10T22:46:21.126737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680780+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:45:58.680780+0000","last_peered":"2024-09-10T22:45:58.680780+0000","last_clean":"2024-09-10T22:45:58.680780+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:45:58.680780+0000","last_undegraded":"2024-09-10T22:45:58.680780+0000","last_fullsized":"2024-09-10T22:45:58.680780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680557+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:58.680557+0000","last_peered":"2024-09-10T22:45:58.680557+0000","last_clean":"2024-09-10T22:45:58.680557+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:58.680557+0000","last_undegraded":"2024-09-10T22:45:58.680557+0000","last_fullsized":"2024-09-10T22:45:58.680557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296623+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:58.296623+0000","last_peered":"2024-09-10T22:45:58.296623+0000","last_clean":"2024-09-10T22:45:58.296623+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:58.296623+0000","last_undegraded":"2024-09-10T22:45:58.296623+0000","last_fullsized":"2024-09-10T22:45:58.296623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680642+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:45:58.680642+0000","last_peered":"2024-09-10T22:45:58.680642+0000","last_clean":"2024-09-10T22:45:58.680642+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:45:58.680642+0000","last_undegraded":"2024-09-10T22:45:58.680642+0000","last_fullsized":"2024-09-10T22:45:58.680642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680509+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:58.680509+0000","last_peered":"2024-09-10T22:45:58.680509+0000","last_clean":"2024-09-10T22:45:58.680509+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:58.680509+0000","last_undegraded":"2024-09-10T22:45:58.680509+0000","last_fullsized":"2024-09-10T22:45:58.680509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297753+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:45:58.297753+0000","last_peered":"2024-09-10T22:45:58.297753+0000","last_clean":"2024-09-10T22:45:58.297753+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:45:58.297753+0000","last_undegraded":"2024-09-10T22:45:58.297753+0000","last_fullsized":"2024-09-10T22:45:58.297753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.145028+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:46:23.145028+0000","last_peered":"2024-09-10T22:46:23.145028+0000","last_clean":"2024-09-10T22:46:23.145028+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:46:23.145028+0000","last_undegraded":"2024-09-10T22:46:23.145028+0000","last_fullsized":"2024-09-10T22:46:23.145028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.304559+0000","last_change":"2024-09-10T22:45:58.304503+0000","last_active":"2024-09-10T22:45:58.304559+0000","last_peered":"2024-09-10T22:45:58.304559+0000","last_clean":"2024-09-10T22:45:58.304559+0000","last_became_active":"2024-09-10T22:45:58.304243+0000","last_became_peered":"2024-09-10T22:45:58.304243+0000","last_unstale":"2024-09-10T22:45:58.304559+0000","last_undegraded":"2024-09-10T22:45:58.304559+0000","last_fullsized":"2024-09-10T22:45:58.304559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:33.190092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.128897+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:46:23.128897+0000","last_peered":"2024-09-10T22:46:23.128897+0000","last_clean":"2024-09-10T22:46:23.128897+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:46:23.128897+0000","last_undegraded":"2024-09-10T22:46:23.128897+0000","last_fullsized":"2024-09-10T22:46:23.128897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298494+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:45:58.298494+0000","last_peered":"2024-09-10T22:45:58.298494+0000","last_clean":"2024-09-10T22:45:58.298494+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:45:58.298494+0000","last_undegraded":"2024-09-10T22:45:58.298494+0000","last_fullsized":"2024-09-10T22:45:58.298494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.599710+0000","last_change":"2024-09-10T22:45:58.599612+0000","last_active":"2024-09-10T22:45:58.599710+0000","last_peered":"2024-09-10T22:45:58.599710+0000","last_clean":"2024-09-10T22:45:58.599710+0000","last_became_active":"2024-09-10T22:45:58.599226+0000","last_became_peered":"2024-09-10T22:45:58.599226+0000","last_unstale":"2024-09-10T22:45:58.599710+0000","last_undegraded":"2024-09-10T22:45:58.599710+0000","last_fullsized":"2024-09-10T22:45:58.599710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.112571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298177+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:45:58.298177+0000","last_peered":"2024-09-10T22:45:58.298177+0000","last_clean":"2024-09-10T22:45:58.298177+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:45:58.298177+0000","last_undegraded":"2024-09-10T22:45:58.298177+0000","last_fullsized":"2024-09-10T22:45:58.298177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299088+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:58.299088+0000","last_peered":"2024-09-10T22:45:58.299088+0000","last_clean":"2024-09-10T22:45:58.299088+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:58.299088+0000","last_undegraded":"2024-09-10T22:45:58.299088+0000","last_fullsized":"2024-09-10T22:45:58.299088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298311+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:45:58.298311+0000","last_peered":"2024-09-10T22:45:58.298311+0000","last_clean":"2024-09-10T22:45:58.298311+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:45:58.298311+0000","last_undegraded":"2024-09-10T22:45:58.298311+0000","last_fullsized":"2024-09-10T22:45:58.298311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297631+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:58.297631+0000","last_peered":"2024-09-10T22:45:58.297631+0000","last_clean":"2024-09-10T22:45:58.297631+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:58.297631+0000","last_undegraded":"2024-09-10T22:45:58.297631+0000","last_fullsized":"2024-09-10T22:45:58.297631+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680493+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:45:58.680493+0000","last_peered":"2024-09-10T22:45:58.680493+0000","last_clean":"2024-09-10T22:45:58.680493+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:45:58.680493+0000","last_undegraded":"2024-09-10T22:45:58.680493+0000","last_fullsized":"2024-09-10T22:45:58.680493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296692+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:58.296692+0000","last_peered":"2024-09-10T22:45:58.296692+0000","last_clean":"2024-09-10T22:45:58.296692+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:58.296692+0000","last_undegraded":"2024-09-10T22:45:58.296692+0000","last_fullsized":"2024-09-10T22:45:58.296692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299664+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:45:58.299664+0000","last_peered":"2024-09-10T22:45:58.299664+0000","last_clean":"2024-09-10T22:45:58.299664+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:45:58.299664+0000","last_undegraded":"2024-09-10T22:45:58.299664+0000","last_fullsized":"2024-09-10T22:45:58.299664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.602110+0000","last_change":"2024-09-10T22:45:58.602021+0000","last_active":"2024-09-10T22:45:58.602110+0000","last_peered":"2024-09-10T22:45:58.602110+0000","last_clean":"2024-09-10T22:45:58.602110+0000","last_became_active":"2024-09-10T22:45:58.599358+0000","last_became_peered":"2024-09-10T22:45:58.599358+0000","last_unstale":"2024-09-10T22:45:58.602110+0000","last_undegraded":"2024-09-10T22:45:58.602110+0000","last_fullsized":"2024-09-10T22:45:58.602110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:40.510120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298611+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:45:58.298611+0000","last_peered":"2024-09-10T22:45:58.298611+0000","last_clean":"2024-09-10T22:45:58.298611+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:45:58.298611+0000","last_undegraded":"2024-09-10T22:45:58.298611+0000","last_fullsized":"2024-09-10T22:45:58.298611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680327+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:58.680327+0000","last_peered":"2024-09-10T22:45:58.680327+0000","last_clean":"2024-09-10T22:45:58.680327+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:58.680327+0000","last_undegraded":"2024-09-10T22:45:58.680327+0000","last_fullsized":"2024-09-10T22:45:58.680327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598523+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:45:58.598523+0000","last_peered":"2024-09-10T22:45:58.598523+0000","last_clean":"2024-09-10T22:45:58.598523+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:45:58.598523+0000","last_undegraded":"2024-09-10T22:45:58.598523+0000","last_fullsized":"2024-09-10T22:45:58.598523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303229+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:58.303229+0000","last_peered":"2024-09-10T22:45:58.303229+0000","last_clean":"2024-09-10T22:45:58.303229+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:58.303229+0000","last_undegraded":"2024-09-10T22:45:58.303229+0000","last_fullsized":"2024-09-10T22:45:58.303229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680378+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:45:58.680378+0000","last_peered":"2024-09-10T22:45:58.680378+0000","last_clean":"2024-09-10T22:45:58.680378+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:45:58.680378+0000","last_undegraded":"2024-09-10T22:45:58.680378+0000","last_fullsized":"2024-09-10T22:45:58.680378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298787+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:58.298787+0000","last_peered":"2024-09-10T22:45:58.298787+0000","last_clean":"2024-09-10T22:45:58.298787+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:58.298787+0000","last_undegraded":"2024-09-10T22:45:58.298787+0000","last_fullsized":"2024-09-10T22:45:58.298787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680519+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:45:58.680519+0000","last_peered":"2024-09-10T22:45:58.680519+0000","last_clean":"2024-09-10T22:45:58.680519+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:45:58.680519+0000","last_undegraded":"2024-09-10T22:45:58.680519+0000","last_fullsized":"2024-09-10T22:45:58.680519+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672240+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:45:58.672240+0000","last_peered":"2024-09-10T22:45:58.672240+0000","last_clean":"2024-09-10T22:45:58.672240+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:45:58.672240+0000","last_undegraded":"2024-09-10T22:45:58.672240+0000","last_fullsized":"2024-09-10T22:45:58.672240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298925+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:58.298925+0000","last_peered":"2024-09-10T22:45:58.298925+0000","last_clean":"2024-09-10T22:45:58.298925+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:58.298925+0000","last_undegraded":"2024-09-10T22:45:58.298925+0000","last_fullsized":"2024-09-10T22:45:58.298925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301300+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:45:58.301300+0000","last_peered":"2024-09-10T22:45:58.301300+0000","last_clean":"2024-09-10T22:45:58.301300+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:45:58.301300+0000","last_undegraded":"2024-09-10T22:45:58.301300+0000","last_fullsized":"2024-09-10T22:45:58.301300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299045+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:58.299045+0000","last_peered":"2024-09-10T22:45:58.299045+0000","last_clean":"2024-09-10T22:45:58.299045+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:58.299045+0000","last_undegraded":"2024-09-10T22:45:58.299045+0000","last_fullsized":"2024-09-10T22:45:58.299045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299905+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:45:58.299905+0000","last_peered":"2024-09-10T22:45:58.299905+0000","last_clean":"2024-09-10T22:45:58.299905+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:45:58.299905+0000","last_undegraded":"2024-09-10T22:45:58.299905+0000","last_fullsized":"2024-09-10T22:45:58.299905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680612+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:58.680612+0000","last_peered":"2024-09-10T22:45:58.680612+0000","last_clean":"2024-09-10T22:45:58.680612+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:58.680612+0000","last_undegraded":"2024-09-10T22:45:58.680612+0000","last_fullsized":"2024-09-10T22:45:58.680612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680503+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:45:58.680503+0000","last_peered":"2024-09-10T22:45:58.680503+0000","last_clean":"2024-09-10T22:45:58.680503+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:45:58.680503+0000","last_undegraded":"2024-09-10T22:45:58.680503+0000","last_fullsized":"2024-09-10T22:45:58.680503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":89,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.125543+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:46:21.125543+0000","last_peered":"2024-09-10T22:46:21.125543+0000","last_clean":"2024-09-10T22:46:21.125543+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:46:21.125543+0000","last_undegraded":"2024-09-10T22:46:21.125543+0000","last_fullsized":"2024-09-10T22:46:21.125543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66184,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":357,"num_write":240,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":405978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1056769,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":864256,"data_stored":405908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495663,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":998413,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":405625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855416,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":998091,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":405486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":864256,"data_stored":405450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":466520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":466300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":406964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:46:26.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:46:26.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:26 smithi121 ceph-mon[35119]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:26.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:26 smithi117 ceph-mon[35081]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:26.918 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:27.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 -- 172.21.15.96:0/3878809000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac1076a0 msgr2=0x7f42ac107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:27.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/3878809000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac1076a0 0x7f42ac107ae0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4298009900 tx=0x7f429802a410 comp rx=0 tx=0).stop 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 -- 172.21.15.96:0/3878809000 shutdown_connections 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/3878809000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac108020 0x7f42ac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/3878809000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac1076a0 0x7f42ac107ae0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/3878809000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac101c90 0x7f42ac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 -- 172.21.15.96:0/3878809000 >> 172.21.15.96:0/3878809000 conn(0x7f42ac0fd700 msgr2=0x7f42ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 -- 172.21.15.96:0/3878809000 shutdown_connections 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.428+0000 7f42b2ce8640 1 -- 172.21.15.96:0/3878809000 wait complete. 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 Processor -- start 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 -- start start 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac101c90 0x7f42ac1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac1076a0 0x7f42ac199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:27.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac111c70 con 0x7f42ac108020 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac111af0 con 0x7f42ac101c90 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b2ce8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac111df0 con 0x7f42ac1076a0 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56722/0 (socket says 172.21.15.96:56722) 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 -- 172.21.15.96:0/1677521834 learned_addr learned my addr 172.21.15.96:0/1677521834 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b0a84640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac101c90 0x7f42ac1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 -- 172.21.15.96:0/1677521834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac1076a0 msgr2=0x7f42ac199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac1076a0 0x7f42ac199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 -- 172.21.15.96:0/1677521834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac101c90 msgr2=0x7f42ac1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac101c90 0x7f42ac1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.429+0000 7f42b1285640 1 -- 172.21.15.96:0/1677521834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19f380 con 0x7f42ac108020 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42b1285640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42a400eb10 tx=0x7f42a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:27.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4004b20 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac19f670 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a4004cc0 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4010670 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.430+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a5d30 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.431+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac102070 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.432+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f42a4005000 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.433+0000 7f42a9ffb640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 0x7f429c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.433+0000 7f42b0a84640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 0x7f429c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.433+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f42a40e6430 con 0x7f42ac108020 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.433+0000 7f42b0a84640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 0x7f429c0b7830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005d00 tx=0x7f42a000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:27.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.436+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f42a4056220 con 0x7f42ac108020 2024-09-10T22:46:27.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.611+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42ac1002d0 con 0x7f429c0b5370 2024-09-10T22:46:27.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.615+0000 7f42a9ffb640 1 -- 172.21.15.96:0/1677521834 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258881 (secure 0 0 0) 0x7f42ac1002d0 con 0x7f429c0b5370 2024-09-10T22:46:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:27.620 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:46:27.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 msgr2=0x7f429c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:27.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 0x7f429c0b7830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005d00 tx=0x7f42a000b410 comp rx=0 tx=0).stop 2024-09-10T22:46:27.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 msgr2=0x7f42ac19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:27.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42a400eb10 tx=0x7f42a400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 shutdown_connections 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f429c0b5370 0x7f429c0b7830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42ac108020 0x7f42ac19ebb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42ac1076a0 0x7f42ac199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 --2- 172.21.15.96:0/1677521834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42ac101c90 0x7f42ac1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 >> 172.21.15.96:0/1677521834 conn(0x7f42ac0fd700 msgr2=0x7f42ac1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 shutdown_connections 2024-09-10T22:46:27.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:27.618+0000 7f42b2ce8640 1 -- 172.21.15.96:0/1677521834 wait complete. 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:27 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:28.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:27 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:46:28.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:27 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:46:28.338 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-10T22:46:27.314000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656786,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":421,"num_write":365,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355592,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586232,"statfs":{"total":1151940427776,"available":1151576301568,"internally_reserved":0,"allocated":10665984,"data_stored":6827790,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003193"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598491+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:45:58.598491+0000","last_peered":"2024-09-10T22:45:58.598491+0000","last_clean":"2024-09-10T22:45:58.598491+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:45:58.598491+0000","last_undegraded":"2024-09-10T22:45:58.598491+0000","last_fullsized":"2024-09-10T22:45:58.598491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299573+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:45:58.299573+0000","last_peered":"2024-09-10T22:45:58.299573+0000","last_clean":"2024-09-10T22:45:58.299573+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:45:58.299573+0000","last_undegraded":"2024-09-10T22:45:58.299573+0000","last_fullsized":"2024-09-10T22:45:58.299573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299520+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:45:58.299520+0000","last_peered":"2024-09-10T22:45:58.299520+0000","last_clean":"2024-09-10T22:45:58.299520+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:45:58.299520+0000","last_undegraded":"2024-09-10T22:45:58.299520+0000","last_fullsized":"2024-09-10T22:45:58.299520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296446+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:45:58.296446+0000","last_peered":"2024-09-10T22:45:58.296446+0000","last_clean":"2024-09-10T22:45:58.296446+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:45:58.296446+0000","last_undegraded":"2024-09-10T22:45:58.296446+0000","last_fullsized":"2024-09-10T22:45:58.296446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298692+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:45:58.298692+0000","last_peered":"2024-09-10T22:45:58.298692+0000","last_clean":"2024-09-10T22:45:58.298692+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:45:58.298692+0000","last_undegraded":"2024-09-10T22:45:58.298692+0000","last_fullsized":"2024-09-10T22:45:58.298692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298140+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:45:58.298140+0000","last_peered":"2024-09-10T22:45:58.298140+0000","last_clean":"2024-09-10T22:45:58.298140+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:45:58.298140+0000","last_undegraded":"2024-09-10T22:45:58.298140+0000","last_fullsized":"2024-09-10T22:45:58.298140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680391+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:45:58.680391+0000","last_peered":"2024-09-10T22:45:58.680391+0000","last_clean":"2024-09-10T22:45:58.680391+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:45:58.680391+0000","last_undegraded":"2024-09-10T22:45:58.680391+0000","last_fullsized":"2024-09-10T22:45:58.680391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680156+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:45:58.680156+0000","last_peered":"2024-09-10T22:45:58.680156+0000","last_clean":"2024-09-10T22:45:58.680156+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:45:58.680156+0000","last_undegraded":"2024-09-10T22:45:58.680156+0000","last_fullsized":"2024-09-10T22:45:58.680156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301757+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:45:58.301757+0000","last_peered":"2024-09-10T22:45:58.301757+0000","last_clean":"2024-09-10T22:45:58.301757+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:45:58.301757+0000","last_undegraded":"2024-09-10T22:45:58.301757+0000","last_fullsized":"2024-09-10T22:45:58.301757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.305030+0000","last_change":"2024-09-10T22:45:58.304922+0000","last_active":"2024-09-10T22:45:58.305030+0000","last_peered":"2024-09-10T22:45:58.305030+0000","last_clean":"2024-09-10T22:45:58.305030+0000","last_became_active":"2024-09-10T22:45:58.304541+0000","last_became_peered":"2024-09-10T22:45:58.304541+0000","last_unstale":"2024-09-10T22:45:58.305030+0000","last_undegraded":"2024-09-10T22:45:58.305030+0000","last_fullsized":"2024-09-10T22:45:58.305030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:09.043873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680739+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:45:58.680739+0000","last_peered":"2024-09-10T22:45:58.680739+0000","last_clean":"2024-09-10T22:45:58.680739+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:45:58.680739+0000","last_undegraded":"2024-09-10T22:45:58.680739+0000","last_fullsized":"2024-09-10T22:45:58.680739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296574+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:45:58.296574+0000","last_peered":"2024-09-10T22:45:58.296574+0000","last_clean":"2024-09-10T22:45:58.296574+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:45:58.296574+0000","last_undegraded":"2024-09-10T22:45:58.296574+0000","last_fullsized":"2024-09-10T22:45:58.296574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298285+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:45:58.298285+0000","last_peered":"2024-09-10T22:45:58.298285+0000","last_clean":"2024-09-10T22:45:58.298285+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:45:58.298285+0000","last_undegraded":"2024-09-10T22:45:58.298285+0000","last_fullsized":"2024-09-10T22:45:58.298285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299518+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:45:58.299518+0000","last_peered":"2024-09-10T22:45:58.299518+0000","last_clean":"2024-09-10T22:45:58.299518+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:45:58.299518+0000","last_undegraded":"2024-09-10T22:45:58.299518+0000","last_fullsized":"2024-09-10T22:45:58.299518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299311+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:45:58.299311+0000","last_peered":"2024-09-10T22:45:58.299311+0000","last_clean":"2024-09-10T22:45:58.299311+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:45:58.299311+0000","last_undegraded":"2024-09-10T22:45:58.299311+0000","last_fullsized":"2024-09-10T22:45:58.299311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301179+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:45:58.301179+0000","last_peered":"2024-09-10T22:45:58.301179+0000","last_clean":"2024-09-10T22:45:58.301179+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:45:58.301179+0000","last_undegraded":"2024-09-10T22:45:58.301179+0000","last_fullsized":"2024-09-10T22:45:58.301179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598540+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:45:58.598540+0000","last_peered":"2024-09-10T22:45:58.598540+0000","last_clean":"2024-09-10T22:45:58.598540+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:45:58.598540+0000","last_undegraded":"2024-09-10T22:45:58.598540+0000","last_fullsized":"2024-09-10T22:45:58.598540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297599+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:45:58.297599+0000","last_peered":"2024-09-10T22:45:58.297599+0000","last_clean":"2024-09-10T22:45:58.297599+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:45:58.297599+0000","last_undegraded":"2024-09-10T22:45:58.297599+0000","last_fullsized":"2024-09-10T22:45:58.297599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303833+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:45:58.303833+0000","last_peered":"2024-09-10T22:45:58.303833+0000","last_clean":"2024-09-10T22:45:58.303833+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:45:58.303833+0000","last_undegraded":"2024-09-10T22:45:58.303833+0000","last_fullsized":"2024-09-10T22:45:58.303833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598358+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:45:58.598358+0000","last_peered":"2024-09-10T22:45:58.598358+0000","last_clean":"2024-09-10T22:45:58.598358+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:45:58.598358+0000","last_undegraded":"2024-09-10T22:45:58.598358+0000","last_fullsized":"2024-09-10T22:45:58.598358+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298333+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:45:58.298333+0000","last_peered":"2024-09-10T22:45:58.298333+0000","last_clean":"2024-09-10T22:45:58.298333+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:45:58.298333+0000","last_undegraded":"2024-09-10T22:45:58.298333+0000","last_fullsized":"2024-09-10T22:45:58.298333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301369+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:45:58.301369+0000","last_peered":"2024-09-10T22:45:58.301369+0000","last_clean":"2024-09-10T22:45:58.301369+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:45:58.301369+0000","last_undegraded":"2024-09-10T22:45:58.301369+0000","last_fullsized":"2024-09-10T22:45:58.301369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680216+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:45:58.680216+0000","last_peered":"2024-09-10T22:45:58.680216+0000","last_clean":"2024-09-10T22:45:58.680216+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:45:58.680216+0000","last_undegraded":"2024-09-10T22:45:58.680216+0000","last_fullsized":"2024-09-10T22:45:58.680216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672102+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:45:58.672102+0000","last_peered":"2024-09-10T22:45:58.672102+0000","last_clean":"2024-09-10T22:45:58.672102+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:45:58.672102+0000","last_undegraded":"2024-09-10T22:45:58.672102+0000","last_fullsized":"2024-09-10T22:45:58.672102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680490+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:45:58.680490+0000","last_peered":"2024-09-10T22:45:58.680490+0000","last_clean":"2024-09-10T22:45:58.680490+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:45:58.680490+0000","last_undegraded":"2024-09-10T22:45:58.680490+0000","last_fullsized":"2024-09-10T22:45:58.680490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303849+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:45:58.303849+0000","last_peered":"2024-09-10T22:45:58.303849+0000","last_clean":"2024-09-10T22:45:58.303849+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:45:58.303849+0000","last_undegraded":"2024-09-10T22:45:58.303849+0000","last_fullsized":"2024-09-10T22:45:58.303849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598214+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:45:58.598214+0000","last_peered":"2024-09-10T22:45:58.598214+0000","last_clean":"2024-09-10T22:45:58.598214+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:45:58.598214+0000","last_undegraded":"2024-09-10T22:45:58.598214+0000","last_fullsized":"2024-09-10T22:45:58.598214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301320+0000","last_change":"2024-09-10T22:45:58.301222+0000","last_active":"2024-09-10T22:45:58.301320+0000","last_peered":"2024-09-10T22:45:58.301320+0000","last_clean":"2024-09-10T22:45:58.301320+0000","last_became_active":"2024-09-10T22:45:58.300835+0000","last_became_peered":"2024-09-10T22:45:58.300835+0000","last_unstale":"2024-09-10T22:45:58.301320+0000","last_undegraded":"2024-09-10T22:45:58.301320+0000","last_fullsized":"2024-09-10T22:45:58.301320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.772354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299680+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:45:58.299680+0000","last_peered":"2024-09-10T22:45:58.299680+0000","last_clean":"2024-09-10T22:45:58.299680+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:45:58.299680+0000","last_undegraded":"2024-09-10T22:45:58.299680+0000","last_fullsized":"2024-09-10T22:45:58.299680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303384+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:45:58.303384+0000","last_peered":"2024-09-10T22:45:58.303384+0000","last_clean":"2024-09-10T22:45:58.303384+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:45:58.303384+0000","last_undegraded":"2024-09-10T22:45:58.303384+0000","last_fullsized":"2024-09-10T22:45:58.303384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303157+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:45:58.303157+0000","last_peered":"2024-09-10T22:45:58.303157+0000","last_clean":"2024-09-10T22:45:58.303157+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:45:58.303157+0000","last_undegraded":"2024-09-10T22:45:58.303157+0000","last_fullsized":"2024-09-10T22:45:58.303157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598494+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:45:58.598494+0000","last_peered":"2024-09-10T22:45:58.598494+0000","last_clean":"2024-09-10T22:45:58.598494+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:45:58.598494+0000","last_undegraded":"2024-09-10T22:45:58.598494+0000","last_fullsized":"2024-09-10T22:45:58.598494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298625+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:45:58.298625+0000","last_peered":"2024-09-10T22:45:58.298625+0000","last_clean":"2024-09-10T22:45:58.298625+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:45:58.298625+0000","last_undegraded":"2024-09-10T22:45:58.298625+0000","last_fullsized":"2024-09-10T22:45:58.298625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.126955+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:46:21.126955+0000","last_peered":"2024-09-10T22:46:21.126955+0000","last_clean":"2024-09-10T22:46:21.126955+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:46:21.126955+0000","last_undegraded":"2024-09-10T22:46:21.126955+0000","last_fullsized":"2024-09-10T22:46:21.126955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672197+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:45:58.672197+0000","last_peered":"2024-09-10T22:45:58.672197+0000","last_clean":"2024-09-10T22:45:58.672197+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:45:58.672197+0000","last_undegraded":"2024-09-10T22:45:58.672197+0000","last_fullsized":"2024-09-10T22:45:58.672197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.306152+0000","last_change":"2024-09-10T22:45:58.306070+0000","last_active":"2024-09-10T22:45:58.306152+0000","last_peered":"2024-09-10T22:45:58.306152+0000","last_clean":"2024-09-10T22:45:58.306152+0000","last_became_active":"2024-09-10T22:45:58.300281+0000","last_became_peered":"2024-09-10T22:45:58.300281+0000","last_unstale":"2024-09-10T22:45:58.306152+0000","last_undegraded":"2024-09-10T22:45:58.306152+0000","last_fullsized":"2024-09-10T22:45:58.306152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:01.832411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297848+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:58.297848+0000","last_peered":"2024-09-10T22:45:58.297848+0000","last_clean":"2024-09-10T22:45:58.297848+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:58.297848+0000","last_undegraded":"2024-09-10T22:45:58.297848+0000","last_fullsized":"2024-09-10T22:45:58.297848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680216+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:45:58.680216+0000","last_peered":"2024-09-10T22:45:58.680216+0000","last_clean":"2024-09-10T22:45:58.680216+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:45:58.680216+0000","last_undegraded":"2024-09-10T22:45:58.680216+0000","last_fullsized":"2024-09-10T22:45:58.680216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297330+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:45:58.297330+0000","last_peered":"2024-09-10T22:45:58.297330+0000","last_clean":"2024-09-10T22:45:58.297330+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:45:58.297330+0000","last_undegraded":"2024-09-10T22:45:58.297330+0000","last_fullsized":"2024-09-10T22:45:58.297330+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.128945+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:46:23.128945+0000","last_peered":"2024-09-10T22:46:23.128945+0000","last_clean":"2024-09-10T22:46:23.128945+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:46:23.128945+0000","last_undegraded":"2024-09-10T22:46:23.128945+0000","last_fullsized":"2024-09-10T22:46:23.128945+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301637+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:45:58.301637+0000","last_peered":"2024-09-10T22:45:58.301637+0000","last_clean":"2024-09-10T22:45:58.301637+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:45:58.301637+0000","last_undegraded":"2024-09-10T22:45:58.301637+0000","last_fullsized":"2024-09-10T22:45:58.301637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680757+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:58.680757+0000","last_peered":"2024-09-10T22:45:58.680757+0000","last_clean":"2024-09-10T22:45:58.680757+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:58.680757+0000","last_undegraded":"2024-09-10T22:45:58.680757+0000","last_fullsized":"2024-09-10T22:45:58.680757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299932+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:45:58.299932+0000","last_peered":"2024-09-10T22:45:58.299932+0000","last_clean":"2024-09-10T22:45:58.299932+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:45:58.299932+0000","last_undegraded":"2024-09-10T22:45:58.299932+0000","last_fullsized":"2024-09-10T22:45:58.299932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672340+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:58.672340+0000","last_peered":"2024-09-10T22:45:58.672340+0000","last_clean":"2024-09-10T22:45:58.672340+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:58.672340+0000","last_undegraded":"2024-09-10T22:45:58.672340+0000","last_fullsized":"2024-09-10T22:45:58.672340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301345+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:45:58.301345+0000","last_peered":"2024-09-10T22:45:58.301345+0000","last_clean":"2024-09-10T22:45:58.301345+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:45:58.301345+0000","last_undegraded":"2024-09-10T22:45:58.301345+0000","last_fullsized":"2024-09-10T22:45:58.301345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'30","reported_seq":168,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.131424+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:46:23.131424+0000","last_peered":"2024-09-10T22:46:23.131424+0000","last_clean":"2024-09-10T22:46:23.131424+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:46:23.131424+0000","last_undegraded":"2024-09-10T22:46:23.131424+0000","last_fullsized":"2024-09-10T22:46:23.131424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298966+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:45:58.298966+0000","last_peered":"2024-09-10T22:45:58.298966+0000","last_clean":"2024-09-10T22:45:58.298966+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:45:58.298966+0000","last_undegraded":"2024-09-10T22:45:58.298966+0000","last_fullsized":"2024-09-10T22:45:58.298966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303539+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:58.303539+0000","last_peered":"2024-09-10T22:45:58.303539+0000","last_clean":"2024-09-10T22:45:58.303539+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:58.303539+0000","last_undegraded":"2024-09-10T22:45:58.303539+0000","last_fullsized":"2024-09-10T22:45:58.303539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298508+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:45:58.298508+0000","last_peered":"2024-09-10T22:45:58.298508+0000","last_clean":"2024-09-10T22:45:58.298508+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:45:58.298508+0000","last_undegraded":"2024-09-10T22:45:58.298508+0000","last_fullsized":"2024-09-10T22:45:58.298508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680545+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:45:58.680545+0000","last_peered":"2024-09-10T22:45:58.680545+0000","last_clean":"2024-09-10T22:45:58.680545+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:45:58.680545+0000","last_undegraded":"2024-09-10T22:45:58.680545+0000","last_fullsized":"2024-09-10T22:45:58.680545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:22.144222+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:46:22.144222+0000","last_peered":"2024-09-10T22:46:22.144222+0000","last_clean":"2024-09-10T22:46:22.144222+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:46:22.144222+0000","last_undegraded":"2024-09-10T22:46:22.144222+0000","last_fullsized":"2024-09-10T22:46:22.144222+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680351+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:45:58.680351+0000","last_peered":"2024-09-10T22:45:58.680351+0000","last_clean":"2024-09-10T22:45:58.680351+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:45:58.680351+0000","last_undegraded":"2024-09-10T22:45:58.680351+0000","last_fullsized":"2024-09-10T22:45:58.680351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298597+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:58.298597+0000","last_peered":"2024-09-10T22:45:58.298597+0000","last_clean":"2024-09-10T22:45:58.298597+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:58.298597+0000","last_undegraded":"2024-09-10T22:45:58.298597+0000","last_fullsized":"2024-09-10T22:45:58.298597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672094+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:45:58.672094+0000","last_peered":"2024-09-10T22:45:58.672094+0000","last_clean":"2024-09-10T22:45:58.672094+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:45:58.672094+0000","last_undegraded":"2024-09-10T22:45:58.672094+0000","last_fullsized":"2024-09-10T22:45:58.672094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299235+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:58.299235+0000","last_peered":"2024-09-10T22:45:58.299235+0000","last_clean":"2024-09-10T22:45:58.299235+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:58.299235+0000","last_undegraded":"2024-09-10T22:45:58.299235+0000","last_fullsized":"2024-09-10T22:45:58.299235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598312+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:45:58.598312+0000","last_peered":"2024-09-10T22:45:58.598312+0000","last_clean":"2024-09-10T22:45:58.598312+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:45:58.598312+0000","last_undegraded":"2024-09-10T22:45:58.598312+0000","last_fullsized":"2024-09-10T22:45:58.598312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298492+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:58.298492+0000","last_peered":"2024-09-10T22:45:58.298492+0000","last_clean":"2024-09-10T22:45:58.298492+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:58.298492+0000","last_undegraded":"2024-09-10T22:45:58.298492+0000","last_fullsized":"2024-09-10T22:45:58.298492+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680736+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:45:58.680736+0000","last_peered":"2024-09-10T22:45:58.680736+0000","last_clean":"2024-09-10T22:45:58.680736+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:45:58.680736+0000","last_undegraded":"2024-09-10T22:45:58.680736+0000","last_fullsized":"2024-09-10T22:45:58.680736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672395+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:58.672395+0000","last_peered":"2024-09-10T22:45:58.672395+0000","last_clean":"2024-09-10T22:45:58.672395+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:58.672395+0000","last_undegraded":"2024-09-10T22:45:58.672395+0000","last_fullsized":"2024-09-10T22:45:58.672395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680546+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:45:58.680546+0000","last_peered":"2024-09-10T22:45:58.680546+0000","last_clean":"2024-09-10T22:45:58.680546+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:45:58.680546+0000","last_undegraded":"2024-09-10T22:45:58.680546+0000","last_fullsized":"2024-09-10T22:45:58.680546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":82,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.126737+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:46:21.126737+0000","last_peered":"2024-09-10T22:46:21.126737+0000","last_clean":"2024-09-10T22:46:21.126737+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:46:21.126737+0000","last_undegraded":"2024-09-10T22:46:21.126737+0000","last_fullsized":"2024-09-10T22:46:21.126737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680780+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:45:58.680780+0000","last_peered":"2024-09-10T22:45:58.680780+0000","last_clean":"2024-09-10T22:45:58.680780+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:45:58.680780+0000","last_undegraded":"2024-09-10T22:45:58.680780+0000","last_fullsized":"2024-09-10T22:45:58.680780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680557+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:58.680557+0000","last_peered":"2024-09-10T22:45:58.680557+0000","last_clean":"2024-09-10T22:45:58.680557+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:58.680557+0000","last_undegraded":"2024-09-10T22:45:58.680557+0000","last_fullsized":"2024-09-10T22:45:58.680557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296623+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:58.296623+0000","last_peered":"2024-09-10T22:45:58.296623+0000","last_clean":"2024-09-10T22:45:58.296623+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:58.296623+0000","last_undegraded":"2024-09-10T22:45:58.296623+0000","last_fullsized":"2024-09-10T22:45:58.296623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680642+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:45:58.680642+0000","last_peered":"2024-09-10T22:45:58.680642+0000","last_clean":"2024-09-10T22:45:58.680642+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:45:58.680642+0000","last_undegraded":"2024-09-10T22:45:58.680642+0000","last_fullsized":"2024-09-10T22:45:58.680642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680509+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:58.680509+0000","last_peered":"2024-09-10T22:45:58.680509+0000","last_clean":"2024-09-10T22:45:58.680509+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:58.680509+0000","last_undegraded":"2024-09-10T22:45:58.680509+0000","last_fullsized":"2024-09-10T22:45:58.680509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297753+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:45:58.297753+0000","last_peered":"2024-09-10T22:45:58.297753+0000","last_clean":"2024-09-10T22:45:58.297753+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:45:58.297753+0000","last_undegraded":"2024-09-10T22:45:58.297753+0000","last_fullsized":"2024-09-10T22:45:58.297753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.145028+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:46:23.145028+0000","last_peered":"2024-09-10T22:46:23.145028+0000","last_clean":"2024-09-10T22:46:23.145028+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:46:23.145028+0000","last_undegraded":"2024-09-10T22:46:23.145028+0000","last_fullsized":"2024-09-10T22:46:23.145028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.304559+0000","last_change":"2024-09-10T22:45:58.304503+0000","last_active":"2024-09-10T22:45:58.304559+0000","last_peered":"2024-09-10T22:45:58.304559+0000","last_clean":"2024-09-10T22:45:58.304559+0000","last_became_active":"2024-09-10T22:45:58.304243+0000","last_became_peered":"2024-09-10T22:45:58.304243+0000","last_unstale":"2024-09-10T22:45:58.304559+0000","last_undegraded":"2024-09-10T22:45:58.304559+0000","last_fullsized":"2024-09-10T22:45:58.304559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:33.190092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"75'5","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:23.128897+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:46:23.128897+0000","last_peered":"2024-09-10T22:46:23.128897+0000","last_clean":"2024-09-10T22:46:23.128897+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:46:23.128897+0000","last_undegraded":"2024-09-10T22:46:23.128897+0000","last_fullsized":"2024-09-10T22:46:23.128897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298494+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:45:58.298494+0000","last_peered":"2024-09-10T22:45:58.298494+0000","last_clean":"2024-09-10T22:45:58.298494+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:45:58.298494+0000","last_undegraded":"2024-09-10T22:45:58.298494+0000","last_fullsized":"2024-09-10T22:45:58.298494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.599710+0000","last_change":"2024-09-10T22:45:58.599612+0000","last_active":"2024-09-10T22:45:58.599710+0000","last_peered":"2024-09-10T22:45:58.599710+0000","last_clean":"2024-09-10T22:45:58.599710+0000","last_became_active":"2024-09-10T22:45:58.599226+0000","last_became_peered":"2024-09-10T22:45:58.599226+0000","last_unstale":"2024-09-10T22:45:58.599710+0000","last_undegraded":"2024-09-10T22:45:58.599710+0000","last_fullsized":"2024-09-10T22:45:58.599710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.112571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298177+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:45:58.298177+0000","last_peered":"2024-09-10T22:45:58.298177+0000","last_clean":"2024-09-10T22:45:58.298177+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:45:58.298177+0000","last_undegraded":"2024-09-10T22:45:58.298177+0000","last_fullsized":"2024-09-10T22:45:58.298177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299088+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:58.299088+0000","last_peered":"2024-09-10T22:45:58.299088+0000","last_clean":"2024-09-10T22:45:58.299088+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:58.299088+0000","last_undegraded":"2024-09-10T22:45:58.299088+0000","last_fullsized":"2024-09-10T22:45:58.299088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298311+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:45:58.298311+0000","last_peered":"2024-09-10T22:45:58.298311+0000","last_clean":"2024-09-10T22:45:58.298311+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:45:58.298311+0000","last_undegraded":"2024-09-10T22:45:58.298311+0000","last_fullsized":"2024-09-10T22:45:58.298311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297631+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:58.297631+0000","last_peered":"2024-09-10T22:45:58.297631+0000","last_clean":"2024-09-10T22:45:58.297631+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:58.297631+0000","last_undegraded":"2024-09-10T22:45:58.297631+0000","last_fullsized":"2024-09-10T22:45:58.297631+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680493+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:45:58.680493+0000","last_peered":"2024-09-10T22:45:58.680493+0000","last_clean":"2024-09-10T22:45:58.680493+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:45:58.680493+0000","last_undegraded":"2024-09-10T22:45:58.680493+0000","last_fullsized":"2024-09-10T22:45:58.680493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296692+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:58.296692+0000","last_peered":"2024-09-10T22:45:58.296692+0000","last_clean":"2024-09-10T22:45:58.296692+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:58.296692+0000","last_undegraded":"2024-09-10T22:45:58.296692+0000","last_fullsized":"2024-09-10T22:45:58.296692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299664+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:45:58.299664+0000","last_peered":"2024-09-10T22:45:58.299664+0000","last_clean":"2024-09-10T22:45:58.299664+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:45:58.299664+0000","last_undegraded":"2024-09-10T22:45:58.299664+0000","last_fullsized":"2024-09-10T22:45:58.299664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.602110+0000","last_change":"2024-09-10T22:45:58.602021+0000","last_active":"2024-09-10T22:45:58.602110+0000","last_peered":"2024-09-10T22:45:58.602110+0000","last_clean":"2024-09-10T22:45:58.602110+0000","last_became_active":"2024-09-10T22:45:58.599358+0000","last_became_peered":"2024-09-10T22:45:58.599358+0000","last_unstale":"2024-09-10T22:45:58.602110+0000","last_undegraded":"2024-09-10T22:45:58.602110+0000","last_fullsized":"2024-09-10T22:45:58.602110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:40.510120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298611+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:45:58.298611+0000","last_peered":"2024-09-10T22:45:58.298611+0000","last_clean":"2024-09-10T22:45:58.298611+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:45:58.298611+0000","last_undegraded":"2024-09-10T22:45:58.298611+0000","last_fullsized":"2024-09-10T22:45:58.298611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680327+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:58.680327+0000","last_peered":"2024-09-10T22:45:58.680327+0000","last_clean":"2024-09-10T22:45:58.680327+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:58.680327+0000","last_undegraded":"2024-09-10T22:45:58.680327+0000","last_fullsized":"2024-09-10T22:45:58.680327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.598523+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:45:58.598523+0000","last_peered":"2024-09-10T22:45:58.598523+0000","last_clean":"2024-09-10T22:45:58.598523+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:45:58.598523+0000","last_undegraded":"2024-09-10T22:45:58.598523+0000","last_fullsized":"2024-09-10T22:45:58.598523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303229+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:58.303229+0000","last_peered":"2024-09-10T22:45:58.303229+0000","last_clean":"2024-09-10T22:45:58.303229+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:58.303229+0000","last_undegraded":"2024-09-10T22:45:58.303229+0000","last_fullsized":"2024-09-10T22:45:58.303229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680378+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:45:58.680378+0000","last_peered":"2024-09-10T22:45:58.680378+0000","last_clean":"2024-09-10T22:45:58.680378+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:45:58.680378+0000","last_undegraded":"2024-09-10T22:45:58.680378+0000","last_fullsized":"2024-09-10T22:45:58.680378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298787+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:58.298787+0000","last_peered":"2024-09-10T22:45:58.298787+0000","last_clean":"2024-09-10T22:45:58.298787+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:58.298787+0000","last_undegraded":"2024-09-10T22:45:58.298787+0000","last_fullsized":"2024-09-10T22:45:58.298787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680519+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:45:58.680519+0000","last_peered":"2024-09-10T22:45:58.680519+0000","last_clean":"2024-09-10T22:45:58.680519+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:45:58.680519+0000","last_undegraded":"2024-09-10T22:45:58.680519+0000","last_fullsized":"2024-09-10T22:45:58.680519+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672240+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:45:58.672240+0000","last_peered":"2024-09-10T22:45:58.672240+0000","last_clean":"2024-09-10T22:45:58.672240+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:45:58.672240+0000","last_undegraded":"2024-09-10T22:45:58.672240+0000","last_fullsized":"2024-09-10T22:45:58.672240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298925+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:58.298925+0000","last_peered":"2024-09-10T22:45:58.298925+0000","last_clean":"2024-09-10T22:45:58.298925+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:58.298925+0000","last_undegraded":"2024-09-10T22:45:58.298925+0000","last_fullsized":"2024-09-10T22:45:58.298925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.301300+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:45:58.301300+0000","last_peered":"2024-09-10T22:45:58.301300+0000","last_clean":"2024-09-10T22:45:58.301300+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:45:58.301300+0000","last_undegraded":"2024-09-10T22:45:58.301300+0000","last_fullsized":"2024-09-10T22:45:58.301300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299045+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:58.299045+0000","last_peered":"2024-09-10T22:45:58.299045+0000","last_clean":"2024-09-10T22:45:58.299045+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:58.299045+0000","last_undegraded":"2024-09-10T22:45:58.299045+0000","last_fullsized":"2024-09-10T22:45:58.299045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299905+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:45:58.299905+0000","last_peered":"2024-09-10T22:45:58.299905+0000","last_clean":"2024-09-10T22:45:58.299905+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:45:58.299905+0000","last_undegraded":"2024-09-10T22:45:58.299905+0000","last_fullsized":"2024-09-10T22:45:58.299905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680612+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:58.680612+0000","last_peered":"2024-09-10T22:45:58.680612+0000","last_clean":"2024-09-10T22:45:58.680612+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:58.680612+0000","last_undegraded":"2024-09-10T22:45:58.680612+0000","last_fullsized":"2024-09-10T22:45:58.680612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680503+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:45:58.680503+0000","last_peered":"2024-09-10T22:45:58.680503+0000","last_clean":"2024-09-10T22:45:58.680503+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:45:58.680503+0000","last_undegraded":"2024-09-10T22:45:58.680503+0000","last_fullsized":"2024-09-10T22:45:58.680503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":89,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:21.125543+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:46:21.125543+0000","last_peered":"2024-09-10T22:46:21.125543+0000","last_clean":"2024-09-10T22:46:21.125543+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:46:21.125543+0000","last_undegraded":"2024-09-10T22:46:21.125543+0000","last_fullsized":"2024-09-10T22:46:21.125543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66184,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":357,"num_write":240,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":405978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":1056769,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":864256,"data_stored":405908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495663,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":998413,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":405625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855416,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":998091,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":405486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":864256,"data_stored":405450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":466520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":466300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":406964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:46:28.342 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:46:28.342 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:46:28.342 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:46:28.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T22:46:28.591 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:28.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:28 smithi096 ceph-mon[30869]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:28 smithi121 ceph-mon[35119]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:28 smithi117 ceph-mon[35081]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 -- 172.21.15.96:0/2870695908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f0103b30 msgr2=0x7ff3f0103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/2870695908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f0103b30 0x7ff3f0103f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009900 tx=0x7ff3e002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 -- 172.21.15.96:0/2870695908 shutdown_connections 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/2870695908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f010d0e0 0x7ff3f010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/2870695908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 0x7ff3f010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/2870695908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f0103b30 0x7ff3f0103f10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 -- 172.21.15.96:0/2870695908 >> 172.21.15.96:0/2870695908 conn(0x7ff3f00fd700 msgr2=0x7ff3f00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 -- 172.21.15.96:0/2870695908 shutdown_connections 2024-09-10T22:46:29.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.047+0000 7ff3f5e45640 1 -- 172.21.15.96:0/2870695908 wait complete. 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 Processor -- start 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 -- start start 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f0103b30 0x7ff3f019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 0x7ff3f019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0102920 con 0x7ff3f010d0e0 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01027a0 con 0x7ff3f0103b30 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3f5e45640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0102aa0 con 0x7ff3f0104530 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56744/0 (socket says 172.21.15.96:56744) 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 -- 172.21.15.96:0/1953842799 learned_addr learned my addr 172.21.15.96:0/1953842799 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3eeffd640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 0x7ff3f019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:29.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3ef7fe640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f0103b30 0x7ff3f019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 -- 172.21.15.96:0/1953842799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 msgr2=0x7ff3f019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 0x7ff3f019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 -- 172.21.15.96:0/1953842799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f0103b30 msgr2=0x7ff3f019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f0103b30 0x7ff3f019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.048+0000 7ff3effff640 1 -- 172.21.15.96:0/1953842799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01a15b0 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3effff640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800cce0 tx=0x7ff3e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800f450 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01a18a0 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e800faa0 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8018a80 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.049+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01a7f60 con 0x7ff3f010d0e0 2024-09-10T22:46:29.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.050+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f0103f10 con 0x7ff3f010d0e0 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.051+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff3e8020030 con 0x7ff3f010d0e0 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.052+0000 7ff3ecff9640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 0x7ff3dc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.052+0000 7ff3ef7fe640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 0x7ff3dc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.052+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff3e80e62b0 con 0x7ff3f010d0e0 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.052+0000 7ff3ef7fe640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 0x7ff3dc0b7830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0031440 tx=0x7ff3e00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:29.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.055+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff3e8014030 con 0x7ff3f010d0e0 2024-09-10T22:46:29.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.269+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff3f0106ba0 con 0x7ff3f010d0e0 2024-09-10T22:46:29.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.269+0000 7ff3ecff9640 1 -- 172.21.15.96:0/1953842799 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff3e80ad110 con 0x7ff3f010d0e0 2024-09-10T22:46:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:29.272 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:46:29.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 msgr2=0x7ff3dc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:29.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 0x7ff3dc0b7830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0031440 tx=0x7ff3e00095f0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 msgr2=0x7ff3f01a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800cce0 tx=0x7ff3e800a620 comp rx=0 tx=0).stop 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 shutdown_connections 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3dc0b5370 0x7ff3dc0b7830 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f010d0e0 0x7ff3f01a0de0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3f0104530 0x7ff3f019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 --2- 172.21.15.96:0/1953842799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3f0103b30 0x7ff3f019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 >> 172.21.15.96:0/1953842799 conn(0x7ff3f00fd700 msgr2=0x7ff3f010eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 shutdown_connections 2024-09-10T22:46:29.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:29.272+0000 7ff3f5e45640 1 -- 172.21.15.96:0/1953842799 wait complete. 2024-09-10T22:46:29.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:29 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1953842799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:46:29.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:29 smithi096 ceph-mon[30869]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:29.984 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:46:29.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:46:30.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:29 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1953842799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:46:30.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:29 smithi121 ceph-mon[35119]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:30.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:29 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1953842799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:46:30.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:29 smithi117 ceph-mon[35081]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:30.232 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:30.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 -- 172.21.15.96:0/309150442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 msgr2=0x7f3e50102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:30.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 --2- 172.21.15.96:0/309150442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e50102090 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009900 tx=0x7f3e4002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:30.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 -- 172.21.15.96:0/309150442 shutdown_connections 2024-09-10T22:46:30.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 --2- 172.21.15.96:0/309150442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 0x7f3e50107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 --2- 172.21.15.96:0/309150442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 0x7f3e5010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 --2- 172.21.15.96:0/309150442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e50102090 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 -- 172.21.15.96:0/309150442 >> 172.21.15.96:0/309150442 conn(0x7f3e500fd700 msgr2=0x7f3e500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 -- 172.21.15.96:0/309150442 shutdown_connections 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.722+0000 7f3e54fba640 1 -- 172.21.15.96:0/309150442 wait complete. 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 Processor -- start 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 -- start start 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 0x7f3e5019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 0x7f3e5019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50111d00 con 0x7f3e50101cb0 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50111b80 con 0x7f3e501076c0 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e54fba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50111e80 con 0x7f3e50108040 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e4e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:30.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e4e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56764/0 (socket says 172.21.15.96:56764) 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e4e59c640 1 -- 172.21.15.96:0/1670943521 learned_addr learned my addr 172.21.15.96:0/1670943521 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e4ed9d640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 0x7f3e5019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.723+0000 7f3e4dd9b640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 0x7f3e5019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 -- 172.21.15.96:0/1670943521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 msgr2=0x7f3e5019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 0x7f3e5019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 -- 172.21.15.96:0/1670943521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 msgr2=0x7f3e5019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 0x7f3e5019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 -- 172.21.15.96:0/1670943521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e501a54d0 con 0x7f3e50101cb0 2024-09-10T22:46:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e4e59c640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005940 tx=0x7f3e400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4003f030 con 0x7f3e50101cb0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e501a5760 con 0x7f3e50101cb0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e40003cb0 con 0x7f3e50101cb0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40035070 con 0x7f3e50101cb0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.724+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501a5d10 con 0x7f3e50101cb0 2024-09-10T22:46:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.725+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e50102090 con 0x7f3e50101cb0 2024-09-10T22:46:30.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.727+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f3e40003800 con 0x7f3e50101cb0 2024-09-10T22:46:30.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.727+0000 7f3e3f7fe640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 0x7f3e380b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:30.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.727+0000 7f3e4dd9b640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 0x7f3e380b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:30.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.728+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3e40106910 con 0x7f3e50101cb0 2024-09-10T22:46:30.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.728+0000 7f3e4dd9b640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 0x7f3e380b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e5006c090 tx=0x7f3e44009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:30.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.730+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3e400cd770 con 0x7f3e50101cb0 2024-09-10T22:46:30.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.944+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e5006b8a0 con 0x7f3e50101cb0 2024-09-10T22:46:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.945+0000 7f3e3f7fe640 1 -- 172.21.15.96:0/1670943521 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f3e400cd070 con 0x7f3e50101cb0 2024-09-10T22:46:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:30.947 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:46:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 msgr2=0x7f3e380b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 0x7f3e380b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e5006c090 tx=0x7f3e44009110 comp rx=0 tx=0).stop 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 msgr2=0x7f3e5019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005940 tx=0x7f3e400042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 shutdown_connections 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e501076c0 0x7f3e5019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3e380b53f0 0x7f3e380b78b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e50108040 0x7f3e5019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 --2- 172.21.15.96:0/1670943521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e50101cb0 0x7f3e5019a0f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:30.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.947+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 >> 172.21.15.96:0/1670943521 conn(0x7f3e500fd700 msgr2=0x7f3e500ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:30.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.948+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 shutdown_connections 2024-09-10T22:46:30.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:30.948+0000 7f3e54fba640 1 -- 172.21.15.96:0/1670943521 wait complete. 2024-09-10T22:46:31.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:30 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1670943521' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:31.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:30 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1670943521' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:31.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:30 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1670943521' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:31.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:46:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:32.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:32 smithi121 ceph-mon[35119]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:32.274 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:32 smithi096 ceph-mon[30869]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:32.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:32 smithi117 ceph-mon[35081]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:32.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 -- 172.21.15.96:0/829450423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 msgr2=0x7f5ea810f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:32.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 --2- 172.21.15.96:0/829450423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea810f580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009ca0 tx=0x7f5ea002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 -- 172.21.15.96:0/829450423 shutdown_connections 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 --2- 172.21.15.96:0/829450423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 0x7f5ea810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 --2- 172.21.15.96:0/829450423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea810f580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 --2- 172.21.15.96:0/829450423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 0x7f5ea81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 -- 172.21.15.96:0/829450423 >> 172.21.15.96:0/829450423 conn(0x7f5ea80fd700 msgr2=0x7f5ea80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 -- 172.21.15.96:0/829450423 shutdown_connections 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.489+0000 7f5eadd13640 1 -- 172.21.15.96:0/829450423 wait complete. 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 Processor -- start 2024-09-10T22:46:32.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 -- start start 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 0x7f5ea8199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 0x7f5ea81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112e10 con 0x7f5ea810d120 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112c90 con 0x7f5ea81045e0 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5eadd13640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112f90 con 0x7f5ea8103cc0 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5ea7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5ea7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56782/0 (socket says 172.21.15.96:56782) 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.490+0000 7f5ea7fff640 1 -- 172.21.15.96:0/725194474 learned_addr learned my addr 172.21.15.96:0/725194474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea77fe640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 0x7f5ea8199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea6ffd640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 0x7f5ea81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 -- 172.21.15.96:0/725194474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 msgr2=0x7f5ea8199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 0x7f5ea8199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 -- 172.21.15.96:0/725194474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 msgr2=0x7f5ea81998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 0x7f5ea81998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 -- 172.21.15.96:0/725194474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea819f410 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea7fff640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005cc0 tx=0x7f5ea002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003bf0 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea819f6a0 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ea0003d90 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002fbc0 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.491+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a5dc0 con 0x7f5ea810d120 2024-09-10T22:46:32.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.492+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea81040a0 con 0x7f5ea810d120 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.493+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f5ea0004610 con 0x7f5ea810d120 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.494+0000 7f5ea4ff9640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 0x7f5e900b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.494+0000 7f5ea77fe640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 0x7f5e900b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.494+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5ea01072d0 con 0x7f5ea810d120 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.495+0000 7f5ea77fe640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 0x7f5e900b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005fd0 tx=0x7f5e9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:32.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.497+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ea00ce130 con 0x7f5ea810d120 2024-09-10T22:46:32.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.711+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ea81a6050 con 0x7f5ea810d120 2024-09-10T22:46:32.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.712+0000 7f5ea4ff9640 1 -- 172.21.15.96:0/725194474 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f5ea00cda30 con 0x7f5ea810d120 2024-09-10T22:46:32.714 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:32.714 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:46:32.714 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:32.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 msgr2=0x7f5e900b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:32.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 0x7f5e900b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005fd0 tx=0x7f5e9800b390 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 msgr2=0x7f5ea819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005cc0 tx=0x7f5ea002f380 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 shutdown_connections 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea81045e0 0x7f5ea81998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5e900b5370 0x7f5e900b7830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ea810d120 0x7f5ea819ec40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 --2- 172.21.15.96:0/725194474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ea8103cc0 0x7f5ea8199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 >> 172.21.15.96:0/725194474 conn(0x7f5ea80fd700 msgr2=0x7f5ea80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 shutdown_connections 2024-09-10T22:46:32.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:32.715+0000 7f5eadd13640 1 -- 172.21.15.96:0/725194474 wait complete. 2024-09-10T22:46:33.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:33 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/725194474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:33.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:33 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/725194474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:33.373 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:33 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/725194474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:33.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:46:33.648 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:34.027 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:34 smithi096 ceph-mon[30869]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:34.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2649369088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078101cb0 msgr2=0x7fa078102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:34.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2649369088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078101cb0 0x7fa078102090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa074009900 tx=0x7fa07402a410 comp rx=0 tx=0).stop 2024-09-10T22:46:34.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2649369088 shutdown_connections 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2649369088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078108040 0x7fa07810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2649369088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0781076c0 0x7fa078107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2649369088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078101cb0 0x7fa078102090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2649369088 >> 172.21.15.96:0/2649369088 conn(0x7fa0780fd700 msgr2=0x7fa0780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2649369088 shutdown_connections 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.105+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2649369088 wait complete. 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 Processor -- start 2024-09-10T22:46:34.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 -- start start 2024-09-10T22:46:34.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 0x7fa078103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0781076c0 0x7fa078103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa078111d00 con 0x7fa078108040 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa078111b80 con 0x7fa078101cb0 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ecb9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa078111e80 con 0x7fa0781076c0 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56796/0 (socket says 172.21.15.96:56796) 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 -- 172.21.15.96:0/2233689653 learned_addr learned my addr 172.21.15.96:0/2233689653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:34.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ca55640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 0x7fa078103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 -- 172.21.15.96:0/2233689653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0781076c0 msgr2=0x7fa078103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0781076c0 0x7fa078103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 -- 172.21.15.96:0/2233689653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 msgr2=0x7fa078103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 0x7fa078103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 -- 172.21.15.96:0/2233689653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781a75c0 con 0x7fa078108040 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07ca55640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 0x7fa078103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.106+0000 7fa07d256640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa07000cce0 tx=0x7fa07000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.107+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000f450 con 0x7fa078108040 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.107+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07000faa0 con 0x7fa078108040 2024-09-10T22:46:34.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.107+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781a78b0 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.107+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070018b70 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.107+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781a7da0 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.108+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078102090 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.109+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fa07000f5f0 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.110+0000 7fa06dffb640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 0x7fa0600b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.111+0000 7fa07ca55640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 0x7fa0600b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.111+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa0700e6450 con 0x7fa078108040 2024-09-10T22:46:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.111+0000 7fa07ca55640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 0x7fa0600b7830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa074031440 tx=0x7fa0740095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.115+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa070014030 con 0x7fa078108040 2024-09-10T22:46:34.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:34 smithi117 ceph-mon[35081]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:34.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.294+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0781a8210 con 0x7fa078108040 2024-09-10T22:46:34.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.295+0000 7fa06dffb640 1 -- 172.21.15.96:0/2233689653 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7fa0700ad2b0 con 0x7fa078108040 2024-09-10T22:46:34.297 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:34.297 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:46:34.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 msgr2=0x7fa0600b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 0x7fa0600b7830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa074031440 tx=0x7fa0740095f0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 msgr2=0x7fa078103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa07000cce0 tx=0x7fa07000a620 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 shutdown_connections 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa0600b5370 0x7fa0600b7830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa078108040 0x7fa078103ff0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0781076c0 0x7fa078103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 --2- 172.21.15.96:0/2233689653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa078101cb0 0x7fa078103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 >> 172.21.15.96:0/2233689653 conn(0x7fa0780fd700 msgr2=0x7fa0780ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 shutdown_connections 2024-09-10T22:46:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:34.298+0000 7fa07ecb9640 1 -- 172.21.15.96:0/2233689653 wait complete. 2024-09-10T22:46:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:34 smithi121 ceph-mon[35119]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:35.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:46:35.060 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:35 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2233689653' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:46:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:35 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2233689653' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:46:35.285 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:35.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:35 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2233689653' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:46:35.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 -- 172.21.15.96:0/1632880053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075540 msgr2=0x7f78f0075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:35.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 --2- 172.21.15.96:0/1632880053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075540 0x7f78f0075920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009900 tx=0x7f78ec02a410 comp rx=0 tx=0).stop 2024-09-10T22:46:35.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 -- 172.21.15.96:0/1632880053 shutdown_connections 2024-09-10T22:46:35.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 --2- 172.21.15.96:0/1632880053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f0075f40 0x7f78f010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:35.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 --2- 172.21.15.96:0/1632880053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f010f3e0 0x7f78f01117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 --2- 172.21.15.96:0/1632880053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075540 0x7f78f0075920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 -- 172.21.15.96:0/1632880053 >> 172.21.15.96:0/1632880053 conn(0x7f78f00fe830 msgr2=0x7f78f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 -- 172.21.15.96:0/1632880053 shutdown_connections 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.779+0000 7f78f6a84640 1 -- 172.21.15.96:0/1632880053 wait complete. 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 Processor -- start 2024-09-10T22:46:35.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 -- start start 2024-09-10T22:46:35.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0075540 0x7f78f019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:35.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f010f3e0 0x7f78f01a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0114f90 con 0x7f78f0075f40 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0114e10 con 0x7f78f010f3e0 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f6a84640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0115110 con 0x7f78f0075540 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56822/0 (socket says 172.21.15.96:56822) 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 -- 172.21.15.96:0/2092579841 learned_addr learned my addr 172.21.15.96:0/2092579841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f4820640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0075540 0x7f78f019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78f5021640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f010f3e0 0x7f78f01a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 -- 172.21.15.96:0/2092579841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0075540 msgr2=0x7f78f019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0075540 0x7f78f019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:35.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 -- 172.21.15.96:0/2092579841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f010f3e0 msgr2=0x7f78f01a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.780+0000 7f78e7fff640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f010f3e0 0x7f78f01a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78e7fff640 1 -- 172.21.15.96:0/2092579841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f01a1510 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78e7fff640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e9f0 tx=0x7f78e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e000cd80 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e00108e0 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f01a1800 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0018980 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.781+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01a7ec0 con 0x7f78f0075f40 2024-09-10T22:46:35.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.782+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f0075920 con 0x7f78f0075f40 2024-09-10T22:46:35.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.783+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f78e0010430 con 0x7f78f0075f40 2024-09-10T22:46:35.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.784+0000 7f78e5ffb640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 0x7f78d80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:35.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.784+0000 7f78f4820640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 0x7f78d80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:35.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.784+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f78e00e65c0 con 0x7f78f0075f40 2024-09-10T22:46:35.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.784+0000 7f78f4820640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 0x7f78d80b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f78ec031440 tx=0x7f78ec0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:35.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.787+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78e00ad420 con 0x7f78f0075f40 2024-09-10T22:46:36.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.998+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78f01a81b0 con 0x7f78f0075f40 2024-09-10T22:46:36.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:35.999+0000 7f78e5ffb640 1 -- 172.21.15.96:0/2092579841 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f78e00acd20 con 0x7f78f0075f40 2024-09-10T22:46:36.002 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:36.002 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:46:36.003 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:36.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.001+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 msgr2=0x7f78d80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:36.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 0x7f78d80b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f78ec031440 tx=0x7f78ec0095f0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 msgr2=0x7f78f019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e9f0 tx=0x7f78e000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 shutdown_connections 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78f010f3e0 0x7f78f01a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f78d80b5370 0x7f78d80b7830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78f0075f40 0x7f78f019b9a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 --2- 172.21.15.96:0/2092579841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78f0075540 0x7f78f019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:36.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 >> 172.21.15.96:0/2092579841 conn(0x7f78f00fe830 msgr2=0x7f78f01115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:36.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 shutdown_connections 2024-09-10T22:46:36.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:36.002+0000 7f78f6a84640 1 -- 172.21.15.96:0/2092579841 wait complete. 2024-09-10T22:46:36.281 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:36 smithi117 ceph-mon[35081]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:36.281 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:36 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2092579841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:36.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:36 smithi096 ceph-mon[30869]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:36.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:36 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2092579841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:36.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:36 smithi121 ceph-mon[35119]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:36.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:36 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2092579841' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:36.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:45:19.551704+0000', 'modified': '2024-09-10T22:45:31.047462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24550, 'mds_2': 24542, 'mds_3': 24529, 'mds_4': 24527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14649': {'gid': 14649, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6837/3503446146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3503446146}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3503446146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14700': {'gid': 14700, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/2726589142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2726589142}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2726589142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14706': {'gid': 14706, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/2618224226', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 2618224226}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 2618224226}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6833/2699941517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 2699941517}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 2699941517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/1599221610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1599221610}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 1599221610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.117:6835/2943423694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2943423694}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2943423694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.121:6837/1420997271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1420997271}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1420997271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/214150535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 214150535}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 214150535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.121:6839/693851001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 693851001}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 693851001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/4053412961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 4053412961}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 4053412961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24550, 24542, 24529, 24566, 24527]} 2024-09-10T22:46:36.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T22:46:36.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-10T22:46:37.002 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:37.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- 172.21.15.96:0/2997565237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 msgr2=0x7fbe7010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/2997565237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe7010f950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009ca0 tx=0x7fbe6802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- 172.21.15.96:0/2997565237 shutdown_connections 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/2997565237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe70108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/2997565237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe7010f950 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/2997565237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe701031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- 172.21.15.96:0/2997565237 >> 172.21.15.96:0/2997565237 conn(0x7fbe700fe830 msgr2=0x7fbe70100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- 172.21.15.96:0/2997565237 shutdown_connections 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- 172.21.15.96:0/2997565237 wait complete. 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 Processor -- start 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.464+0000 7fbe76fd7640 1 -- start start 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe700754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe700759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112e10 con 0x7fbe70109150 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112c90 con 0x7fbe701087d0 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe76fd7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112f90 con 0x7fbe70102dc0 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43644/0 (socket says 172.21.15.96:43644) 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 -- 172.21.15.96:0/794432168 learned_addr learned my addr 172.21.15.96:0/794432168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe74d73640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe700754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe700759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 -- 172.21.15.96:0/794432168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 msgr2=0x7fbe700754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe700754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 -- 172.21.15.96:0/794432168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 msgr2=0x7fbe700759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe700759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 -- 172.21.15.96:0/794432168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe701a7650 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe74d73640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe700754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe700759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe75574640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005cc0 tx=0x7fbe6802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.465+0000 7fbe6dffb640 1 -- 172.21.15.96:0/794432168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68003cf0 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.466+0000 7fbe6dffb640 1 -- 172.21.15.96:0/794432168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe680044f0 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.466+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe701a78e0 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.466+0000 7fbe6dffb640 1 -- 172.21.15.96:0/794432168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6802fd10 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.466+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a7ea0 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe6dffb640 1 -- 172.21.15.96:0/794432168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fbe68004040 con 0x7fbe70109150 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 Processor -- start 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 -- start start 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e78862110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e7886e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e787d9c10 con 0x558e7886e5e0 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e787c9040 con 0x558e78861d30 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.468+0000 7fbe793ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e787cce50 con 0x558e78862650 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43652/0 (socket says 172.21.15.96:43652) 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 -- 172.21.15.96:0/2865323877 learned_addr learned my addr 172.21.15.96:0/2865323877 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe74d73640 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e7886e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e78862110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6dffb640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 0x7fbe600b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 -- 172.21.15.96:0/2865323877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 msgr2=0x558e7886e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e7886e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 -- 172.21.15.96:0/2865323877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 msgr2=0x558e78862110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e78862110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 -- 172.21.15.96:0/2865323877 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e78527540 con 0x558e7886e5e0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe75574640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 0x7fbe600b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00ab70 tx=0x7fbe5c00a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472f518cca69c7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe5affd640 1 -- 172.21.15.96:0/2865323877 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00fbf0 con 0x558e7886e5e0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe6dffb640 1 -- 172.21.15.96:0/794432168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbe68107370 con 0x7fbe70109150 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe5affd640 1 -- 172.21.15.96:0/2865323877 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c019070 con 0x558e7886e5e0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.469+0000 7fbe5affd640 1 -- 172.21.15.96:0/2865323877 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c013370 con 0x558e7886e5e0 2024-09-10T22:46:37.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- 172.21.15.96:0/2865323877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 msgr2=0x558e788709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe75574640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 0x7fbe600b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe680a8470 tx=0x7fbe6808f3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00ab70 tx=0x7fbe5c00a520 comp rx=0 tx=0).stop 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- 172.21.15.96:0/2865323877 shutdown_connections 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788709d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e7886e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 --2- 172.21.15.96:0/2865323877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e78862110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- 172.21.15.96:0/2865323877 >> 172.21.15.96:0/2865323877 conn(0x558e7885fb60 msgr2=0x558e78860d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- 172.21.15.96:0/2865323877 shutdown_connections 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- 172.21.15.96:0/2865323877 wait complete. 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.470+0000 7fbe793ac740 1 -- start start 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 Processor -- start 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e7886e5e0 0x558e7886c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e7862a5f0 con 0x558e78861d30 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e78764010 con 0x558e78862650 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e78871290 con 0x558e7886e5e0 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe793ac740 10 client.0 _ll_register_callbacks cb 0x558e787c3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43654/0 (socket says 172.21.15.96:43654) 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 -- 172.21.15.96:0/395183389 learned_addr learned my addr 172.21.15.96:0/395183389 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e7886e5e0 0x558e7886c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe74d73640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 -- 172.21.15.96:0/395183389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e7886e5e0 msgr2=0x558e7886c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e7886e5e0 0x558e7886c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 -- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 msgr2=0x558e7886bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.471+0000 7fbe6ffff640 1 -- 172.21.15.96:0/395183389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e78791a00 con 0x558e78861d30 2024-09-10T22:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c014cb0 tx=0x7fbe5c00f0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77583a0b25366c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c014620 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbe5c0147c0 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c019070 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c031050 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 10 client.15066 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e7887ce80 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c019070 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbe5c020420 con 0x558e78861d30 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 10 client.15066 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 20 client.15066 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e7887f110 0x558e78880580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 4 client.15066 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.472+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e787dbb60 con 0x558e7887f110 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.473+0000 7fbe74d73640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e7887f110 0x558e78880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.473+0000 7fbe4ffff640 10 client.15066 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.473+0000 7fbe74d73640 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e7887f110 0x558e78880580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b09d9683b1b6a39 server_cookie=f11bc779e91ee0b8 in_seq=0 out_seq=0 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe4ffff640 1 -- 172.21.15.96:0/395183389 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e787dbb60 con 0x558e7887f110 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe4ffff640 10 client.15066 handle_command_reply: tid=0 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 client.15066 shutdown 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 msgr2=0x558e7886b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c014cb0 tx=0x7fbe5c00f0b0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 shutdown_connections 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e7887f110 0x558e78880580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e7886e5e0 0x558e7886c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 --2- 172.21.15.96:0/395183389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e78861d30 0x558e7886b5a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 >> 172.21.15.96:0/395183389 conn(0x558e7885fb60 msgr2=0x558e78865700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe4ffff640 0 client.15066 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 shutdown_connections 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 1 -- 172.21.15.96:0/395183389 wait complete. 2024-09-10T22:46:37.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.474+0000 7fbe793ac740 20 client.15066 trim_cache size 0 max 16384 2024-09-10T22:46:37.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 Processor -- start 2024-09-10T22:46:37.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 -- start start 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78861d30 0x558e7887f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e7887e8c0 con 0x558e7886e5e0 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e7887ef90 con 0x558e78862650 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe793ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e7887f660 con 0x558e78861d30 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43664/0 (socket says 172.21.15.96:43664) 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe6ffff640 1 -- 172.21.15.96:0/367267498 learned_addr learned my addr 172.21.15.96:0/367267498 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.493+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78861d30 0x558e7887f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe74d73640 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 -- 172.21.15.96:0/367267498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78861d30 msgr2=0x558e7887f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78861d30 0x558e7887f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 -- 172.21.15.96:0/367267498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 msgr2=0x558e7886aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 -- 172.21.15.96:0/367267498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e78862480 con 0x558e7886e5e0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c015d30 tx=0x7fbe5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d39ff32e0944c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe5a7fc640 1 -- 172.21.15.96:0/367267498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00a8a0 con 0x558e7886e5e0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe5a7fc640 1 -- 172.21.15.96:0/367267498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c019040 con 0x558e7886e5e0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.494+0000 7fbe5a7fc640 1 -- 172.21.15.96:0/367267498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c01d450 con 0x558e7886e5e0 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- 172.21.15.96:0/367267498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 msgr2=0x558e788689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c015d30 tx=0x7fbe5c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- 172.21.15.96:0/367267498 shutdown_connections 2024-09-10T22:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78862650 0x558e7886aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e788689c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 --2- 172.21.15.96:0/367267498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78861d30 0x558e7887f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- 172.21.15.96:0/367267498 >> 172.21.15.96:0/367267498 conn(0x558e7885fb60 msgr2=0x558e78869e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- 172.21.15.96:0/367267498 shutdown_connections 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- 172.21.15.96:0/367267498 wait complete. 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.495+0000 7fbe793ac740 1 -- start start 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 Processor -- start 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e7885ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e78888950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e78880400 con 0x558e7886e5e0 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e7887fd30 con 0x558e78861d30 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e78880ad0 con 0x558e78862650 2024-09-10T22:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe793ac740 10 client.0 _ll_register_callbacks cb 0x7fbe64008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43666/0 (socket says 172.21.15.96:43666) 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 -- 172.21.15.96:0/719803316 learned_addr learned my addr 172.21.15.96:0/719803316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe74d73640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e78888950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e7885ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 -- 172.21.15.96:0/719803316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 msgr2=0x558e78888950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e78888950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 -- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 msgr2=0x558e7885ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e7885ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.496+0000 7fbe6ffff640 1 -- 172.21.15.96:0/719803316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e78879550 con 0x558e7886e5e0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe6ffff640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c004700 tx=0x7fbe5c00aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03af4b6bc4775ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00a8a0 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbe5c014830 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c019040 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c00a4a0 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe793ac740 10 client.15072 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e788ebc60 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c019040 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbe5c014ce0 con 0x558e7886e5e0 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.497+0000 7fbe793ac740 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe793ac740 10 client.15072 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe793ac740 20 client.15072 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e788f0580 0x558e788ef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe793ac740 4 client.15072 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "status"}) -- 0x558e787d7610 con 0x558e788f0580 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe74d73640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e788f0580 0x558e788ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe74d73640 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e788f0580 0x558e788ef290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de54ecffb9d6f68c server_cookie=ace915fcc1a34f0a in_seq=0 out_seq=0 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.498+0000 7fbe4ffff640 10 client.15072 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe4ffff640 1 -- 172.21.15.96:0/719803316 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558e787d7610 con 0x558e788f0580 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe4ffff640 10 client.15072 handle_command_reply: tid=0 2024-09-10T22:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 client.15072 shutdown 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 msgr2=0x558e78888eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c004700 tx=0x7fbe5c00aea0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 shutdown_connections 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558e788f0580 0x558e788ef290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e7886e5e0 0x558e78888eb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e78862650 0x558e78888950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe4ffff640 0 client.15072 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 --2- 172.21.15.96:0/719803316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e78861d30 0x558e7885ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 >> 172.21.15.96:0/719803316 conn(0x558e7885fb60 msgr2=0x558e78860bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 shutdown_connections 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 1 -- 172.21.15.96:0/719803316 wait complete. 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.499+0000 7fbe793ac740 20 client.15072 trim_cache size 0 max 16384 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:46:37.520 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24527, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 67.456231622000004, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 107.45640035 2024-09-10T22:46:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 msgr2=0x7fbe600b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 0x7fbe600b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe680a8470 tx=0x7fbe6808f3e0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 msgr2=0x7fbe70075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbe68005cc0 tx=0x7fbe6802f470 comp rx=0 tx=0).stop 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 shutdown_connections 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe701087d0 0x7fbe700759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbe600b5370 0x7fbe600b7830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70109150 0x7fbe70075f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 --2- 172.21.15.96:0/794432168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe70102dc0 0x7fbe700754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:37.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 >> 172.21.15.96:0/794432168 conn(0x7fbe700fe830 msgr2=0x7fbe70100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:37.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 shutdown_connections 2024-09-10T22:46:37.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:37.500+0000 7fbe76fd7640 1 -- 172.21.15.96:0/794432168 wait complete. 2024-09-10T22:46:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:37 smithi096 ceph-mon[30869]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:37 smithi121 ceph-mon[35119]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:37 smithi117 ceph-mon[35081]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:38.209 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T22:46:38.457 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 -- 172.21.15.96:0/1565423891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb41076b0 msgr2=0x7fddb4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/1565423891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb41076b0 0x7fddb4107af0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009900 tx=0x7fdda402a410 comp rx=0 tx=0).stop 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 -- 172.21.15.96:0/1565423891 shutdown_connections 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/1565423891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb4108030 0x7fddb410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/1565423891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb41076b0 0x7fddb4107af0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/1565423891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 0x7fddb4102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 -- 172.21.15.96:0/1565423891 >> 172.21.15.96:0/1565423891 conn(0x7fddb40fd6f0 msgr2=0x7fddb40ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 -- 172.21.15.96:0/1565423891 shutdown_connections 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.956+0000 7fddbbe9a640 1 -- 172.21.15.96:0/1565423891 wait complete. 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 Processor -- start 2024-09-10T22:46:38.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 -- start start 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 0x7fddb41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb41076b0 0x7fddb4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111cb0 con 0x7fddb4108030 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111b30 con 0x7fddb4101ca0 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddbbe9a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4111e30 con 0x7fddb41076b0 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddba437640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddba437640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43694/0 (socket says 172.21.15.96:43694) 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddba437640 1 -- 172.21.15.96:0/2666359340 learned_addr learned my addr 172.21.15.96:0/2666359340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddb9c36640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 0x7fddb41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.957+0000 7fddba437640 1 -- 172.21.15.96:0/2666359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb41076b0 msgr2=0x7fddb4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddba437640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb41076b0 0x7fddb4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddba437640 1 -- 172.21.15.96:0/2666359340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 msgr2=0x7fddb41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddba437640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 0x7fddb41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddba437640 1 -- 172.21.15.96:0/2666359340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb419f380 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddba437640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fddac00eb10 tx=0x7fddac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddaaffd640 1 -- 172.21.15.96:0/2666359340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac004b20 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb419f670 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddaaffd640 1 -- 172.21.15.96:0/2666359340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddac004cc0 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddaaffd640 1 -- 172.21.15.96:0/2666359340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac010690 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.958+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb41a5d30 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.960+0000 7fddaaffd640 1 -- 172.21.15.96:0/2666359340 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fddac005020 con 0x7fddb4108030 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.960+0000 7fddbe26f740 1 Processor -- start 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 -- start start 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c498110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 0x55599c4a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55599c40fc10 con 0x55599c4a89f0 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55599c3ff040 con 0x55599c498650 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddbe26f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55599c402e50 con 0x55599c497d30 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43708/0 (socket says 172.21.15.96:43708) 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 -- 172.21.15.96:0/833743701 learned_addr learned my addr 172.21.15.96:0/833743701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9c36640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 0x55599c4a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddaaffd640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 0x7fdda00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c498110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddba437640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 0x7fdda00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 -- 172.21.15.96:0/833743701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 msgr2=0x55599c498110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c498110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 -- 172.21.15.96:0/833743701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 msgr2=0x55599c4a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 0x55599c4a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9435640 1 -- 172.21.15.96:0/833743701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599c15d540 con 0x55599c4a89f0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddb9c36640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 0x55599c4a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.961+0000 7fddaaffd640 1 -- 172.21.15.96:0/2666359340 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fddac0e6290 con 0x7fddb4108030 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddb9435640 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdda40096c0 tx=0x7fdda4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de24951feee5ef9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fdd9ffff640 1 -- 172.21.15.96:0/833743701 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4035070 con 0x55599c4a89f0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddba437640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 0x7fdda00b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddac06bff0 tx=0x7fddac013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fdd9ffff640 1 -- 172.21.15.96:0/833743701 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda4031e20 con 0x55599c4a89f0 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fdd9ffff640 1 -- 172.21.15.96:0/833743701 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda40049b0 con 0x55599c4a89f0 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddbe26f740 1 -- 172.21.15.96:0/833743701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 msgr2=0x55599c4aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddbe26f740 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdda40096c0 tx=0x7fdda4000bc0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddbe26f740 1 -- 172.21.15.96:0/833743701 shutdown_connections 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddbe26f740 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c498650 0x55599c4a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.962+0000 7fddbe26f740 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4aade0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 --2- 172.21.15.96:0/833743701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c498110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- 172.21.15.96:0/833743701 >> 172.21.15.96:0/833743701 conn(0x55599c495b60 msgr2=0x55599c496d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- 172.21.15.96:0/833743701 shutdown_connections 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- 172.21.15.96:0/833743701 wait complete. 2024-09-10T22:46:38.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- start start 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 Processor -- start 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55599c2605f0 con 0x55599c4a89f0 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55599c39a010 con 0x55599c497d30 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55599c4ab640 con 0x55599c498650 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.963+0000 7fddbe26f740 10 client.0 _ll_register_callbacks cb 0x55599c3f9340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43718/0 (socket says 172.21.15.96:43718) 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 -- 172.21.15.96:0/2146013189 learned_addr learned my addr 172.21.15.96:0/2146013189 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9c36640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 -- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 msgr2=0x55599c4a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 -- 172.21.15.96:0/2146013189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 msgr2=0x55599c4a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 -- 172.21.15.96:0/2146013189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599c3c7a00 con 0x55599c4a89f0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddb9435640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdda40056a0 tx=0x7fdda4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa73589b6e64d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fdd9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4035070 con 0x55599c4a89f0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdda4004050 con 0x55599c4a89f0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda4031e20 con 0x55599c4a89f0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdda40318e0 con 0x55599c4a89f0 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddbe26f740 10 client.15084 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:38.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.964+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55599c4b7160 con 0x55599c4a89f0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda4031e20 con 0x55599c4a89f0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdda403a040 con 0x55599c4a89f0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 10 client.15084 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 20 client.15084 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c4b84b0 0x55599c4ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 4 client.15084 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55599c411b60 con 0x55599c4b84b0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.965+0000 7fddb9c36640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c4b84b0 0x55599c4ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fdd9d7fa640 10 client.15084 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fddb9c36640 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c4b84b0 0x55599c4ba630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=722108b16501151f server_cookie=a47f21231ab7330a in_seq=0 out_seq=0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/2146013189 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55599c411b60 con 0x55599c4b84b0 2024-09-10T22:46:38.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fdd9d7fa640 10 client.15084 handle_command_reply: tid=0 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fddbe26f740 1 client.15084 shutdown 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 msgr2=0x55599c4a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.966+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdda40056a0 tx=0x7fdda4005fd0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 shutdown_connections 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c4b84b0 0x55599c4ba630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4a63a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 --2- 172.21.15.96:0/2146013189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fdd9d7fa640 0 client.15084 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 >> 172.21.15.96:0/2146013189 conn(0x55599c495b60 msgr2=0x55599c494d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 shutdown_connections 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 1 -- 172.21.15.96:0/2146013189 wait complete. 2024-09-10T22:46:38.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.967+0000 7fddbe26f740 20 client.15084 trim_cache size 0 max 16384 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.984+0000 7fddbe26f740 1 Processor -- start 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.984+0000 7fddbe26f740 1 -- start start 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c4a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 0x55599c4a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55599c4b8970 con 0x55599c498650 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55599c4b9040 con 0x55599c4a89f0 2024-09-10T22:46:38.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddbe26f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55599c4b9710 con 0x55599c497d30 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43722/0 (socket says 172.21.15.96:43722) 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 -- 172.21.15.96:0/3287069953 learned_addr learned my addr 172.21.15.96:0/3287069953 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9435640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c4a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9435640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 0x55599c4a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 -- 172.21.15.96:0/3287069953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 msgr2=0x55599c4a6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c4a6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 -- 172.21.15.96:0/3287069953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 msgr2=0x55599c4a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 0x55599c4a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 -- 172.21.15.96:0/3287069953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599c4914f0 con 0x55599c498650 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9435640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 0x55599c4a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fddb9c36640 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55599c4b8920 tx=0x7fddb0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d6b25ed75b548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fdd9f7fe640 1 -- 172.21.15.96:0/3287069953 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003d20 con 0x55599c498650 2024-09-10T22:46:38.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fdd9f7fe640 1 -- 172.21.15.96:0/3287069953 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb00108e0 con 0x55599c498650 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.985+0000 7fdd9f7fe640 1 -- 172.21.15.96:0/3287069953 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb001f780 con 0x55599c498650 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- 172.21.15.96:0/3287069953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 msgr2=0x55599c4a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55599c4b8920 tx=0x7fddb0005f90 comp rx=0 tx=0).stop 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- 172.21.15.96:0/3287069953 shutdown_connections 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c4a89f0 0x55599c4a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c498650 0x55599c4a7490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 --2- 172.21.15.96:0/3287069953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c497d30 0x55599c4a6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- 172.21.15.96:0/3287069953 >> 172.21.15.96:0/3287069953 conn(0x55599c495b60 msgr2=0x55599c4aa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- 172.21.15.96:0/3287069953 shutdown_connections 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- 172.21.15.96:0/3287069953 wait complete. 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 -- start start 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.986+0000 7fddbe26f740 1 Processor -- start 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55599c4ba4b0 con 0x55599c4a89f0 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55599c4b9de0 con 0x55599c497d30 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55599c4bab80 con 0x55599c498650 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddbe26f740 10 client.0 _ll_register_callbacks cb 0x7fddb00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43736/0 (socket says 172.21.15.96:43736) 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 -- 172.21.15.96:0/575354042 learned_addr learned my addr 172.21.15.96:0/575354042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9c36640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 -- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 msgr2=0x55599c4c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 -- 172.21.15.96:0/575354042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 msgr2=0x55599c4c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 -- 172.21.15.96:0/575354042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599c4bdd80 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9c36640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fddb9435640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005c40 tx=0x7fdda4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6e15620cc5c054 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.987+0000 7fdd9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda4035040 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdda40057a0 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb00108e0 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdda406c050 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 10 client.15090 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55599c526190 con 0x55599c4a89f0 2024-09-10T22:46:38.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb00108e0 con 0x55599c4a89f0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdda403d7e0 con 0x55599c4a89f0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 20 client.15090 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c52aca0 0x55599c5299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 4 client.15090 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "status"}) -- 0x55599c40d610 con 0x55599c52aca0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.988+0000 7fddb9c36640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c52aca0 0x55599c5299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddb9c36640 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c52aca0 0x55599c5299b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4d5f17ae06c55 server_cookie=a935b2cae4064a70 in_seq=0 out_seq=0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fdd9d7fa640 10 client.15090 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fdd9d7fa640 1 -- 172.21.15.96:0/575354042 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55599c40d610 con 0x55599c52aca0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fdd9d7fa640 10 client.15090 handle_command_reply: tid=0 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 client.15090 shutdown 2024-09-10T22:46:38.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 msgr2=0x55599c4bd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005c40 tx=0x7fdda4005fd0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 shutdown_connections 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55599c52aca0 0x55599c5299b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55599c4a89f0 0x55599c4bd840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fdd9d7fa640 0 client.15090 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55599c498650 0x55599c4c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 --2- 172.21.15.96:0/575354042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55599c497d30 0x55599c4c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 >> 172.21.15.96:0/575354042 conn(0x55599c495b60 msgr2=0x55599c4a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 shutdown_connections 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.989+0000 7fddbe26f740 1 -- 172.21.15.96:0/575354042 wait complete. 2024-09-10T22:46:38.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbe26f740 20 client.15090 trim_cache size 0 max 16384 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24529, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 69.961535451000003, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:46:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 103.750116291 2024-09-10T22:46:39.000 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:46:39.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 msgr2=0x7fdda00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:39.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 0x7fdda00b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddac06bff0 tx=0x7fddac013000 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 msgr2=0x7fddb419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fddac00eb10 tx=0x7fddac00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 shutdown_connections 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdda00b5370 0x7fdda00b7830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddb4108030 0x7fddb419ebb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb41076b0 0x7fddb4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 --2- 172.21.15.96:0/2666359340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb4101ca0 0x7fddb41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 >> 172.21.15.96:0/2666359340 conn(0x7fddb40fd6f0 msgr2=0x7fddb406b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 shutdown_connections 2024-09-10T22:46:39.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:38.990+0000 7fddbbe9a640 1 -- 172.21.15.96:0/2666359340 wait complete. 2024-09-10T22:46:39.672 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:39 smithi096 ceph-mon[30869]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:39.699 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:46:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:39 smithi121 ceph-mon[35119]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:39 smithi117 ceph-mon[35081]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:39.945 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 -- 172.21.15.96:0/1361546529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 msgr2=0x7f2820103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 --2- 172.21.15.96:0/1361546529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820103180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2810009900 tx=0x7f281002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 -- 172.21.15.96:0/1361546529 shutdown_connections 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 --2- 172.21.15.96:0/1361546529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2820109130 0x7f282010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 --2- 172.21.15.96:0/1361546529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28201087b0 0x7f2820108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 --2- 172.21.15.96:0/1361546529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820103180 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 -- 172.21.15.96:0/1361546529 >> 172.21.15.96:0/1361546529 conn(0x7f28200fe810 msgr2=0x7f2820100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 -- 172.21.15.96:0/1361546529 shutdown_connections 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.437+0000 7f28272f6640 1 -- 172.21.15.96:0/1361546529 wait complete. 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 Processor -- start 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 -- start start 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28201087b0 0x7f28201997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2820109130 0x7f282019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2820112df0 con 0x7f2820102da0 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2820112c70 con 0x7f28201087b0 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f28272f6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2820112f70 con 0x7f2820109130 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f2825092640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f2825092640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43744/0 (socket says 172.21.15.96:43744) 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f2825092640 1 -- 172.21.15.96:0/3139101805 learned_addr learned my addr 172.21.15.96:0/3139101805 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.438+0000 7f2824891640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28201087b0 0x7f28201997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 -- 172.21.15.96:0/3139101805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2820109130 msgr2=0x7f282019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2820109130 0x7f282019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 -- 172.21.15.96:0/3139101805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28201087b0 msgr2=0x7f28201997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28201087b0 0x7f28201997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 -- 172.21.15.96:0/3139101805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282019f330 con 0x7f2820102da0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f2825092640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2810006010 tx=0x7f28100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f281e7fc640 1 -- 172.21.15.96:0/3139101805 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281003f030 con 0x7f2820102da0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282019f5c0 con 0x7f2820102da0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f281e7fc640 1 -- 172.21.15.96:0/3139101805 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2810003800 con 0x7f2820102da0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f281e7fc640 1 -- 172.21.15.96:0/3139101805 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810035070 con 0x7f2820102da0 2024-09-10T22:46:40.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.439+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201a5ce0 con 0x7f2820102da0 2024-09-10T22:46:40.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 Processor -- start 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f281e7fc640 1 -- 172.21.15.96:0/3139101805 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f2810003cb0 con 0x7f2820102da0 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 -- start start 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd38650 0x55d9ffd484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffcafc10 con 0x55d9ffd37d30 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffc9f040 con 0x55d9ffd489f0 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f28296cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffca2e50 con 0x55d9ffd38650 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43752/0 (socket says 172.21.15.96:43752) 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 -- 172.21.15.96:0/4220997473 learned_addr learned my addr 172.21.15.96:0/4220997473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:40.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825092640 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd38650 0x55d9ffd484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 -- 172.21.15.96:0/4220997473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd38650 msgr2=0x55d9ffd484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd38650 0x55d9ffd484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 -- 172.21.15.96:0/4220997473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 msgr2=0x55d9ffd4ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.442+0000 7f2825893640 1 -- 172.21.15.96:0/4220997473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ff9fd540 con 0x55d9ffd37d30 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f281e7fc640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 0x7f28080b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f2825893640 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2818007d20 tx=0x7f281800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a526fb92300324cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f2825092640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 0x7f28080b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f28077fe640 1 -- 172.21.15.96:0/4220997473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818014070 con 0x55d9ffd37d30 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f28077fe640 1 -- 172.21.15.96:0/4220997473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281800fb00 con 0x55d9ffd37d30 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f28077fe640 1 -- 172.21.15.96:0/4220997473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281800eb90 con 0x55d9ffd37d30 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.443+0000 7f281e7fc640 1 -- 172.21.15.96:0/3139101805 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f28101065a0 con 0x7f2820102da0 2024-09-10T22:46:40.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f2825092640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 0x7f28080b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55d9ffd3aeb0 tx=0x7f281002a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- 172.21.15.96:0/4220997473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 msgr2=0x55d9ffd38110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2818007d20 tx=0x7f281800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- 172.21.15.96:0/4220997473 shutdown_connections 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd38650 0x55d9ffd484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 --2- 172.21.15.96:0/4220997473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd38110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- 172.21.15.96:0/4220997473 >> 172.21.15.96:0/4220997473 conn(0x55d9ffd35b60 msgr2=0x55d9ffd36d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- 172.21.15.96:0/4220997473 shutdown_connections 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- 172.21.15.96:0/4220997473 wait complete. 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 -- start start 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.444+0000 7f28296cb740 1 Processor -- start 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd46470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffb005f0 con 0x55d9ffd37d30 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffc3a010 con 0x55d9ffd38650 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd4b6a0 con 0x55d9ffd489f0 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 10 client.0 _ll_register_callbacks cb 0x55d9ffc99340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43764/0 (socket says 172.21.15.96:43764) 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 -- 172.21.15.96:0/3276509205 learned_addr learned my addr 172.21.15.96:0/3276509205 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2824891640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd45f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 -- 172.21.15.96:0/3276509205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 msgr2=0x55d9ffd46470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd46470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 -- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 msgr2=0x55d9ffd45f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd45f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 -- 172.21.15.96:0/3276509205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ffc67a00 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2824891640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2825893640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f281800e670 tx=0x7f281800e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5fa85232ca530f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818014050 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2818003d80 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.445+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281800fb00 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f281800f450 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 10 client.15102 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9ffd57290 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f281800fb00 con 0x55d9ffd37d30 2024-09-10T22:46:40.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2818004200 con 0x55d9ffd37d30 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 20 client.15102 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffd595b0 0x55d9ffd5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 4 client.15102 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9ffcb1b60 con 0x55d9ffd595b0 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.446+0000 7f2824891640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffd595b0 0x55d9ffd5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.447+0000 7f2824891640 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffd595b0 0x55d9ffd5aa20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7adfc59706cd28ee server_cookie=8979299e3d543a3a in_seq=0 out_seq=0 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.447+0000 7f2804ff9640 10 client.15102 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f2804ff9640 1 -- 172.21.15.96:0/3276509205 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9ffcb1b60 con 0x55d9ffd595b0 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f2804ff9640 10 client.15102 handle_command_reply: tid=0 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 client.15102 shutdown 2024-09-10T22:46:40.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 msgr2=0x55d9ffd459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f281800e670 tx=0x7f281800e480 comp rx=0 tx=0).stop 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 shutdown_connections 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffd595b0 0x55d9ffd5aa20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd46470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 --2- 172.21.15.96:0/3276509205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd459f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 >> 172.21.15.96:0/3276509205 conn(0x55d9ffd35b60 msgr2=0x55d9ffd34d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f2804ff9640 0 client.15102 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 shutdown_connections 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 1 -- 172.21.15.96:0/3276509205 wait complete. 2024-09-10T22:46:40.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.448+0000 7f28296cb740 20 client.15102 trim_cache size 0 max 16384 2024-09-10T22:46:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 Processor -- start 2024-09-10T22:46:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 -- start start 2024-09-10T22:46:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd37d30 0x55d9ffd43480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd58d60 con 0x55d9ffd38650 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd59430 con 0x55d9ffd489f0 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f28296cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd59b00 con 0x55d9ffd37d30 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f2824891640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f2824891640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43766/0 (socket says 172.21.15.96:43766) 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.466+0000 7f2824891640 1 -- 172.21.15.96:0/3069467148 learned_addr learned my addr 172.21.15.96:0/3069467148 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 -- 172.21.15.96:0/3069467148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd37d30 msgr2=0x55d9ffd43480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd37d30 0x55d9ffd43480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 -- 172.21.15.96:0/3069467148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 msgr2=0x55d9ffd4a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 -- 172.21.15.96:0/3069467148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ffd35990 con 0x55d9ffd38650 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2824891640 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28201020e0 tx=0x7f2814008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb22a7eab3fb743 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2806ffd640 1 -- 172.21.15.96:0/3069467148 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814016a90 con 0x55d9ffd38650 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2806ffd640 1 -- 172.21.15.96:0/3069467148 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2814016c30 con 0x55d9ffd38650 2024-09-10T22:46:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f2806ffd640 1 -- 172.21.15.96:0/3069467148 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140146a0 con 0x55d9ffd38650 2024-09-10T22:46:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f28296cb740 1 -- 172.21.15.96:0/3069467148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 msgr2=0x55d9ffd49ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.467+0000 7f28296cb740 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28201020e0 tx=0x7f2814008040 comp rx=0 tx=0).stop 2024-09-10T22:46:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- 172.21.15.96:0/3069467148 shutdown_connections 2024-09-10T22:46:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd489f0 0x55d9ffd4a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd38650 0x55d9ffd49ad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- 172.21.15.96:0/3069467148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd37d30 0x55d9ffd43480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- 172.21.15.96:0/3069467148 >> 172.21.15.96:0/3069467148 conn(0x55d9ffd35b60 msgr2=0x55d9ffd4aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- 172.21.15.96:0/3069467148 shutdown_connections 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- 172.21.15.96:0/3069467148 wait complete. 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- start start 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 Processor -- start 2024-09-10T22:46:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd5dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd5a8a0 con 0x55d9ffd37d30 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd5a1d0 con 0x55d9ffd38650 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9ffd5af70 con 0x55d9ffd489f0 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f28296cb740 10 client.0 _ll_register_callbacks cb 0x7f2818007b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f2825893640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43782/0 (socket says 172.21.15.96:43782) 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.468+0000 7f2825893640 1 -- 172.21.15.96:0/42844867 learned_addr learned my addr 172.21.15.96:0/42844867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 -- 172.21.15.96:0/42844867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 msgr2=0x55d9ffd5dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd5dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 -- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 msgr2=0x55d9ffd634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 -- 172.21.15.96:0/42844867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ffd5e480 con 0x55d9ffd37d30 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2825893640 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2818007fd0 tx=0x7f28180071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17591622d614a8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818033030 con 0x55d9ffd37d30 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2818002030 con 0x55d9ffd37d30 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2814016c30 con 0x55d9ffd37d30 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2818014070 con 0x55d9ffd37d30 2024-09-10T22:46:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f28296cb740 10 client.15108 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9ffdc6a30 con 0x55d9ffd37d30 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.469+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2814016c30 con 0x55d9ffd37d30 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f281800ec70 con 0x55d9ffd37d30 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 20 client.15108 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffdcb610 0x55d9ffdca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 4 client.15108 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "status"}) -- 0x55d9ffcad610 con 0x55d9ffdcb610 2024-09-10T22:46:40.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2824891640 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffdcb610 0x55d9ffdca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2824891640 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffdcb610 0x55d9ffdca320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e00d1b7978f2e36 server_cookie=69ff6ff95a24bf98 in_seq=0 out_seq=0 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2804ff9640 10 client.15108 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2804ff9640 1 -- 172.21.15.96:0/42844867 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55d9ffcad610 con 0x55d9ffdcb610 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.470+0000 7f2804ff9640 10 client.15108 handle_command_reply: tid=0 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 client.15108 shutdown 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 msgr2=0x55d9ffd62f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2818007fd0 tx=0x7f28180071c0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 shutdown_connections 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55d9ffdcb610 0x55d9ffdca320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9ffd38650 0x55d9ffd634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9ffd489f0 0x55d9ffd5dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f2804ff9640 0 client.15108 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 --2- 172.21.15.96:0/42844867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9ffd37d30 0x55d9ffd62f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 >> 172.21.15.96:0/42844867 conn(0x55d9ffd35b60 msgr2=0x55d9ffd49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 shutdown_connections 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 1 -- 172.21.15.96:0/42844867 wait complete. 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28296cb740 20 client.15108 trim_cache size 0 max 16384 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:46:40.480 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24542, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 72.450330248, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 102.522375823 2024-09-10T22:46:40.481 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 msgr2=0x7f28080b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 0x7f28080b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55d9ffd3aeb0 tx=0x7f281002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 msgr2=0x7f2820199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2810006010 tx=0x7f28100042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 shutdown_connections 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28201087b0 0x7f28201997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f28080b5370 0x7f28080b7830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2820109130 0x7f282019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.471+0000 7f28272f6640 1 --2- 172.21.15.96:0/3139101805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2820102da0 0x7f2820199280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:40.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.472+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 >> 172.21.15.96:0/3139101805 conn(0x7f28200fe810 msgr2=0x7f2820100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:40.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.472+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 shutdown_connections 2024-09-10T22:46:40.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:40.472+0000 7f28272f6640 1 -- 172.21.15.96:0/3139101805 wait complete. 2024-09-10T22:46:41.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T22:46:41.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:41 smithi096 ceph-mon[30869]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:41.504 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:41 smithi121 ceph-mon[35119]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:41 smithi117 ceph-mon[35081]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:42.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 -- 172.21.15.96:0/3262580939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 msgr2=0x7f7da410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/3262580939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da410f950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009be0 tx=0x7f7d9c02a6c0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 -- 172.21.15.96:0/3262580939 shutdown_connections 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/3262580939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da410f950 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/3262580939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da41087d0 0x7f7da4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/3262580939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da4102dc0 0x7f7da41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 -- 172.21.15.96:0/3262580939 >> 172.21.15.96:0/3262580939 conn(0x7f7da40fe830 msgr2=0x7f7da4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 -- 172.21.15.96:0/3262580939 shutdown_connections 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.971+0000 7f7da9c5e640 1 -- 172.21.15.96:0/3262580939 wait complete. 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 Processor -- start 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 -- start start 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da4102dc0 0x7f7da4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 0x7f7da41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112dd0 con 0x7f7da4109150 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112c50 con 0x7f7da41087d0 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da9c5e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112f50 con 0x7f7da4102dc0 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43802/0 (socket says 172.21.15.96:43802) 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da3fff640 1 -- 172.21.15.96:0/47232922 learned_addr learned my addr 172.21.15.96:0/47232922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7da37fe640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da4102dc0 0x7f7da4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.972+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 0x7f7da41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 -- 172.21.15.96:0/47232922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da4102dc0 msgr2=0x7f7da4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da4102dc0 0x7f7da4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 -- 172.21.15.96:0/47232922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 msgr2=0x7f7da41997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 0x7f7da41997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 -- 172.21.15.96:0/47232922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419f330 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 0x7f7da41997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da3fff640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c004af0 tx=0x7f7d9c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da17fa640 1 -- 172.21.15.96:0/47232922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c041020 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da17fa640 1 -- 172.21.15.96:0/47232922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9c003e10 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419f5c0 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da17fa640 1 -- 172.21.15.96:0/47232922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c036040 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.973+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a5ce0 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.975+0000 7f7da17fa640 1 -- 172.21.15.96:0/47232922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f7d9c030b70 con 0x7f7da4109150 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.975+0000 7f7dac033740 1 Processor -- start 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.975+0000 7f7dac033740 1 -- start start 2024-09-10T22:46:42.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3ed30 0x55d819d3f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d819cb6c10 con 0x55d819d4b5e0 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d819ca6040 con 0x55d819d3ed30 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7dac033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d819ca9e50 con 0x55d819d3f650 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43806/0 (socket says 172.21.15.96:43806) 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 -- 172.21.15.96:0/2701180804 learned_addr learned my addr 172.21.15.96:0/2701180804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3ed30 0x55d819d3f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 -- 172.21.15.96:0/2701180804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 msgr2=0x55d819d4b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d4b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 -- 172.21.15.96:0/2701180804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3ed30 msgr2=0x55d819d3f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3ed30 0x55d819d3f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 -- 172.21.15.96:0/2701180804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d819a04540 con 0x55d819d4b5e0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7da17fa640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 0x7f7d900b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c012850 tx=0x7f7d8c009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fa4bd7c52be935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7da37fe640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 0x7f7d900b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7da17fa640 1 -- 172.21.15.96:0/47232922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7d9c106ab0 con 0x7f7da4109150 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.976+0000 7f7d99ffb640 1 -- 172.21.15.96:0/2701180804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c005020 con 0x55d819d4b5e0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7d99ffb640 1 -- 172.21.15.96:0/2701180804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c0047a0 con 0x55d819d4b5e0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7d99ffb640 1 -- 172.21.15.96:0/2701180804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c010a30 con 0x55d819d4b5e0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 -- 172.21.15.96:0/2701180804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 msgr2=0x55d819d4d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7da37fe640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 0x7f7d900b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55d819d41eb0 tx=0x7f7d94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c012850 tx=0x7f7d8c009720 comp rx=0 tx=0).stop 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 -- 172.21.15.96:0/2701180804 shutdown_connections 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d4d9d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d4b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 --2- 172.21.15.96:0/2701180804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3ed30 0x55d819d3f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 -- 172.21.15.96:0/2701180804 >> 172.21.15.96:0/2701180804 conn(0x55d819d3cb60 msgr2=0x55d819d3dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 -- 172.21.15.96:0/2701180804 shutdown_connections 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.977+0000 7f7dac033740 1 -- 172.21.15.96:0/2701180804 wait complete. 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 -- start start 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 Processor -- start 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d48600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 0x55d819d48b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d819b075f0 con 0x55d819d4b5e0 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d819c41010 con 0x55d819d3f650 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d819d4e290 con 0x55d819d3ed30 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7dac033740 10 client.0 _ll_register_callbacks cb 0x55d819ca0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43822/0 (socket says 172.21.15.96:43822) 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7da3fff640 1 -- 172.21.15.96:0/2581808478 learned_addr learned my addr 172.21.15.96:0/2581808478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 0x55d819d48b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.978+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d48600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 -- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 msgr2=0x55d819d48600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d48600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 -- 172.21.15.96:0/2581808478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 msgr2=0x55d819d48b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 0x55d819d48b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 -- 172.21.15.96:0/2581808478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d819c6ea00 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 0x55d819d48b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7da3fff640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c02ac80 tx=0x7f7d9c0cd110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08dc4e9f4c119df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004d40 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7d9c0cc470 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c0047a0 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c0ccd20 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7dac033740 10 client.15120 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d819d59ed0 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c0047a0 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d9c0f4020 con 0x55d819d4b5e0 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.979+0000 7f7dac033740 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:42.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7dac033740 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7dac033740 20 client.15120 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819d5c580 0x55d819d5d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7dac033740 4 client.15120 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d819cb8b60 con 0x55d819d5c580 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819d5c580 0x55d819d5d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7d82ffd640 10 client.15120 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.980+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819d5c580 0x55d819d5d9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90914690a11cb71a server_cookie=fa0da56443980f2e in_seq=0 out_seq=0 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2581808478 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d819cb8b60 con 0x55d819d5c580 2024-09-10T22:46:42.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7d82ffd640 10 client.15120 handle_command_reply: tid=0 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 client.15120 shutdown 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 msgr2=0x55d819d49080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c02ac80 tx=0x7f7d9c0cd110 comp rx=0 tx=0).stop 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 shutdown_connections 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819d5c580 0x55d819d5d9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d3f650 0x55d819d48b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d4b5e0 0x55d819d49080 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 --2- 172.21.15.96:0/2581808478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d48600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7d82ffd640 0 client.15120 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 >> 172.21.15.96:0/2581808478 conn(0x55d819d3cb60 msgr2=0x55d819d42700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 shutdown_connections 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 1 -- 172.21.15.96:0/2581808478 wait complete. 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.981+0000 7f7dac033740 20 client.15120 trim_cache size 0 max 16384 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 Processor -- start 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 -- start start 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d3dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5bd30 con 0x55d819d3f650 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5c400 con 0x55d819d4b5e0 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7dac033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5cad0 con 0x55d819d3ed30 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43832/0 (socket says 172.21.15.96:43832) 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 -- 172.21.15.96:0/772812182 learned_addr learned my addr 172.21.15.96:0/772812182 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7da3fff640 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 -- 172.21.15.96:0/772812182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 msgr2=0x55d819d3dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d3dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:41.999+0000 7f7d9bfff640 1 -- 172.21.15.96:0/772812182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 msgr2=0x55d819d473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d9bfff640 1 -- 172.21.15.96:0/772812182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d819d3c990 con 0x55d819d3f650 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7da3fff640 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c012fa0 tx=0x7f7d8c012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8635d4ec58f413 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d997fa640 1 -- 172.21.15.96:0/772812182 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c02d070 con 0x55d819d3f650 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d997fa640 1 -- 172.21.15.96:0/772812182 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c004660 con 0x55d819d3f650 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7d997fa640 1 -- 172.21.15.96:0/772812182 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c01f9a0 con 0x55d819d3f650 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7dac033740 1 -- 172.21.15.96:0/772812182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 msgr2=0x55d819d46e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.000+0000 7f7dac033740 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c012fa0 tx=0x7f7d8c012f30 comp rx=0 tx=0).stop 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- 172.21.15.96:0/772812182 shutdown_connections 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3f650 0x55d819d46e80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- 172.21.15.96:0/772812182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3ed30 0x55d819d3dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- 172.21.15.96:0/772812182 >> 172.21.15.96:0/772812182 conn(0x55d819d3cb60 msgr2=0x55d819d43870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- 172.21.15.96:0/772812182 shutdown_connections 2024-09-10T22:46:42.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- 172.21.15.96:0/772812182 wait complete. 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- start start 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 Processor -- start 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d5f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5d870 con 0x55d819d3ed30 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5d1a0 con 0x55d819d4b5e0 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7dac033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d819d5df40 con 0x55d819d3f650 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.001+0000 7f7da37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7dac033740 10 client.0 _ll_register_callbacks cb 0x7f7d9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43846/0 (socket says 172.21.15.96:43846) 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 -- 172.21.15.96:0/2668731032 learned_addr learned my addr 172.21.15.96:0/2668731032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da3fff640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d5f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 -- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 msgr2=0x55d819d5f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d5f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 -- 172.21.15.96:0/2668731032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 msgr2=0x55d819d5f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d5f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 -- 172.21.15.96:0/2668731032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d819d65d00 con 0x55d819d3ed30 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7da37fe640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7d94004230 tx=0x7f7d94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb8b0f463f2b287 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94005020 con 0x55d819d3ed30 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7d940051c0 con 0x55d819d3ed30 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c004660 con 0x55d819d3ed30 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d94039050 con 0x55d819d3ed30 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7dac033740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:42.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d819dcd6a0 con 0x55d819d3ed30 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.002+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c004660 con 0x55d819d3ed30 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d9402b470 con 0x55d819d3ed30 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 20 client.15126 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819dd22a0 0x55d819dd0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 4 client.15126 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "status"}) -- 0x55d819cb4610 con 0x55d819dd22a0 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819dd22a0 0x55d819dd0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7d82ffd640 10 client.15126 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:46:42.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.003+0000 7f7d9bfff640 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819dd22a0 0x55d819dd0fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c8a847161d11eb5 server_cookie=1c6b7c16b7920274 in_seq=0 out_seq=0 2024-09-10T22:46:42.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7d82ffd640 1 -- 172.21.15.96:0/2668731032 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d819cb4610 con 0x55d819dd22a0 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7d82ffd640 10 client.15126 handle_command_reply: tid=0 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 client.15126 shutdown 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 msgr2=0x55d819d5eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7d94004230 tx=0x7f7d94004730 comp rx=0 tx=0).stop 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 shutdown_connections 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d819dd22a0 0x55d819dd0fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d819d4b5e0 0x55d819d5f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7d82ffd640 0 client.15126 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d819d3f650 0x55d819d5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 --2- 172.21.15.96:0/2668731032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d819d3ed30 0x55d819d5eba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 >> 172.21.15.96:0/2668731032 conn(0x55d819d3cb60 msgr2=0x55d819d57790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 shutdown_connections 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 1 -- 172.21.15.96:0/2668731032 wait complete. 2024-09-10T22:46:42.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7dac033740 20 client.15126 trim_cache size 0 max 16384 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24550, 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 74.992962594000005, 2024-09-10T22:46:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:46:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:46:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:46:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 101.42452190500001 2024-09-10T22:46:42.024 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:46:42.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 msgr2=0x7f7d900b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 0x7f7d900b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55d819d41eb0 tx=0x7f7d94005f90 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 msgr2=0x7f7da419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.004+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c004af0 tx=0x7f7d9c003c20 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 shutdown_connections 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da41087d0 0x7f7da41997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d900b53f0 0x7f7d900b78b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da4109150 0x7f7da419eb60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 --2- 172.21.15.96:0/47232922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7da4102dc0 0x7f7da4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 >> 172.21.15.96:0/47232922 conn(0x7f7da40fe830 msgr2=0x7f7da4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 shutdown_connections 2024-09-10T22:46:42.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:42.005+0000 7f7da9c5e640 1 -- 172.21.15.96:0/47232922 wait complete. 2024-09-10T22:46:42.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:46:43.011 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- 172.21.15.96:0/753923195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 msgr2=0x7f41b0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/753923195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b0108c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009900 tx=0x7f41a402a410 comp rx=0 tx=0).stop 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- 172.21.15.96:0/753923195 shutdown_connections 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/753923195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 0x7f41b010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/753923195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b0108c30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/753923195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- 172.21.15.96:0/753923195 >> 172.21.15.96:0/753923195 conn(0x7f41b00fe870 msgr2=0x7f41b0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- 172.21.15.96:0/753923195 shutdown_connections 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- 172.21.15.96:0/753923195 wait complete. 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 Processor -- start 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.537+0000 7f41b7bbf640 1 -- start start 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 0x7f41b019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41b0112df0 con 0x7f41b01087f0 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41b0112c70 con 0x7f41b0102de0 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f41b0112f70 con 0x7f41b0109170 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43864/0 (socket says 172.21.15.96:43864) 2024-09-10T22:46:43.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 -- 172.21.15.96:0/1863663543 learned_addr learned my addr 172.21.15.96:0/1863663543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b615c640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 0x7f41b019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b595b640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 -- 172.21.15.96:0/1863663543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 msgr2=0x7f41b019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 0x7f41b019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 -- 172.21.15.96:0/1863663543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 msgr2=0x7f41b01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 -- 172.21.15.96:0/1863663543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b019f350 con 0x7f41b01087f0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b595b640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b515a640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005940 tx=0x7f41a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41aeffd640 1 -- 172.21.15.96:0/1863663543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a403f030 con 0x7f41b01087f0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41aeffd640 1 -- 172.21.15.96:0/1863663543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a4003cb0 con 0x7f41b01087f0 2024-09-10T22:46:43.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b019f5e0 con 0x7f41b01087f0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41aeffd640 1 -- 172.21.15.96:0/1863663543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4035070 con 0x7f41b01087f0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.538+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b01a5d00 con 0x7f41b01087f0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41aeffd640 1 -- 172.21.15.96:0/1863663543 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f41a403bbf0 con 0x7f41b01087f0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 Processor -- start 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 -- start start 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662279110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b6622850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6621f0c10 con 0x55b6622855e0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6621e0040 con 0x55b662278d30 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b9f94740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b6621e3e50 con 0x55b662279650 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43874/0 (socket says 172.21.15.96:43874) 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b515a640 1 -- 172.21.15.96:0/3588723622 learned_addr learned my addr 172.21.15.96:0/3588723622 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.541+0000 7f41b595b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b6622850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b615c640 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662279110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41aeffd640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 0x7f41980b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b615c640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 0x7f41980b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 -- 172.21.15.96:0/3588723622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 msgr2=0x55b6622850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b6622850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 -- 172.21.15.96:0/3588723622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 msgr2=0x55b662279110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662279110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 -- 172.21.15.96:0/3588723622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b661f3e540 con 0x55b6622855e0 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b595b640 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b6622850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:46:43.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41aeffd640 1 -- 172.21.15.96:0/1863663543 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f41a4106480 con 0x7f41b01087f0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b515a640 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f41a403df60 tx=0x7f41a404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a84bca437045e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f41b615c640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 0x7f41980b7830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005fd0 tx=0x7f41a0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f4197fff640 1 -- 172.21.15.96:0/3588723622 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40cc530 con 0x55b6622855e0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f4197fff640 1 -- 172.21.15.96:0/3588723622 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a40cc6f0 con 0x55b6622855e0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.542+0000 7f4197fff640 1 -- 172.21.15.96:0/3588723622 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40cc350 con 0x55b6622855e0 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- 172.21.15.96:0/3588723622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 msgr2=0x55b6622879d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f41a403df60 tx=0x7f41a404c000 comp rx=0 tx=0).stop 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- 172.21.15.96:0/3588723622 shutdown_connections 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b6622879d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b6622850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3588723622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662279110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- 172.21.15.96:0/3588723622 >> 172.21.15.96:0/3588723622 conn(0x55b662276b60 msgr2=0x55b662277d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- 172.21.15.96:0/3588723622 shutdown_connections 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- 172.21.15.96:0/3588723622 wait complete. 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.543+0000 7f41b9f94740 1 -- start start 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 Processor -- start 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662282550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b662282a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6620415f0 con 0x55b6622855e0 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b66217b010 con 0x55b662278d30 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b6621f0c10 con 0x55b662279650 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b9f94740 10 client.0 _ll_register_callbacks cb 0x55b6621da340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43878/0 (socket says 172.21.15.96:43878) 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 -- 172.21.15.96:0/1294874166 learned_addr learned my addr 172.21.15.96:0/1294874166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b595b640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662282550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 -- 172.21.15.96:0/1294874166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 msgr2=0x55b662282a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b662282a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 -- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 msgr2=0x55b662282550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662282550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 -- 172.21.15.96:0/1294874166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6621a8a00 con 0x55b6622855e0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41b615c640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f41a0010a30 tx=0x7f41a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01db4a99c0cbdee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.544+0000 7f41957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0018070 con 0x55b6622855e0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f41a001dd80 con 0x55b6622855e0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a40cc6f0 con 0x55b6622855e0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41a0004b20 con 0x55b6622855e0 2024-09-10T22:46:43.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 10 client.15138 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b662293a90 con 0x55b6622855e0 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a40cc6f0 con 0x55b6622855e0 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f41a000f040 con 0x55b6622855e0 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 20 client.15138 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b662296130 0x55b6622975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 4 client.15138 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.545+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6621f2b60 con 0x55b662296130 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.546+0000 7f41b595b640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b662296130 0x55b6622975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.546+0000 7f41b595b640 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b662296130 0x55b6622975a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07fddf5138dddc4 server_cookie=1a2bd7b1800c8a4e in_seq=0 out_seq=0 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.546+0000 7f41957fa640 10 client.15138 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41957fa640 1 -- 172.21.15.96:0/1294874166 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6621f2b60 con 0x55b662296130 2024-09-10T22:46:43.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41957fa640 10 client.15138 handle_command_reply: tid=0 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 client.15138 shutdown 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 msgr2=0x55b662282fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f41a0010a30 tx=0x7f41a0004a40 comp rx=0 tx=0).stop 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 shutdown_connections 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b662296130 0x55b6622975a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6622855e0 0x55b662282fd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b662279650 0x55b662282a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1294874166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662282550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41957fa640 0 client.15138 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 >> 172.21.15.96:0/1294874166 conn(0x55b662276b60 msgr2=0x55b66227c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 shutdown_connections 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 1 -- 172.21.15.96:0/1294874166 wait complete. 2024-09-10T22:46:43.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.547+0000 7f41b9f94740 20 client.15138 trim_cache size 0 max 16384 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.565+0000 7f41b9f94740 1 Processor -- start 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.565+0000 7f41b9f94740 1 -- start start 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b66227cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66229fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6622958e0 con 0x55b662279650 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b662295fb0 con 0x55b662278d30 2024-09-10T22:46:43.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b9f94740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b662296680 con 0x55b6622855e0 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43882/0 (socket says 172.21.15.96:43882) 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 -- 172.21.15.96:0/3440283936 learned_addr learned my addr 172.21.15.96:0/3440283936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b615c640 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66229fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b595b640 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b66227cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 -- 172.21.15.96:0/3440283936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 msgr2=0x55b66229fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66229fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 -- 172.21.15.96:0/3440283936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 msgr2=0x55b66227cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b66227cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 -- 172.21.15.96:0/3440283936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b662276990 con 0x55b662279650 2024-09-10T22:46:43.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41b515a640 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41a4003fd0 tx=0x7f41a40ccab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dca757ef6e2768 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41977fe640 1 -- 172.21.15.96:0/3440283936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4049070 con 0x55b662279650 2024-09-10T22:46:43.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41977fe640 1 -- 172.21.15.96:0/3440283936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a4105040 con 0x55b662279650 2024-09-10T22:46:43.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.566+0000 7f41977fe640 1 -- 172.21.15.96:0/3440283936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a403b460 con 0x55b662279650 2024-09-10T22:46:43.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- 172.21.15.96:0/3440283936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 msgr2=0x55b662284290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41a4003fd0 tx=0x7f41a40ccab0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- 172.21.15.96:0/3440283936 shutdown_connections 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66229fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b662284290 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 --2- 172.21.15.96:0/3440283936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b66227cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- 172.21.15.96:0/3440283936 >> 172.21.15.96:0/3440283936 conn(0x55b662276b60 msgr2=0x55b662283b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- 172.21.15.96:0/3440283936 shutdown_connections 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- 172.21.15.96:0/3440283936 wait complete. 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.567+0000 7f41b9f94740 1 -- start start 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 Processor -- start 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662280020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66227ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b662297420 con 0x55b662279650 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b662296d50 con 0x55b662278d30 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b662297af0 con 0x55b6622855e0 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b9f94740 10 client.0 _ll_register_callbacks cb 0x7f41a00106e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43886/0 (socket says 172.21.15.96:43886) 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 -- 172.21.15.96:0/1748034333 learned_addr learned my addr 172.21.15.96:0/1748034333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b615c640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66227ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b595b640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662280020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 -- 172.21.15.96:0/1748034333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 msgr2=0x55b66227ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66227ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 -- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 msgr2=0x55b662280020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:46:43.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662280020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b515a640 1 -- 172.21.15.96:0/1748034333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6622775e0 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.568+0000 7f41b595b640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662280020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b515a640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41a4103fb0 tx=0x7f41a40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63de0f4b8005bd02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4049040 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f41a40cd050 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a4105040 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41a40cc350 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 10 client.15144 fetch_fsmap learned FSMap version 27 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b6623079c0 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a4105040 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f41a40d5ce0 con 0x55b662279650 2024-09-10T22:46:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 20 client.15144 populate_metadata read hostname 'smithi096' 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b66230c130 0x55b66230ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 4 client.15144 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.569+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "status"}) -- 0x55b6621ee610 con 0x55b66230c130 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b595b640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b66230c130 0x55b66230ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b595b640 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b66230c130 0x55b66230ae40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340521cda9c8a68f server_cookie=2819ddac53e9c426 in_seq=0 out_seq=0 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41957fa640 10 client.15144 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41957fa640 1 -- 172.21.15.96:0/1748034333 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b6621ee610 con 0x55b66230c130 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41957fa640 10 client.15144 handle_command_reply: tid=0 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 client.15144 shutdown 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 msgr2=0x55b66227e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41a4103fb0 tx=0x7f41a40045a0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 shutdown_connections 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b66230c130 0x55b66230ae40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6622855e0 0x55b66227ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b662279650 0x55b66227e770 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41957fa640 0 client.15144 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 --2- 172.21.15.96:0/1748034333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b662278d30 0x55b662280020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.570+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 >> 172.21.15.96:0/1748034333 conn(0x55b662276b60 msgr2=0x55b6622843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 shutdown_connections 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b9f94740 1 -- 172.21.15.96:0/1748034333 wait complete. 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b9f94740 20 client.15144 trim_cache size 0 max 16384 2024-09-10T22:46:43.578 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24566, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 83.613981538000004, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 94.328008608999994 2024-09-10T22:46:43.579 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 msgr2=0x7f41980b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 0x7f41980b7830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005fd0 tx=0x7f41a0004ab0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 msgr2=0x7f41b01997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005940 tx=0x7f41a40042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 shutdown_connections 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41980b5370 0x7f41980b7830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41b0109170 0x7f41b019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41b01087f0 0x7f41b01997e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 --2- 172.21.15.96:0/1863663543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b0102de0 0x7f41b01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 >> 172.21.15.96:0/1863663543 conn(0x7f41b00fe870 msgr2=0x7f41b0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 shutdown_connections 2024-09-10T22:46:43.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:43.571+0000 7f41b7bbf640 1 -- 172.21.15.96:0/1863663543 wait complete. 2024-09-10T22:46:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:43 smithi096 ceph-mon[30869]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:43 smithi121 ceph-mon[35119]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:43 smithi117 ceph-mon[35081]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:44.282 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T22:46:44.282 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:46:44.294 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:46:44.294 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T22:46:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 -- 172.21.15.96:0/2604272817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 msgr2=0x7f74a4102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 --2- 172.21.15.96:0/2604272817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a4102f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7498009930 tx=0x7f749802a420 comp rx=0 tx=0).stop 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 -- 172.21.15.96:0/2604272817 shutdown_connections 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 --2- 172.21.15.96:0/2604272817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a4109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 --2- 172.21.15.96:0/2604272817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a4102f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 --2- 172.21.15.96:0/2604272817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 0x7f74a4101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:44.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 -- 172.21.15.96:0/2604272817 >> 172.21.15.96:0/2604272817 conn(0x7f74a40fd210 msgr2=0x7f74a40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 -- 172.21.15.96:0/2604272817 shutdown_connections 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.978+0000 7f74a8f36640 1 -- 172.21.15.96:0/2604272817 wait complete. 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.979+0000 7f74a8f36640 1 Processor -- start 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.979+0000 7f74a8f36640 1 -- start start 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.979+0000 7f74a8f36640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 0x7f74a406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410d5a0 con 0x7f74a4102aa0 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410d420 con 0x7f74a4103460 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f74a410d720 con 0x7f74a4101940 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:44.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43896/0 (socket says 172.21.15.96:43896) 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 -- 172.21.15.96:0/3091130653 learned_addr learned my addr 172.21.15.96:0/3091130653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a259c640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 0x7f74a406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a2d9d640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 -- 172.21.15.96:0/3091130653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 msgr2=0x7f74a406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 0x7f74a406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 -- 172.21.15.96:0/3091130653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 msgr2=0x7f74a41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 -- 172.21.15.96:0/3091130653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a41a09f0 con 0x7f74a4102aa0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a2d9d640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a1d9b640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7498005ab0 tx=0x7f7498004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749803f030 con 0x7f74a4102aa0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a41a0c80 con 0x7f74a4102aa0 2024-09-10T22:46:44.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7498003aa0 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.980+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498035070 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.981+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41a11b0 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.981+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a4108cc0 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.983+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f74980035f0 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.984+0000 7f74937fe640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 0x7f748c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.984+0000 7f74a259c640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 0x7f748c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.984+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7498106b90 con 0x7f74a4102aa0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.984+0000 7f74a259c640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 0x7f748c0b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74940058c0 tx=0x7f749400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:44.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:44.987+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74980cd9f0 con 0x7f74a4102aa0 2024-09-10T22:46:45.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.182+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f74a4063590 con 0x7f748c0b5370 2024-09-10T22:46:45.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.183+0000 7f74937fe640 1 -- 172.21.15.96:0/3091130653 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f74a4063590 con 0x7f748c0b5370 2024-09-10T22:46:45.185 INFO:teuthology.orchestra.run.smithi096.stdout:Backend: cephadm 2024-09-10T22:46:45.185 INFO:teuthology.orchestra.run.smithi096.stdout:Available: Yes 2024-09-10T22:46:45.186 INFO:teuthology.orchestra.run.smithi096.stdout:Paused: No 2024-09-10T22:46:45.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 msgr2=0x7f748c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:45.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 0x7f748c0b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74940058c0 tx=0x7f749400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:46:45.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 msgr2=0x7f74a419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:45.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7498005ab0 tx=0x7f7498004270 comp rx=0 tx=0).stop 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 shutdown_connections 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a4103460 0x7f74a41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f748c0b5370 0x7f748c0b7830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74a4102aa0 0x7f74a419db40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 --2- 172.21.15.96:0/3091130653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a4101940 0x7f74a406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:45.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 >> 172.21.15.96:0/3091130653 conn(0x7f74a40fd210 msgr2=0x7f74a4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:45.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 shutdown_connections 2024-09-10T22:46:45.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:45.185+0000 7f74a8f36640 1 -- 172.21.15.96:0/3091130653 wait complete. 2024-09-10T22:46:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:45 smithi096 ceph-mon[30869]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:45 smithi096 ceph-mon[30869]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:45 smithi121 ceph-mon[35119]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:45 smithi121 ceph-mon[35119]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:45 smithi117 ceph-mon[35081]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:45 smithi117 ceph-mon[35081]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:45.897 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T22:46:46.142 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:46.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 -- 172.21.15.96:0/830825977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b18106550 msgr2=0x7f6b18106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:46.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/830825977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b18106550 0x7f6b18106a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009ca0 tx=0x7f6b1002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 -- 172.21.15.96:0/830825977 shutdown_connections 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/830825977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b18106550 0x7f6b18106a00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/830825977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b181039a0 0x7f6b18105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/830825977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b1806c0c0 0x7f6b18103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 -- 172.21.15.96:0/830825977 >> 172.21.15.96:0/830825977 conn(0x7f6b180fd210 msgr2=0x7f6b180ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 -- 172.21.15.96:0/830825977 shutdown_connections 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.600+0000 7f6b1fb54640 1 -- 172.21.15.96:0/830825977 wait complete. 2024-09-10T22:46:46.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 Processor -- start 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 -- start start 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b181039a0 0x7f6b1819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b18106550 0x7f6b181a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18114cd0 con 0x7f6b1806c0c0 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18114b50 con 0x7f6b181039a0 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1fb54640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18114e50 con 0x7f6b18106550 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:46.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48060/0 (socket says 172.21.15.96:48060) 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 -- 172.21.15.96:0/3136644161 learned_addr learned my addr 172.21.15.96:0/3136644161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 -- 172.21.15.96:0/3136644161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b18106550 msgr2=0x7f6b181a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b18106550 0x7f6b181a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 -- 172.21.15.96:0/3136644161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b181039a0 msgr2=0x7f6b1819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d0ef640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b181039a0 0x7f6b1819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b181039a0 0x7f6b1819fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.601+0000 7f6b1d8f0640 1 -- 172.21.15.96:0/3136644161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b181a0f40 con 0x7f6b1806c0c0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b1d8f0640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b1806c0c0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181a1230 con 0x7f6b1806c0c0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b14009c30 con 0x7f6b1806c0c0 2024-09-10T22:46:46.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400f620 con 0x7f6b1806c0c0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.602+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181a7910 con 0x7f6b1806c0c0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.603+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b180fed00 con 0x7f6b1806c0c0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.604+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f6b14010460 con 0x7f6b1806c0c0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.605+0000 7f6b0effd640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 0x7f6b040b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.605+0000 7f6b1d0ef640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 0x7f6b040b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.605+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6b140e5950 con 0x7f6b1806c0c0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.605+0000 7f6b1d0ef640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 0x7f6b040b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b080058c0 tx=0x7f6b08005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.608+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6b140ac7b0 con 0x7f6b1806c0c0 2024-09-10T22:46:46.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.793+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6b18063590 con 0x7f6b040b5370 2024-09-10T22:46:46.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.805+0000 7f6b0effd640 1 -- 172.21.15.96:0/3136644161 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6b18063590 con 0x7f6b040b5370 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.a smithi096 running (100s) 20s ago 100s 17.2M - 19.3.0-4854-g8f219c47 589dd18248e4 27c924faf23d 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.b smithi117 running (97s) 30s ago 97s 26.8M - 19.3.0-4854-g8f219c47 589dd18248e4 9f34090c0e36 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.c smithi121 running (111s) 30s ago 111s 18.5M - 19.3.0-4854-g8f219c47 589dd18248e4 0352284e6c40 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.d smithi096 running (119s) 20s ago 119s 17.4M - 19.3.0-4854-g8f219c47 589dd18248e4 0c1d29f176c5 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.e smithi117 running (117s) 30s ago 116s 18.5M - 19.3.0-4854-g8f219c47 589dd18248e4 ae7683f8f0b4 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.f smithi121 running (114s) 30s ago 114s 17.6M - 19.3.0-4854-g8f219c47 589dd18248e4 3e485c048c2d 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.g smithi096 running (103s) 20s ago 102s 17.3M - 19.3.0-4854-g8f219c47 589dd18248e4 4644660f4c7a 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.h smithi117 running (109s) 30s ago 108s 20.0M - 19.3.0-4854-g8f219c47 589dd18248e4 03196e6d22be 2024-09-10T22:46:46.808 INFO:teuthology.orchestra.run.smithi096.stdout:mds.i smithi121 running (106s) 30s ago 106s 20.0M - 19.3.0-4854-g8f219c47 589dd18248e4 68c6abbe83e7 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mds.j smithi096 running (2m) 20s ago 2m 13.6M - 19.3.0-4854-g8f219c47 589dd18248e4 2c5fc78c04c6 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mds.k smithi117 running (2m) 30s ago 2m 14.6M - 19.3.0-4854-g8f219c47 589dd18248e4 134167cf3d4c 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mds.l smithi121 running (2m) 30s ago 2m 17.3M - 19.3.0-4854-g8f219c47 589dd18248e4 8ca6ea62709a 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.x smithi096 *:9283,8765 running (9m) 20s ago 9m 523M - 19.3.0-4854-g8f219c47 589dd18248e4 2513db03b94e 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.y smithi117 *:8443,8765 running (7m) 30s ago 7m 455M - 19.3.0-4854-g8f219c47 589dd18248e4 f99a087ec9e1 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.z smithi121 *:8443,8765 running (7m) 30s ago 7m 458M - 19.3.0-4854-g8f219c47 589dd18248e4 626facc9c807 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a smithi096 running (9m) 20s ago 9m 52.0M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 d87d899ba613 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b smithi117 running (8m) 30s ago 8m 42.1M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 fa3be3535633 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c smithi121 running (8m) 30s ago 8m 42.1M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 36238d29c8ed 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 smithi096 running (7m) 20s ago 7m 44.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 17414d68ceef 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 smithi117 running (6m) 30s ago 6m 59.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 eb416a8bea2a 2024-09-10T22:46:46.809 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 smithi121 running (6m) 30s ago 6m 45.2M 4271M 19.3.0-4854-g8f219c47 589dd18248e4 3b6da4fd8831 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 smithi096 running (5m) 20s ago 5m 56.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 a92205c58d6e 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 smithi117 running (5m) 30s ago 5m 43.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 63c8a705fa28 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 smithi121 running (5m) 30s ago 5m 53.8M 4271M 19.3.0-4854-g8f219c47 589dd18248e4 1a116bef3803 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 smithi096 running (4m) 20s ago 4m 42.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 1bfa2703310c 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 smithi117 running (4m) 30s ago 4m 45.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 5e52a1bcae4c 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 smithi121 running (3m) 30s ago 3m 52.7M 4271M 19.3.0-4854-g8f219c47 589dd18248e4 e577a60d4c79 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 smithi096 running (3m) 20s ago 3m 54.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e06eba0f77f0 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 smithi117 running (2m) 30s ago 2m 45.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 dc5af199426d 2024-09-10T22:46:46.810 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 smithi121 running (2m) 30s ago 2m 40.8M 4271M 19.3.0-4854-g8f219c47 589dd18248e4 abd3d30330c4 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.807+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 msgr2=0x7f6b040b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.807+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 0x7f6b040b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b080058c0 tx=0x7f6b08005c80 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.807+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 msgr2=0x7f6b1819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.807+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d950 tx=0x7f6b1400de10 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.807+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 shutdown_connections 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b181039a0 0x7f6b1819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b040b5370 0x7f6b040b7830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b18106550 0x7f6b181a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 --2- 172.21.15.96:0/3136644161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b1806c0c0 0x7f6b1819f410 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 >> 172.21.15.96:0/3136644161 conn(0x7f6b180fd210 msgr2=0x7f6b18101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 shutdown_connections 2024-09-10T22:46:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:46.808+0000 7f6b1fb54640 1 -- 172.21.15.96:0/3136644161 wait complete. 2024-09-10T22:46:47.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:47 smithi096 ceph-mon[30869]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:47.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:47 smithi096 ceph-mon[30869]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:47.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T22:46:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:47 smithi121 ceph-mon[35119]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:47.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:47 smithi121 ceph-mon[35119]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:47 smithi117 ceph-mon[35081]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:47 smithi117 ceph-mon[35081]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:47.828 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 -- 172.21.15.96:0/852212108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0102ab0 msgr2=0x7f83e0102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 --2- 172.21.15.96:0/852212108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0102ab0 0x7f83e0102f30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009930 tx=0x7f83d402a420 comp rx=0 tx=0).stop 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 -- 172.21.15.96:0/852212108 shutdown_connections 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 --2- 172.21.15.96:0/852212108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0103470 0x7f83e0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 --2- 172.21.15.96:0/852212108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0102ab0 0x7f83e0102f30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 --2- 172.21.15.96:0/852212108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0101950 0x7f83e0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 -- 172.21.15.96:0/852212108 >> 172.21.15.96:0/852212108 conn(0x7f83e00fd200 msgr2=0x7f83e00ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 -- 172.21.15.96:0/852212108 shutdown_connections 2024-09-10T22:46:48.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.343+0000 7f83e63e1640 1 -- 172.21.15.96:0/852212108 wait complete. 2024-09-10T22:46:48.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 Processor -- start 2024-09-10T22:46:48.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 -- start start 2024-09-10T22:46:48.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0101950 0x7f83e006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0102ab0 0x7f83e019ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010d5b0 con 0x7f83e0103470 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010d430 con 0x7f83e0101950 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e63e1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010d730 con 0x7f83e0102ab0 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48076/0 (socket says 172.21.15.96:48076) 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 -- 172.21.15.96:0/3577325168 learned_addr learned my addr 172.21.15.96:0/3577325168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83dffff640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0101950 0x7f83e006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83df7fe640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0102ab0 0x7f83e019ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 -- 172.21.15.96:0/3577325168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0102ab0 msgr2=0x7f83e019ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0102ab0 0x7f83e019ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 -- 172.21.15.96:0/3577325168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0101950 msgr2=0x7f83e006bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.344+0000 7f83e497e640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0101950 0x7f83e006bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83e497e640 1 -- 172.21.15.96:0/3577325168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e01a2c20 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83e497e640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83d800cad0 tx=0x7f83d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800ad60 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d8005020 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e01a2f10 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800f6b0 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.345+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a3370 con 0x7f83e0103470 2024-09-10T22:46:48.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.346+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e0108c70 con 0x7f83e0103470 2024-09-10T22:46:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.347+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f83d80054f0 con 0x7f83e0103470 2024-09-10T22:46:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.348+0000 7f83dd7fa640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 0x7f83c80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.348+0000 7f83dffff640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 0x7f83c80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.348+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f83d80e5bc0 con 0x7f83e0103470 2024-09-10T22:46:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.348+0000 7f83dffff640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 0x7f83c80b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83d000abe0 tx=0x7f83d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:48.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.352+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f83d8014030 con 0x7f83e0103470 2024-09-10T22:46:48.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.537+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f83e0063590 con 0x7f83c80b5370 2024-09-10T22:46:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.543+0000 7f83dd7fa640 1 -- 172.21.15.96:0/3577325168 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f83e0063590 con 0x7f83c80b5370 2024-09-10T22:46:48.546 INFO:teuthology.orchestra.run.smithi096.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T22:46:48.546 INFO:teuthology.orchestra.run.smithi096.stdout:mds.all 12/12 32s ago 2m smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2024-09-10T22:46:48.546 INFO:teuthology.orchestra.run.smithi096.stdout:mgr 3/3 32s ago 7m smithi096=x;smithi117=y;smithi121=z;count:3 2024-09-10T22:46:48.546 INFO:teuthology.orchestra.run.smithi096.stdout:mon 3/3 32s ago 8m smithi096:172.21.15.96=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2024-09-10T22:46:48.546 INFO:teuthology.orchestra.run.smithi096.stdout:osd 12 32s ago - 2024-09-10T22:46:48.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.545+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 msgr2=0x7f83c80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:48.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.545+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 0x7f83c80b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83d000abe0 tx=0x7f83d000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.545+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 msgr2=0x7f83e01a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:48.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.545+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83d800cad0 tx=0x7f83d800ced0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 shutdown_connections 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f83c80b5370 0x7f83c80b7830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0103470 0x7f83e01a24a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e0102ab0 0x7f83e019ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 --2- 172.21.15.96:0/3577325168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e0101950 0x7f83e006bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 >> 172.21.15.96:0/3577325168 conn(0x7f83e00fd200 msgr2=0x7f83e0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 shutdown_connections 2024-09-10T22:46:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:48.546+0000 7f83e63e1640 1 -- 172.21.15.96:0/3577325168 wait complete. 2024-09-10T22:46:49.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T22:46:49.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:49 smithi096 ceph-mon[30869]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:49.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:49 smithi096 ceph-mon[30869]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:49.513 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:49 smithi121 ceph-mon[35119]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:49.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:49 smithi121 ceph-mon[35119]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:49 smithi117 ceph-mon[35081]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:49 smithi117 ceph-mon[35081]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:50.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 -- 172.21.15.96:0/2820727474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770102a50 msgr2=0x7fd770102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:50.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 --2- 172.21.15.96:0/2820727474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770102a50 0x7fd770102e50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd760009940 tx=0x7fd76002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:50.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 -- 172.21.15.96:0/2820727474 shutdown_connections 2024-09-10T22:46:50.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 --2- 172.21.15.96:0/2820727474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770103bb0 0x7fd770104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 --2- 172.21.15.96:0/2820727474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 0x7fd77010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 --2- 172.21.15.96:0/2820727474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770102a50 0x7fd770102e50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 -- 172.21.15.96:0/2820727474 >> 172.21.15.96:0/2820727474 conn(0x7fd7700fe320 msgr2=0x7fd770100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 -- 172.21.15.96:0/2820727474 shutdown_connections 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.991+0000 7fd774dc1640 1 -- 172.21.15.96:0/2820727474 wait complete. 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 Processor -- start 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 -- start start 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770102a50 0x7fd770198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 0x7fd77019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010e780 con 0x7fd770103bb0 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010e600 con 0x7fd770102a50 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd774dc1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010e900 con 0x7fd770104570 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48098/0 (socket says 172.21.15.96:48098) 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76dd9b640 1 -- 172.21.15.96:0/1131541260 learned_addr learned my addr 172.21.15.96:0/1131541260 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76ed9d640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 0x7fd77019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76e59c640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770102a50 0x7fd770198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.992+0000 7fd76dd9b640 1 -- 172.21.15.96:0/1131541260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 msgr2=0x7fd77019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd76dd9b640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 0x7fd77019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd76dd9b640 1 -- 172.21.15.96:0/1131541260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770102a50 msgr2=0x7fd770198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd76dd9b640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770102a50 0x7fd770198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd76dd9b640 1 -- 172.21.15.96:0/1131541260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77019a9a0 con 0x7fd770103bb0 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd76dd9b640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764007c70 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77019ac90 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764007e10 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764018820 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.993+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701a1370 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.994+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7701a1540 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.995+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd76400f450 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.996+0000 7fd75f7fe640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 0x7fd7580b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.997+0000 7fd76e59c640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 0x7fd7580b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.997+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd7640e61e0 con 0x7fd770103bb0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:49.997+0000 7fd76e59c640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 0x7fd7580b7830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd760005bc0 tx=0x7fd760009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:50.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.000+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd764014030 con 0x7fd770103bb0 2024-09-10T22:46:50.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.181+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd7701055d0 con 0x7fd7580b5370 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.183+0000 7fd75f7fe640 1 -- 172.21.15.96:0/1131541260 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd7701055d0 con 0x7fd7580b5370 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:HOST ADDR LABELS STATUS 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 172.21.15.96 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:smithi117 172.21.15.117 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:smithi121 172.21.15.121 2024-09-10T22:46:50.185 INFO:teuthology.orchestra.run.smithi096.stdout:3 hosts in cluster 2024-09-10T22:46:50.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 msgr2=0x7fd7580b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:50.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 0x7fd7580b7830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd760005bc0 tx=0x7fd760009500 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 msgr2=0x7fd7701994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cde0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 shutdown_connections 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7580b5370 0x7fd7580b7830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd770104570 0x7fd77019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770103bb0 0x7fd7701994c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 --2- 172.21.15.96:0/1131541260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd770102a50 0x7fd770198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 >> 172.21.15.96:0/1131541260 conn(0x7fd7700fe320 msgr2=0x7fd770104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 shutdown_connections 2024-09-10T22:46:50.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:50.185+0000 7fd774dc1640 1 -- 172.21.15.96:0/1131541260 wait complete. 2024-09-10T22:46:50.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:50 smithi096 ceph-mon[30869]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:50.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:50 smithi121 ceph-mon[35119]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:50.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:50 smithi117 ceph-mon[35081]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:51.058 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T22:46:51.303 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:51.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:51 smithi096 ceph-mon[30869]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:51 smithi121 ceph-mon[35119]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:51.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 -- 172.21.15.96:0/2711966396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 msgr2=0x7f5600104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:51.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 --2- 172.21.15.96:0/2711966396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f5600104010 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009930 tx=0x7f55f002a420 comp rx=0 tx=0).stop 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 -- 172.21.15.96:0/2711966396 shutdown_connections 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 --2- 172.21.15.96:0/2711966396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600104550 0x7f560010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 --2- 172.21.15.96:0/2711966396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f5600104010 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 --2- 172.21.15.96:0/2711966396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600102a30 0x7f5600102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 -- 172.21.15.96:0/2711966396 >> 172.21.15.96:0/2711966396 conn(0x7f56000fe320 msgr2=0x7f5600100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 -- 172.21.15.96:0/2711966396 shutdown_connections 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.748+0000 7f5605d76640 1 -- 172.21.15.96:0/2711966396 wait complete. 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 Processor -- start 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 -- start start 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600102a30 0x7f5600198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:51.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600104550 0x7f560019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e710 con 0x7f5600103b90 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e590 con 0x7f5600104550 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f5605d76640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f560010e890 con 0x7f5600102a30 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f55feffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f55feffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48114/0 (socket says 172.21.15.96:48114) 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.749+0000 7f55feffd640 1 -- 172.21.15.96:0/1832251439 learned_addr learned my addr 172.21.15.96:0/1832251439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55ff7fe640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600102a30 0x7f5600198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55fffff640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600104550 0x7f560019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 -- 172.21.15.96:0/1832251439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600102a30 msgr2=0x7f5600198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600102a30 0x7f5600198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 -- 172.21.15.96:0/1832251439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600104550 msgr2=0x7f560019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:51.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600104550 0x7f560019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 -- 172.21.15.96:0/1832251439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560019a9c0 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55feffd640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005ec0 tx=0x7f55f0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003f030 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f0003cb0 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560019ac50 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0035070 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.750+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a11c0 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.751+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600109d50 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.753+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f55f003bc70 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.753+0000 7f55fcff9640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 0x7f55e80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.753+0000 7f55ff7fe640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 0x7f55e80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.754+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55f0106a90 con 0x7f5600103b90 2024-09-10T22:46:51.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.754+0000 7f55ff7fe640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 0x7f55e80b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009990 tx=0x7f55f4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:51.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.756+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f55f00cd870 con 0x7f5600103b90 2024-09-10T22:46:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:51 smithi117 ceph-mon[35081]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:51.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.937+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5600063590 con 0x7f55e80b5370 2024-09-10T22:46:51.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.939+0000 7f55fcff9640 1 -- 172.21.15.96:0/1832251439 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5600063590 con 0x7f55e80b5370 2024-09-10T22:46:51.942 INFO:teuthology.orchestra.run.smithi096.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T22:46:51.942 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T22:46:51.942 INFO:teuthology.orchestra.run.smithi096.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T22:46:51.942 INFO:teuthology.orchestra.run.smithi096.stdout:smithi121 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZF400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T22:46:51.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 msgr2=0x7f55e80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 0x7f55e80b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009990 tx=0x7f55f4009110 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 msgr2=0x7f56001994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55f0005ec0 tx=0x7f55f0004360 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 shutdown_connections 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5600104550 0x7f560019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55e80b5370 0x7f55e80b7830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5600103b90 0x7f56001994e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 --2- 172.21.15.96:0/1832251439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5600102a30 0x7f5600198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 >> 172.21.15.96:0/1832251439 conn(0x7f56000fe320 msgr2=0x7f5600106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 shutdown_connections 2024-09-10T22:46:51.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:51.942+0000 7f5605d76640 1 -- 172.21.15.96:0/1832251439 wait complete. 2024-09-10T22:46:52.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:52 smithi096 ceph-mon[30869]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:52.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:52 smithi121 ceph-mon[35119]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:52.764 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:46:52.773 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:46:52.773 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T22:46:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:52 smithi117 ceph-mon[35081]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:46:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- 172.21.15.96:0/3194457650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4105c30 msgr2=0x7f8dd4108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- 172.21.15.96:0/3194457650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4105c30 0x7f8dd4108020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009900 tx=0x7f8dc002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- 172.21.15.96:0/3194457650 shutdown_connections 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- 172.21.15.96:0/3194457650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd4108760 0x7f8dd410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- 172.21.15.96:0/3194457650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4105c30 0x7f8dd4108020 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- 172.21.15.96:0/3194457650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd406c000 0x7f8dd41056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- 172.21.15.96:0/3194457650 >> 172.21.15.96:0/3194457650 conn(0x7f8dd40fd200 msgr2=0x7f8dd40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- 172.21.15.96:0/3194457650 shutdown_connections 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- 172.21.15.96:0/3194457650 wait complete. 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 Processor -- start 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- start start 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd406c000 0x7f8dd419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd4105c30 0x7f8dd419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd410e790 con 0x7f8dd4108760 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd410e610 con 0x7f8dd406c000 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.517+0000 7f8ddacac640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd410e910 con 0x7f8dd4105c30 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48132/0 (socket says 172.21.15.96:48132) 2024-09-10T22:46:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 -- 172.21.15.96:0/2284042167 learned_addr learned my addr 172.21.15.96:0/2284042167 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd3fff640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd4105c30 0x7f8dd419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd8a48640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd406c000 0x7f8dd419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 -- 172.21.15.96:0/2284042167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd4105c30 msgr2=0x7f8dd419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd4105c30 0x7f8dd419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 -- 172.21.15.96:0/2284042167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd406c000 msgr2=0x7f8dd419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd406c000 0x7f8dd419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 -- 172.21.15.96:0/2284042167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd419cc60 con 0x7f8dd4108760 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd9249640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b970 tx=0x7f8dcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc019070 con 0x7f8dd4108760 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dcc014440 con 0x7f8dd4108760 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd419cf50 con 0x7f8dd4108760 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0075d0 con 0x7f8dd4108760 2024-09-10T22:46:53.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.518+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd41a3630 con 0x7f8dd4108760 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.519+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd41a3860 con 0x7f8dd4108760 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.521+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8dcc00cb40 con 0x7f8dd4108760 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.521+0000 7f8dd1ffb640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 0x7f8dc40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.521+0000 7f8dd8a48640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 0x7f8dc40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.521+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8dcc0e53a0 con 0x7f8dd4108760 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.522+0000 7f8dd8a48640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 0x7f8dc40b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009890 tx=0x7f8dc8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:53.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.525+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8dcc0abbb0 con 0x7f8dd4108760 2024-09-10T22:46:53.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.736+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8dd41a3a30 con 0x7f8dd4108760 2024-09-10T22:46:53.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:53 smithi096 ceph-mon[30869]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:53.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.736+0000 7f8dd1ffb640 1 -- 172.21.15.96:0/2284042167 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3132 (secure 0 0 0) 0x7f8dcc0b05e0 con 0x7f8dd4108760 2024-09-10T22:46:53.738 INFO:teuthology.orchestra.run.smithi096.stdout:e27 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:btime 2024-09-10T22:45:31:047482+0000 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:legacy client fscid: 1 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem 'cephfs' (1) 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:fs_name cephfs 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 27 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-10T22:45:19.551704+0000 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-09-10T22:45:31.047462+0000 2024-09-10T22:46:53.739 INFO:teuthology.orchestra.run.smithi096.stdout:tableserver 0 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:root 0 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:session_timeout 300 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:session_autoclose 300 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:max_file_size 1099511627776 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:max_xattr_size 65536 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:required_client_features {} 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure 0 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure_osd_epoch 0 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:max_mds 5 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:in 0,1,2,3,4 2024-09-10T22:46:53.740 INFO:teuthology.orchestra.run.smithi096.stdout:up {0=24566,1=24550,2=24542,3=24529,4=24527} 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:failed 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:damaged 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:stopped 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:data_pools [3] 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:metadata_pool 2 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:inline_data disabled 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:balancer 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:bal_rank_mask -1 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:standby_count_wanted 0 2024-09-10T22:46:53.741 INFO:teuthology.orchestra.run.smithi096.stdout:qdb_cluster leader: 24566 members: 24550,24542,24529,24566,24527 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b{0:24566} state up:active seq 4 addr [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.f{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.121:6834/1599221610,v1:172.21.15.121:6835/1599221610] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.i{1:24550} state up:active seq 8 addr [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.l{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.121:6832/2699941517,v1:172.21.15.121:6833/2699941517] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.h{2:24542} state up:active seq 9 addr [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a{2:14706} state up:standby-replay seq 1 addr [v2:172.21.15.96:6840/2618224226,v1:172.21.15.96:6841/2618224226] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c{3:24529} state up:active seq 10 addr [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.g{3:14700} state up:standby-replay seq 1 addr [v2:172.21.15.96:6838/2726589142,v1:172.21.15.96:6839/2726589142] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.e{4:24527} state up:active seq 12 addr [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d{4:14649} state up:standby-replay seq 1 addr [v2:172.21.15.96:6836/3503446146,v1:172.21.15.96:6837/3503446146] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:53.742 INFO:teuthology.orchestra.run.smithi096.stdout:Standby daemons: 2024-09-10T22:46:53.743 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:53.743 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.k{-1:14625} state up:standby seq 1 addr [v2:172.21.15.117:6832/317768324,v1:172.21.15.117:6833/317768324] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.743 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.j{-1:14637} state up:standby seq 1 addr [v2:172.21.15.96:6834/3071875945,v1:172.21.15.96:6835/3071875945] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:46:53.743 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:53.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.738+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 msgr2=0x7f8dc40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.738+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 0x7f8dc40b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009890 tx=0x7f8dc8008040 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 msgr2=0x7f8dd419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b970 tx=0x7f8dcc00be30 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 shutdown_connections 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8dc40b53f0 0x7f8dc40b78b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd4108760 0x7f8dd419c3c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dd4105c30 0x7f8dd419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 --2- 172.21.15.96:0/2284042167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dd406c000 0x7f8dd419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 >> 172.21.15.96:0/2284042167 conn(0x7f8dd40fd200 msgr2=0x7f8dd40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 shutdown_connections 2024-09-10T22:46:53.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:53.739+0000 7f8ddacac640 1 -- 172.21.15.96:0/2284042167 wait complete. 2024-09-10T22:46:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:53 smithi121 ceph-mon[35119]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:53 smithi117 ceph-mon[35081]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:46:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:54 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2284042167' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:46:54.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T22:46:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:54 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2284042167' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:46:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:54 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2284042167' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:46:54.817 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:55.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 -- 172.21.15.96:0/4050964518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 msgr2=0x7fb77010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:55.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 --2- 172.21.15.96:0/4050964518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77010aef0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb768009cd0 tx=0x7fb76802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 -- 172.21.15.96:0/4050964518 shutdown_connections 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 --2- 172.21.15.96:0/4050964518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 0x7fb770104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 --2- 172.21.15.96:0/4050964518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77010aef0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 --2- 172.21.15.96:0/4050964518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 0x7fb770102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 -- 172.21.15.96:0/4050964518 >> 172.21.15.96:0/4050964518 conn(0x7fb7700fe320 msgr2=0x7fb770100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 -- 172.21.15.96:0/4050964518 shutdown_connections 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.300+0000 7fb776460640 1 -- 172.21.15.96:0/4050964518 wait complete. 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 Processor -- start 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 -- start start 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 0x7fb770198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 0x7fb7701994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:55.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e730 con 0x7fb770104570 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e5b0 con 0x7fb770103bb0 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb776460640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e8b0 con 0x7fb770102a50 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48156/0 (socket says 172.21.15.96:48156) 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 -- 172.21.15.96:0/3868941315 learned_addr learned my addr 172.21.15.96:0/3868941315 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb76f7fe640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 0x7fb7701994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb76ffff640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 0x7fb770198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 -- 172.21.15.96:0/3868941315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 msgr2=0x7fb770198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 0x7fb770198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 -- 172.21.15.96:0/3868941315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 msgr2=0x7fb7701994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.301+0000 7fb7749fd640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 0x7fb7701994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb7749fd640 1 -- 172.21.15.96:0/3868941315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77019a9a0 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb7749fd640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7680317d0 tx=0x7fb768003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768004450 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7680045f0 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77019ac30 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768045670 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.302+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a1370 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.303+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770109d70 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.304+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb76804d070 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.305+0000 7fb76d7fa640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 0x7fb7580b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.305+0000 7fb76ffff640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 0x7fb7580b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.305+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb76802e030 con 0x7fb770104570 2024-09-10T22:46:55.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.305+0000 7fb76ffff640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 0x7fb7580b78b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb76000a810 tx=0x7fb760008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:55.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.308+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb76804a060 con 0x7fb770104570 2024-09-10T22:46:55.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.491+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb77002e5c0 con 0x7fb770104570 2024-09-10T22:46:55.493 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:55 smithi096 ceph-mon[30869]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:55.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.492+0000 7fb76d7fa640 1 -- 172.21.15.96:0/3868941315 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5231 (secure 0 0 0) 0x7fb7680cdd60 con 0x7fb770104570 2024-09-10T22:46:55.493 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 77 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-10T22:37:12.788608+0000 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-09-10T22:45:58.291718+0000 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 26 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-09-10T22:46:55.494 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client luminous 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 12 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 8 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6802/2443850477,v1:172.21.15.96:6803/2443850477] [v2:172.21.15.96:6804/2443850477,v1:172.21.15.96:6805/2443850477] exists,up a28ec19e-b4a4-4a13-8674-1c7d77449432 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/3283899162,v1:172.21.15.117:6801/3283899162] [v2:172.21.15.117:6802/3283899162,v1:172.21.15.117:6803/3283899162] exists,up afe84389-be35-42dc-bd7b-23672bffb969 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 18 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/203762338,v1:172.21.15.121:6801/203762338] [v2:172.21.15.121:6802/203762338,v1:172.21.15.121:6803/203762338] exists,up 7d696b9b-398f-4457-a05b-c14dbb9a42f5 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 24 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6810/2944383556,v1:172.21.15.96:6811/2944383556] [v2:172.21.15.96:6812/2944383556,v1:172.21.15.96:6813/2944383556] exists,up 0e749818-335f-4852-b188-2898bf14e7ab 2024-09-10T22:46:55.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 30 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/2758163149,v1:172.21.15.117:6809/2758163149] [v2:172.21.15.117:6810/2758163149,v1:172.21.15.117:6811/2758163149] exists,up 43f285c1-14f0-4b31-a928-1705584f5f0b 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 35 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/1183691363,v1:172.21.15.121:6809/1183691363] [v2:172.21.15.121:6810/1183691363,v1:172.21.15.121:6811/1183691363] exists,up 5520a789-19f0-4775-a38f-0d7707878cdf 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 41 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6818/201788565,v1:172.21.15.96:6819/201788565] [v2:172.21.15.96:6820/201788565,v1:172.21.15.96:6821/201788565] exists,up 372e13d3-06be-488a-aac3-1bae70af8eb9 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 46 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/918709723,v1:172.21.15.117:6817/918709723] [v2:172.21.15.117:6818/918709723,v1:172.21.15.117:6819/918709723] exists,up 783ef350-a8c5-4e79-b761-13da9d17c03b 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 up in weight 1 up_from 52 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/4251994917,v1:172.21.15.121:6817/4251994917] [v2:172.21.15.121:6818/4251994917,v1:172.21.15.121:6819/4251994917] exists,up 55e87abf-88f6-47fe-9297-9c71f8997270 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 up in weight 1 up_from 58 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6826/1287082888,v1:172.21.15.96:6827/1287082888] [v2:172.21.15.96:6828/1287082888,v1:172.21.15.96:6829/1287082888] exists,up b757f430-ca41-4632-8891-d74c23873b69 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 up in weight 1 up_from 63 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/3915077352,v1:172.21.15.117:6825/3915077352] [v2:172.21.15.117:6826/3915077352,v1:172.21.15.117:6827/3915077352] exists,up e49921d8-dcf9-427a-8a29-57f0450576f4 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 up in weight 1 up_from 69 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/2582866969,v1:172.21.15.121:6825/2582866969] [v2:172.21.15.121:6826/2582866969,v1:172.21.15.121:6827/2582866969] exists,up a5084606-c0d3-4731-8e88-17f0ef95d91d 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.4 [2,11] 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.8 [2,11] 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1a [8,4] 2024-09-10T22:46:55.496 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.0 [5,9] 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.24 [4,9] 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.36 [4,9] 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/827103951 expires 2024-09-11T22:37:57.209878+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1292035523 expires 2024-09-11T22:37:57.209878+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/1530138287 expires 2024-09-11T22:37:57.209878+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/1530138287 expires 2024-09-11T22:37:57.209878+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/3039659416 expires 2024-09-11T22:37:34.381409+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/3039659416 expires 2024-09-11T22:37:34.381409+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/3734789223 expires 2024-09-11T22:37:34.381409+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1542157874 expires 2024-09-11T22:37:34.381409+0000 2024-09-10T22:46:55.497 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/217725041 expires 2024-09-11T22:37:57.209878+0000 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1680117294 expires 2024-09-11T22:37:34.381409+0000 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 msgr2=0x7fb7580b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 0x7fb7580b78b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb76000a810 tx=0x7fb760008040 comp rx=0 tx=0).stop 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 msgr2=0x7fb77019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7680317d0 tx=0x7fb768003ce0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 shutdown_connections 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb770103bb0 0x7fb7701994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb7580b53f0 0x7fb7580b78b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb770104570 0x7fb77019a100 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 --2- 172.21.15.96:0/3868941315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb770102a50 0x7fb770198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 >> 172.21.15.96:0/3868941315 conn(0x7fb7700fe320 msgr2=0x7fb770106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 shutdown_connections 2024-09-10T22:46:55.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:55.494+0000 7fb776460640 1 -- 172.21.15.96:0/3868941315 wait complete. 2024-09-10T22:46:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:55 smithi121 ceph-mon[35119]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:55 smithi117 ceph-mon[35081]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:46:56.162 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T22:46:56.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:46:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:56.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:56 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3868941315' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:46:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:56 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3868941315' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:46:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:56 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3868941315' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 -- 172.21.15.96:0/2159767304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514102dc0 msgr2=0x7f75141031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 --2- 172.21.15.96:0/2159767304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514102dc0 0x7f75141031a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7510009900 tx=0x7f751002a410 comp rx=0 tx=0).stop 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 -- 172.21.15.96:0/2159767304 shutdown_connections 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 --2- 172.21.15.96:0/2159767304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75141087d0 0x7f7514108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 --2- 172.21.15.96:0/2159767304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7514109150 0x7f751410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 --2- 172.21.15.96:0/2159767304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514102dc0 0x7f75141031a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 -- 172.21.15.96:0/2159767304 >> 172.21.15.96:0/2159767304 conn(0x7f75140fe830 msgr2=0x7f7514100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:56.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 -- 172.21.15.96:0/2159767304 shutdown_connections 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.908+0000 7f751c296640 1 -- 172.21.15.96:0/2159767304 wait complete. 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 Processor -- start 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 -- start start 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 0x7f75141993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75141087d0 0x7f7514199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7514112ea0 con 0x7f7514109150 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7514112d20 con 0x7f7514102dc0 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751c296640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7514113020 con 0x7f75141087d0 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a833640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a833640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59884/0 (socket says 172.21.15.96:59884) 2024-09-10T22:46:56.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a833640 1 -- 172.21.15.96:0/3840004717 learned_addr learned my addr 172.21.15.96:0/3840004717 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a032640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 0x7f75141993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a833640 1 -- 172.21.15.96:0/3840004717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75141087d0 msgr2=0x7f7514199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.909+0000 7f751a833640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75141087d0 0x7f7514199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751a833640 1 -- 172.21.15.96:0/3840004717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 msgr2=0x7f75141993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751a833640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 0x7f75141993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751a833640 1 -- 172.21.15.96:0/3840004717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751419f4a0 con 0x7f7514109150 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751a032640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 0x7f75141993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751a833640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b970 tx=0x7f750c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c019070 con 0x7f7514109150 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c014440 con 0x7f7514109150 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751419f790 con 0x7f7514109150 2024-09-10T22:46:56.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0075d0 con 0x7f7514109150 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.910+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a5ed0 con 0x7f7514109150 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.911+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7514103220 con 0x7f7514109150 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.912+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f750c00cb40 con 0x7f7514109150 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.913+0000 7f750b7fe640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 0x7f75000b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.913+0000 7f751a032640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 0x7f75000b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.913+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f750c0e5960 con 0x7f7514109150 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.913+0000 7f751a032640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 0x7f75000b78b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7510005ab0 tx=0x7f7510000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:56.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:56.916+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f750c0ac6c0 con 0x7f7514109150 2024-09-10T22:46:57.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.129+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f751402e5c0 con 0x7f7514109150 2024-09-10T22:46:57.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.130+0000 7f750b7fe640 1 -- 172.21.15.96:0/3840004717 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f750c0b3e20 con 0x7f7514109150 2024-09-10T22:46:57.132 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:57.132 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:46:57.132 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 msgr2=0x7f75000b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 0x7f75000b78b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7510005ab0 tx=0x7f7510000bc0 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 msgr2=0x7f751419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b970 tx=0x7f750c00be30 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 shutdown_connections 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75000b53f0 0x7f75000b78b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7514109150 0x7f751419ecd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75141087d0 0x7f7514199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 --2- 172.21.15.96:0/3840004717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7514102dc0 0x7f75141993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:57.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 >> 172.21.15.96:0/3840004717 conn(0x7f75140fe830 msgr2=0x7f7514100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:57.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 shutdown_connections 2024-09-10T22:46:57.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:57.132+0000 7f751c296640 1 -- 172.21.15.96:0/3840004717 wait complete. 2024-09-10T22:46:57.562 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:57 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3840004717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:57.562 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:57 smithi096 ceph-mon[30869]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:57 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3840004717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:57.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:57 smithi121 ceph-mon[35119]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:57 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3840004717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:57 smithi117 ceph-mon[35081]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:46:57.845 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:46:58.092 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:46:58.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 -- 172.21.15.96:0/1648407845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 msgr2=0x7f312c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:58.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 --2- 172.21.15.96:0/1648407845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c102070 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f311c009900 tx=0x7f311c02a410 comp rx=0 tx=0).stop 2024-09-10T22:46:58.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 -- 172.21.15.96:0/1648407845 shutdown_connections 2024-09-10T22:46:58.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 --2- 172.21.15.96:0/1648407845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c1076a0 0x7f312c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 --2- 172.21.15.96:0/1648407845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c108020 0x7f312c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 --2- 172.21.15.96:0/1648407845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c102070 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 -- 172.21.15.96:0/1648407845 >> 172.21.15.96:0/1648407845 conn(0x7f312c0fd700 msgr2=0x7f312c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 -- 172.21.15.96:0/1648407845 shutdown_connections 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 -- 172.21.15.96:0/1648407845 wait complete. 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.606+0000 7f31333c4640 1 Processor -- start 2024-09-10T22:46:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 -- start start 2024-09-10T22:46:58.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:58.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c1076a0 0x7f312c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:58.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 0x7f312c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:58.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f312c111ce0 con 0x7f312c101c90 2024-09-10T22:46:58.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f312c111b60 con 0x7f312c108020 2024-09-10T22:46:58.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f31333c4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f312c111e60 con 0x7f312c1076a0 2024-09-10T22:46:58.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:58.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59888/0 (socket says 172.21.15.96:59888) 2024-09-10T22:46:58.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 -- 172.21.15.96:0/2162894428 learned_addr learned my addr 172.21.15.96:0/2162894428 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:46:58.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f313095f640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c1076a0 0x7f312c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:58.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131961640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 0x7f312c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:58.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 -- 172.21.15.96:0/2162894428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c1076a0 msgr2=0x7f312c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:58.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c1076a0 0x7f312c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 -- 172.21.15.96:0/2162894428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 msgr2=0x7f312c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:58.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 0x7f312c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 -- 172.21.15.96:0/2162894428 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1a5470 con 0x7f312c101c90 2024-09-10T22:46:58.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131961640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 0x7f312c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:46:58.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.607+0000 7f3131160640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f311c006010 tx=0x7f311c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:58.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.608+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03f030 con 0x7f312c101c90 2024-09-10T22:46:58.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.608+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c003800 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.608+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1a5700 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.608+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c035070 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.608+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1a5cb0 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.609+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c102070 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.610+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f311c031b10 con 0x7f312c101c90 2024-09-10T22:46:58.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.611+0000 7f31227fc640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 0x7f31180b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:46:58.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.611+0000 7f313095f640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 0x7f31180b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:46:58.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.611+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f311c1067d0 con 0x7f312c101c90 2024-09-10T22:46:58.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.612+0000 7f313095f640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 0x7f31180b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f312c06b6b0 tx=0x7f312800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:46:58.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.614+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f311c0cd5b0 con 0x7f312c101c90 2024-09-10T22:46:58.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.824+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f312c06aec0 con 0x7f312c101c90 2024-09-10T22:46:58.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.825+0000 7f31227fc640 1 -- 172.21.15.96:0/2162894428 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11979 (secure 0 0 0) 0x7f312c06aec0 con 0x7f312c101c90 2024-09-10T22:46:58.827 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:46:58.827 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"btime":"2024-09-10T22:45:31:047482+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:45:31.047462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24550,24542,24529,24566,24527]},"id":1}]} 2024-09-10T22:46:58.827 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-10T22:46:58.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 msgr2=0x7f31180b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 0x7f31180b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f312c06b6b0 tx=0x7f312800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 msgr2=0x7f312c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f311c006010 tx=0x7f311c0042f0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 shutdown_connections 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312c108020 0x7f312c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f31180b53f0 0x7f31180b78b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f312c1076a0 0x7f312c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 --2- 172.21.15.96:0/2162894428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f312c101c90 0x7f312c19a090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 >> 172.21.15.96:0/2162894428 conn(0x7f312c0fd700 msgr2=0x7f312c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 shutdown_connections 2024-09-10T22:46:58.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:46:58.828+0000 7f31333c4640 1 -- 172.21.15.96:0/2162894428 wait complete. 2024-09-10T22:46:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:46:59 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2162894428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:59.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:46:59 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2162894428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:59.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:46:59 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2162894428' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:46:59.639 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:46:59.890 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:00.302 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:00 smithi096 ceph-mon[30869]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- 172.21.15.96:0/2188905855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c108020 msgr2=0x7f632c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 --2- 172.21.15.96:0/2188905855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c108020 0x7f632c10e820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6324009ca0 tx=0x7f632402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- 172.21.15.96:0/2188905855 shutdown_connections 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 --2- 172.21.15.96:0/2188905855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f632c1076a0 0x7f632c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 --2- 172.21.15.96:0/2188905855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c108020 0x7f632c10e820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 --2- 172.21.15.96:0/2188905855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f632c101c90 0x7f632c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- 172.21.15.96:0/2188905855 >> 172.21.15.96:0/2188905855 conn(0x7f632c0fd700 msgr2=0x7f632c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- 172.21.15.96:0/2188905855 shutdown_connections 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- 172.21.15.96:0/2188905855 wait complete. 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 Processor -- start 2024-09-10T22:47:00.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.380+0000 7f6331b66640 1 -- start start 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f632c1076a0 0x7f632c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f632c108020 0x7f632c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f632c111ce0 con 0x7f632c101c90 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f632c111b60 con 0x7f632c108020 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f6331b66640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f632c111e60 con 0x7f632c1076a0 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59894/0 (socket says 172.21.15.96:59894) 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 -- 172.21.15.96:0/3339452434 learned_addr learned my addr 172.21.15.96:0/3339452434 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 -- 172.21.15.96:0/3339452434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f632c1076a0 msgr2=0x7f632c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f632c1076a0 0x7f632c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 -- 172.21.15.96:0/3339452434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f632c108020 msgr2=0x7f632c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f632c108020 0x7f632c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 -- 172.21.15.96:0/3339452434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c19e240 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.381+0000 7f632b7fe640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c890 tx=0x7f631c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.382+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c004220 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.382+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c19e530 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.382+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c0043c0 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.382+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c004a00 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.382+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c1a4c70 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.383+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c102070 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.384+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f631c0079b0 con 0x7f632c101c90 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.384+0000 7f63297fa640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 0x7f63180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:00.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.385+0000 7f6322ffd640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 0x7f63180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:00.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.385+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f631c0e5980 con 0x7f632c101c90 2024-09-10T22:47:00.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.385+0000 7f6322ffd640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 0x7f63180b77a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f632c103eb0 tx=0x7f6314006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:00.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.388+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f631c014030 con 0x7f632c101c90 2024-09-10T22:47:00.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:00 smithi121 ceph-mon[35119]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:00.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:00 smithi117 ceph-mon[35081]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:00.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.564+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f632c1992c0 con 0x7f632c101c90 2024-09-10T22:47:00.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.565+0000 7f63297fa640 1 -- 172.21.15.96:0/3339452434 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7f631c0ac8e0 con 0x7f632c101c90 2024-09-10T22:47:00.567 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:00.567 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 msgr2=0x7f63180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 0x7f63180b77a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f632c103eb0 tx=0x7f6314006c90 comp rx=0 tx=0).stop 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 msgr2=0x7f632c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c890 tx=0x7f631c00cd50 comp rx=0 tx=0).stop 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 shutdown_connections 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f632c108020 0x7f632c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f63180b52e0 0x7f63180b77a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f632c1076a0 0x7f632c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 --2- 172.21.15.96:0/3339452434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f632c101c90 0x7f632c198190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:00.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 >> 172.21.15.96:0/3339452434 conn(0x7f632c0fd700 msgr2=0x7f632c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:00.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.567+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 shutdown_connections 2024-09-10T22:47:00.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:00.568+0000 7f6331b66640 1 -- 172.21.15.96:0/3339452434 wait complete. 2024-09-10T22:47:01.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:01 smithi096 ceph-mon[30869]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:47:01.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:01 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3339452434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:01.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:45:19.551704+0000', 'modified': '2024-09-10T22:45:31.047462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24550, 'mds_2': 24542, 'mds_3': 24529, 'mds_4': 24527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14649': {'gid': 14649, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6837/3503446146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3503446146}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3503446146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14700': {'gid': 14700, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/2726589142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 2726589142}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 2726589142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14706': {'gid': 14706, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/2618224226', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 2618224226}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 2618224226}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6833/2699941517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 2699941517}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 2699941517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/1599221610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1599221610}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 1599221610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.117:6835/2943423694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2943423694}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2943423694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.121:6837/1420997271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1420997271}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1420997271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.117:6837/214150535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 214150535}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 214150535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.121:6839/693851001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 693851001}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 693851001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/4053412961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 4053412961}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 4053412961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24550, 24542, 24529, 24566, 24527]} 2024-09-10T22:47:01.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T22:47:01.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-10T22:47:01.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:01 smithi121 ceph-mon[35119]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:47:01.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:01 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3339452434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:01 smithi117 ceph-mon[35081]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T22:47:01.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:01 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3339452434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:01.661 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3357785401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 msgr2=0x7fecd4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3357785401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4102070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009900 tx=0x7fecd002a410 comp rx=0 tx=0).stop 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3357785401 shutdown_connections 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3357785401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd4108020 0x7fecd410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3357785401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 0x7fecd4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3357785401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4102070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3357785401 >> 172.21.15.96:0/3357785401 conn(0x7fecd40fd700 msgr2=0x7fecd40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3357785401 shutdown_connections 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.256+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3357785401 wait complete. 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 Processor -- start 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 -- start start 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 0x7fecd41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd4108020 0x7fecd419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4111ce0 con 0x7fecd4101c90 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4111b60 con 0x7fecd4108020 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecdbbe1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4111e60 con 0x7fecd41076a0 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59900/0 (socket says 172.21.15.96:59900) 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.257+0000 7fecd997d640 1 -- 172.21.15.96:0/3114504160 learned_addr learned my addr 172.21.15.96:0/3114504160 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 -- 172.21.15.96:0/3114504160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 msgr2=0x7fecd41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd917c640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 0x7fecd41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 0x7fecd41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 -- 172.21.15.96:0/3114504160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd4108020 msgr2=0x7fecd419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd4108020 0x7fecd419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 -- 172.21.15.96:0/3114504160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd419e220 con 0x7fecd4101c90 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecd997d640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fecd0006010 tx=0x7fecd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7feccaffd640 1 -- 172.21.15.96:0/3114504160 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0035050 con 0x7fecd4101c90 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7feccaffd640 1 -- 172.21.15.96:0/3114504160 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd00035f0 con 0x7fecd4101c90 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd419e4b0 con 0x7fecd4101c90 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7feccaffd640 1 -- 172.21.15.96:0/3114504160 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032260 con 0x7fecd4101c90 2024-09-10T22:47:02.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.258+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd406aab0 con 0x7fecd4101c90 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7feccaffd640 1 -- 172.21.15.96:0/3114504160 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fecd0003aa0 con 0x7fecd4101c90 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 Processor -- start 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 -- start start 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be866110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be8749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3be7ddc10 con 0x55b3be866650 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3be7cd040 con 0x55b3be865d30 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecddfb6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b3be7d0e50 con 0x55b3be8725e0 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59916/0 (socket says 172.21.15.96:59916) 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecd997d640 1 -- 172.21.15.96:0/1094168298 learned_addr learned my addr 172.21.15.96:0/1094168298 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:02.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.261+0000 7fecd917c640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be8749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecda17e640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be866110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 -- 172.21.15.96:0/1094168298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 msgr2=0x55b3be8749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be8749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 -- 172.21.15.96:0/1094168298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 msgr2=0x55b3be866110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be866110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 -- 172.21.15.96:0/1094168298 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3be52b540 con 0x55b3be866650 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecda17e640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be866110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7feccaffd640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 0x7fecc00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecda17e640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 0x7fecc00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecd997d640 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecd010bdc0 tx=0x7fecd010beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b612ccab402db407 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7feccaffd640 1 -- 172.21.15.96:0/3114504160 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fecd0107970 con 0x7fecd4101c90 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecbffff640 1 -- 172.21.15.96:0/1094168298 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd00c9fd0 con 0x55b3be866650 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecbffff640 1 -- 172.21.15.96:0/1094168298 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd010f040 con 0x55b3be866650 2024-09-10T22:47:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.262+0000 7fecbffff640 1 -- 172.21.15.96:0/1094168298 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd00cd8a0 con 0x55b3be866650 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- 172.21.15.96:0/1094168298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 msgr2=0x55b3be8720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecda17e640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 0x7fecc00b77a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feccc007d90 tx=0x7feccc007340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecd010bdc0 tx=0x7fecd010beb0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- 172.21.15.96:0/1094168298 shutdown_connections 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be8749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be8720a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 --2- 172.21.15.96:0/1094168298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be866110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- 172.21.15.96:0/1094168298 >> 172.21.15.96:0/1094168298 conn(0x55b3be863b60 msgr2=0x55b3be864d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- 172.21.15.96:0/1094168298 shutdown_connections 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- 172.21.15.96:0/1094168298 wait complete. 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.263+0000 7fecddfb6740 1 -- start start 2024-09-10T22:47:02.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 Processor -- start 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be86f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be870020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3be62e5f0 con 0x55b3be866650 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3be768010 con 0x55b3be865d30 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b3be875050 con 0x55b3be8725e0 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecddfb6740 10 client.0 _ll_register_callbacks cb 0x55b3be7c7340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59932/0 (socket says 172.21.15.96:59932) 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 -- 172.21.15.96:0/3564961288 learned_addr learned my addr 172.21.15.96:0/3564961288 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecda17e640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be870020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd997d640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be86f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 -- 172.21.15.96:0/3564961288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 msgr2=0x55b3be870020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be870020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 -- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 msgr2=0x55b3be86f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be86f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 -- 172.21.15.96:0/3564961288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3be795a00 con 0x55b3be866650 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd997d640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be86f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecd917c640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fecc4010720 tx=0x7fecc4010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5999806b734a08e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.264+0000 7fecbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc400ed40 con 0x55b3be866650 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fecc4005020 con 0x55b3be866650 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd010f040 con 0x55b3be866650 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecc4005260 con 0x55b3be866650 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:02.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3be880b60 con 0x55b3be866650 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd010f040 con 0x55b3be866650 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fecc4018450 con 0x55b3be866650 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 10 client.15216 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 20 client.15216 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be883270 0x55b3be8846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 4 client.15216 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.265+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3be7dfb60 con 0x55b3be883270 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.266+0000 7fecd997d640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be883270 0x55b3be8846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.266+0000 7fecd997d640 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be883270 0x55b3be8846e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=627d4903949592c8 server_cookie=595e6ba6a662f58b in_seq=0 out_seq=0 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.266+0000 7fecbd7fa640 10 client.15216 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3564961288 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3be7dfb60 con 0x55b3be883270 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecbd7fa640 10 client.15216 handle_command_reply: tid=0 2024-09-10T22:47:02.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 client.15216 shutdown 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 msgr2=0x55b3be86fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fecc4010720 tx=0x7fecc4010be0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 shutdown_connections 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be883270 0x55b3be8846e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be870020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be86fae0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecbd7fa640 0 client.15216 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3564961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be865d30 0x55b3be86f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 >> 172.21.15.96:0/3564961288 conn(0x55b3be863b60 msgr2=0x55b3be869700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 shutdown_connections 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 1 -- 172.21.15.96:0/3564961288 wait complete. 2024-09-10T22:47:02.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.267+0000 7fecddfb6740 20 client.15216 trim_cache size 0 max 16384 2024-09-10T22:47:02.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.285+0000 7fecddfb6740 1 Processor -- start 2024-09-10T22:47:02.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.285+0000 7fecddfb6740 1 -- start start 2024-09-10T22:47:02.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 0x55b3be86e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be864a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3be882a20 con 0x55b3be865d30 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3be8830f0 con 0x55b3be866650 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecddfb6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b3be8837c0 con 0x55b3be8725e0 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59946/0 (socket says 172.21.15.96:59946) 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 -- 172.21.15.96:0/2310708346 learned_addr learned my addr 172.21.15.96:0/2310708346 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd917c640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 0x55b3be86e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 -- 172.21.15.96:0/2310708346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 msgr2=0x55b3be864a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecda17e640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be864a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be864a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 -- 172.21.15.96:0/2310708346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 msgr2=0x55b3be86e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 0x55b3be86e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 -- 172.21.15.96:0/2310708346 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3be866480 con 0x55b3be865d30 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd917c640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 0x55b3be86e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecd997d640 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fecd010bdc0 tx=0x7fecd002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c4d6dd84dad31a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.286+0000 7fecbf7fe640 1 -- 172.21.15.96:0/2310708346 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd00cda80 con 0x55b3be865d30 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecbf7fe640 1 -- 172.21.15.96:0/2310708346 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd00cdc40 con 0x55b3be865d30 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecbf7fe640 1 -- 172.21.15.96:0/2310708346 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd00cdf00 con 0x55b3be865d30 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 -- 172.21.15.96:0/2310708346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 msgr2=0x55b3be86de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fecd010bdc0 tx=0x7fecd002a410 comp rx=0 tx=0).stop 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 -- 172.21.15.96:0/2310708346 shutdown_connections 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be866650 0x55b3be86e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be8725e0 0x55b3be864a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 --2- 172.21.15.96:0/2310708346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be865d30 0x55b3be86de20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 -- 172.21.15.96:0/2310708346 >> 172.21.15.96:0/2310708346 conn(0x55b3be863b60 msgr2=0x55b3be882020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 -- 172.21.15.96:0/2310708346 shutdown_connections 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.287+0000 7fecddfb6740 1 -- 172.21.15.96:0/2310708346 wait complete. 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 -- start start 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 Processor -- start 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 0x55b3be886bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be8725e0 0x55b3be887650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3be884560 con 0x55b3be866650 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3be883e90 con 0x55b3be8725e0 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b3be884c30 con 0x55b3be865d30 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecddfb6740 10 client.0 _ll_register_callbacks cb 0x7fecc400b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59962/0 (socket says 172.21.15.96:59962) 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 -- 172.21.15.96:0/3698029335 learned_addr learned my addr 172.21.15.96:0/3698029335 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd997d640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 0x55b3be886bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecda17e640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be8725e0 0x55b3be887650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 -- 172.21.15.96:0/3698029335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 msgr2=0x55b3be886bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 0x55b3be886bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 -- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be8725e0 msgr2=0x55b3be887650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.288+0000 7fecd917c640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be8725e0 0x55b3be887650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecd917c640 1 -- 172.21.15.96:0/3698029335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3be88c1f0 con 0x55b3be866650 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecd997d640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 0x55b3be886bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecd917c640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b3be871e90 tx=0x7fecc40186c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b8941d7d5e35b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4027040 con 0x55b3be866650 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fecc4005600 con 0x55b3be866650 2024-09-10T22:47:02.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd00cdc40 con 0x55b3be866650 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecc4005b10 con 0x55b3be866650 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3be8eff60 con 0x55b3be866650 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd00cdc40 con 0x55b3be866650 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fecc402cb60 con 0x55b3be866650 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 10 client.15222 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 20 client.15222 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be8f4b30 0x55b3be8f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 4 client.15222 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.289+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "status"}) -- 0x55b3be7db610 con 0x55b3be8f4b30 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecd997d640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be8f4b30 0x55b3be8f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecbd7fa640 10 client.15222 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:47:02.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecd997d640 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be8f4b30 0x55b3be8f3840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=300844759d6c9f08 server_cookie=8ab4dca825341a62 in_seq=0 out_seq=0 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecbd7fa640 1 -- 172.21.15.96:0/3698029335 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b3be7db610 con 0x55b3be8f4b30 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecbd7fa640 10 client.15222 handle_command_reply: tid=0 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecddfb6740 1 client.15222 shutdown 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 msgr2=0x55b3be887110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.290+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b3be871e90 tx=0x7fecc40186c0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 shutdown_connections 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55b3be8f4b30 0x55b3be8f3840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3be8725e0 0x55b3be887650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3be866650 0x55b3be887110 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 --2- 172.21.15.96:0/3698029335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b3be865d30 0x55b3be886bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecbd7fa640 0 client.15222 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 >> 172.21.15.96:0/3698029335 conn(0x55b3be863b60 msgr2=0x55b3be86dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 shutdown_connections 2024-09-10T22:47:02.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 1 -- 172.21.15.96:0/3698029335 wait complete. 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecddfb6740 20 client.15222 trim_cache size 0 max 16384 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24527, 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 92.247872762, 2024-09-10T22:47:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 132.24802986899999 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 msgr2=0x7fecc00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 0x7fecc00b77a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feccc007d90 tx=0x7feccc007340 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 msgr2=0x7fecd4198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fecd0006010 tx=0x7fecd0004270 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 shutdown_connections 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecd4108020 0x7fecd419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fecc00b52e0 0x7fecc00b77a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecd41076a0 0x7fecd41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 --2- 172.21.15.96:0/3114504160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fecd4101c90 0x7fecd4198170 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 >> 172.21.15.96:0/3114504160 conn(0x7fecd40fd700 msgr2=0x7fecd40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 shutdown_connections 2024-09-10T22:47:02.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:02.291+0000 7fecdbbe1640 1 -- 172.21.15.96:0/3114504160 wait complete. 2024-09-10T22:47:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:02 smithi096 ceph-mon[30869]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T22:47:02.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:02 smithi121 ceph-mon[35119]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T22:47:02.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:02 smithi117 ceph-mon[35081]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T22:47:03.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T22:47:03.347 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:03 smithi096 ceph-mon[30869]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T22:47:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:03 smithi121 ceph-mon[35119]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T22:47:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:03 smithi117 ceph-mon[35081]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T22:47:03.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 -- 172.21.15.96:0/1517162138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c1076a0 msgr2=0x7f597c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 --2- 172.21.15.96:0/1517162138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c1076a0 0x7f597c107ae0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f597002a410 comp rx=0 tx=0).stop 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 -- 172.21.15.96:0/1517162138 shutdown_connections 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 --2- 172.21.15.96:0/1517162138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c108020 0x7f597c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 --2- 172.21.15.96:0/1517162138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c1076a0 0x7f597c107ae0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 --2- 172.21.15.96:0/1517162138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c101c90 0x7f597c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 -- 172.21.15.96:0/1517162138 >> 172.21.15.96:0/1517162138 conn(0x7f597c0fd700 msgr2=0x7f597c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 -- 172.21.15.96:0/1517162138 shutdown_connections 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.829+0000 7f5981381640 1 -- 172.21.15.96:0/1517162138 wait complete. 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.830+0000 7f5981381640 1 Processor -- start 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.830+0000 7f5981381640 1 -- start start 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 0x7f597c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c108020 0x7f597c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f597c111d20 con 0x7f597c101c90 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f597c111ba0 con 0x7f597c1076a0 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f597c111ea0 con 0x7f597c108020 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59986/0 (socket says 172.21.15.96:59986) 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 -- 172.21.15.96:0/3805334200 learned_addr learned my addr 172.21.15.96:0/3805334200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597a7fc640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 0x7f597c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597b7fe640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c108020 0x7f597c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 -- 172.21.15.96:0/3805334200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c108020 msgr2=0x7f597c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c108020 0x7f597c19dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 -- 172.21.15.96:0/3805334200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 msgr2=0x7f597c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 0x7f597c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 -- 172.21.15.96:0/3805334200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c19e380 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597a7fc640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 0x7f597c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f597affd640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b4a0 tx=0x7f596c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5963fff640 1 -- 172.21.15.96:0/3805334200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c014070 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c19e670 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5963fff640 1 -- 172.21.15.96:0/3805334200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c010ac0 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5963fff640 1 -- 172.21.15.96:0/3805334200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c017c30 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.831+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a4f00 con 0x7f597c101c90 2024-09-10T22:47:03.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.834+0000 7f5983756740 1 Processor -- start 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.834+0000 7f5983756740 1 -- start start 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.834+0000 7f5963fff640 1 -- 172.21.15.96:0/3805334200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f596c010610 con 0x7f597c101c90 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 0x55f5dc381110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc391be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc2f8c10 con 0x55f5dc381650 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc2e8040 con 0x55f5dc380d30 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5983756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc2ebe50 con 0x55f5dc38f7f0 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59988/0 (socket says 172.21.15.96:59988) 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 -- 172.21.15.96:0/463732947 learned_addr learned my addr 172.21.15.96:0/463732947 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597b7fe640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 0x55f5dc381110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597a7fc640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc391be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 -- 172.21.15.96:0/463732947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 msgr2=0x55f5dc391be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc391be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 -- 172.21.15.96:0/463732947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 msgr2=0x55f5dc381110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 0x55f5dc381110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 -- 172.21.15.96:0/463732947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5dc046540 con 0x55f5dc381650 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597b7fe640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 0x55f5dc381110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f597affd640 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f596c0eb760 tx=0x7f596c0eb850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325e3d30488e387c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f59617fa640 1 -- 172.21.15.96:0/463732947 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c0f9070 con 0x55f5dc381650 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.835+0000 7f5963fff640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 0x7f59640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f59617fa640 1 -- 172.21.15.96:0/463732947 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c0e5d60 con 0x55f5dc381650 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f59617fa640 1 -- 172.21.15.96:0/463732947 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c0abad0 con 0x55f5dc381650 2024-09-10T22:47:03.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f597b7fe640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 0x7f59640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5963fff640 1 -- 172.21.15.96:0/3805334200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f596c0e54a0 con 0x7f597c101c90 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 -- 172.21.15.96:0/463732947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 msgr2=0x55f5dc38f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f597b7fe640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 0x7f59640b77a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5974007e40 tx=0x7f5974007510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f596c0eb760 tx=0x7f596c0eb850 comp rx=0 tx=0).stop 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 -- 172.21.15.96:0/463732947 shutdown_connections 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc391be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc38f2b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 --2- 172.21.15.96:0/463732947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc380d30 0x55f5dc381110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 -- 172.21.15.96:0/463732947 >> 172.21.15.96:0/463732947 conn(0x55f5dc37eb60 msgr2=0x55f5dc37fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 -- 172.21.15.96:0/463732947 shutdown_connections 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.836+0000 7f5983756740 1 -- 172.21.15.96:0/463732947 wait complete. 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 -- start start 2024-09-10T22:47:03.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 Processor -- start 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc381650 0x55f5dc38cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc38d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc1495f0 con 0x55f5dc380d30 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc283010 con 0x55f5dc38f7f0 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc3924a0 con 0x55f5dc381650 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f5983756740 10 client.0 _ll_register_callbacks cb 0x55f5dc2e2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59996/0 (socket says 172.21.15.96:59996) 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 -- 172.21.15.96:0/540978539 learned_addr learned my addr 172.21.15.96:0/540978539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:03.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597b7fe640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc38d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597a7fc640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc381650 0x55f5dc38cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 -- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc381650 msgr2=0x55f5dc38cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc381650 0x55f5dc38cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 -- 172.21.15.96:0/540978539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 msgr2=0x55f5dc38d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc38d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597affd640 1 -- 172.21.15.96:0/540978539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5dc2b0a00 con 0x55f5dc380d30 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.837+0000 7f597b7fe640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc38d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:03.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f597affd640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f596c0aba00 tx=0x7f596c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdec45894de54cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c0aa640 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f596c0a99b0 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c0e5d60 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f596c0aa810 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5dc39dfc0 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c0e5d60 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f596c0a7f70 con 0x55f5dc380d30 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:47:03.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 20 client.15234 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc39f310 0x55f5dc3a1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 4 client.15234 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.838+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5dc2fab60 con 0x55f5dc39f310 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.839+0000 7f597a7fc640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc39f310 0x55f5dc3a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.839+0000 7f597a7fc640 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc39f310 0x55f5dc3a1460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a24520fc47e5581 server_cookie=dfdc063acccd742b in_seq=0 out_seq=0 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.839+0000 7f595eff5640 10 client.15234 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f595eff5640 1 -- 172.21.15.96:0/540978539 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5dc2fab60 con 0x55f5dc39f310 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f595eff5640 10 client.15234 handle_command_reply: tid=0 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 client.15234 shutdown 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 msgr2=0x55f5dc38c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f596c0aba00 tx=0x7f596c00bf90 comp rx=0 tx=0).stop 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 shutdown_connections 2024-09-10T22:47:03.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc39f310 0x55f5dc3a1460 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc38d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc381650 0x55f5dc38cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f595eff5640 0 client.15234 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 --2- 172.21.15.96:0/540978539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc38c750 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 >> 172.21.15.96:0/540978539 conn(0x55f5dc37eb60 msgr2=0x55f5dc384700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 shutdown_connections 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 1 -- 172.21.15.96:0/540978539 wait complete. 2024-09-10T22:47:03.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.840+0000 7f5983756740 20 client.15234 trim_cache size 0 max 16384 2024-09-10T22:47:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 Processor -- start 2024-09-10T22:47:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 -- start start 2024-09-10T22:47:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 0x55f5dc38a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc380180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc39f7a0 con 0x55f5dc381650 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc39fe70 con 0x55f5dc38f7f0 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5983756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc3a0540 con 0x55f5dc380d30 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59998/0 (socket says 172.21.15.96:59998) 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 -- 172.21.15.96:0/3614684931 learned_addr learned my addr 172.21.15.96:0/3614684931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597b7fe640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc380180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597affd640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 0x55f5dc38a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 -- 172.21.15.96:0/3614684931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 msgr2=0x55f5dc38a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 0x55f5dc38a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 -- 172.21.15.96:0/3614684931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 msgr2=0x55f5dc380180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc380180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 -- 172.21.15.96:0/3614684931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5dc390340 con 0x55f5dc381650 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597b7fe640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc380180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597affd640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 0x55f5dc38a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f597a7fc640 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f59700092a0 tx=0x7f59700040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddf5d926f736424 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.858+0000 7f5960ff9640 1 -- 172.21.15.96:0/3614684931 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970035040 con 0x55f5dc381650 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5960ff9640 1 -- 172.21.15.96:0/3614684931 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59700054f0 con 0x55f5dc381650 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5960ff9640 1 -- 172.21.15.96:0/3614684931 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970004bf0 con 0x55f5dc381650 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 -- 172.21.15.96:0/3614684931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 msgr2=0x55f5dc37fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f59700092a0 tx=0x7f59700040f0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 -- 172.21.15.96:0/3614684931 shutdown_connections 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc38f7f0 0x55f5dc380180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc381650 0x55f5dc37fc40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 --2- 172.21.15.96:0/3614684931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc380d30 0x55f5dc38a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 -- 172.21.15.96:0/3614684931 >> 172.21.15.96:0/3614684931 conn(0x55f5dc37eb60 msgr2=0x55f5dc390850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 -- 172.21.15.96:0/3614684931 shutdown_connections 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.859+0000 7f5983756740 1 -- 172.21.15.96:0/3614684931 wait complete. 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 -- start start 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 Processor -- start 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc381650 0x55f5dc3aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc3a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc3a12e0 con 0x55f5dc380d30 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc3a0c10 con 0x55f5dc381650 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5dc3a19b0 con 0x55f5dc38f7f0 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f5983756740 10 client.0 _ll_register_callbacks cb 0x7f596c0f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60004/0 (socket says 172.21.15.96:60004) 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 -- 172.21.15.96:0/2589791856 learned_addr learned my addr 172.21.15.96:0/2589791856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597a7fc640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc381650 0x55f5dc3aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597b7fe640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc3a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 -- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 msgr2=0x55f5dc3a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc3a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 -- 172.21.15.96:0/2589791856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc381650 msgr2=0x55f5dc3aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc381650 0x55f5dc3aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 -- 172.21.15.96:0/2589791856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5dc3a7150 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f597affd640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b470 tx=0x7f596c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c565ff5c44a0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f595eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c01e070 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.860+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f596c010c70 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59700054f0 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f596c0aaf60 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5dc40f200 con 0x55f5dc380d30 2024-09-10T22:47:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59700054f0 con 0x55f5dc380d30 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f596c0a7f70 con 0x55f5dc380d30 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 20 client.15240 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc413be0 0x55f5dc4128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 4 client.15240 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "status"}) -- 0x55f5dc2f6610 con 0x55f5dc413be0 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.861+0000 7f597a7fc640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc413be0 0x55f5dc4128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f597a7fc640 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc413be0 0x55f5dc4128f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=982cbed2e29789a1 server_cookie=f69a8bc5de368853 in_seq=0 out_seq=0 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f595eff5640 10 client.15240 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f595eff5640 1 -- 172.21.15.96:0/2589791856 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f5dc2f6610 con 0x55f5dc413be0 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f595eff5640 10 client.15240 handle_command_reply: tid=0 2024-09-10T22:47:03.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 client.15240 shutdown 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 msgr2=0x55f5dc3a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b470 tx=0x7f596c0046c0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 shutdown_connections 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55f5dc413be0 0x55f5dc4128f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5dc381650 0x55f5dc3aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f595eff5640 0 client.15240 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5dc38f7f0 0x55f5dc3a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 --2- 172.21.15.96:0/2589791856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5dc380d30 0x55f5dc3a9b40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 >> 172.21.15.96:0/2589791856 conn(0x55f5dc37eb60 msgr2=0x55f5dc37fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 shutdown_connections 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 1 -- 172.21.15.96:0/2589791856 wait complete. 2024-09-10T22:47:03.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.862+0000 7f5983756740 20 client.15240 trim_cache size 0 max 16384 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24529, 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 94.834400134999996, 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:47:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 128.62298049200001 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 msgr2=0x7f59640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 0x7f59640b77a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5974007e40 tx=0x7f5974007510 comp rx=0 tx=0).stop 2024-09-10T22:47:03.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 msgr2=0x7f597c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b4a0 tx=0x7f596c00b960 comp rx=0 tx=0).stop 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 shutdown_connections 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f597c1076a0 0x7f597c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f59640b52e0 0x7f59640b77a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f597c108020 0x7f597c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 --2- 172.21.15.96:0/3805334200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f597c101c90 0x7f597c1982d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 >> 172.21.15.96:0/3805334200 conn(0x7f597c0fd700 msgr2=0x7f597c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 shutdown_connections 2024-09-10T22:47:03.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:03.863+0000 7f5981381640 1 -- 172.21.15.96:0/3805334200 wait complete. 2024-09-10T22:47:04.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:47:05.015 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:05.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/2745005399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac10d120 msgr2=0x7f7fac10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/2745005399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac10d120 0x7f7fac10f580 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009ca0 tx=0x7f7fa402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/2745005399 shutdown_connections 2024-09-10T22:47:05.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/2745005399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac1045e0 0x7f7fac10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/2745005399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac10d120 0x7f7fac10f580 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/2745005399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac103cc0 0x7f7fac1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/2745005399 >> 172.21.15.96:0/2745005399 conn(0x7f7fac0fd740 msgr2=0x7f7fac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/2745005399 shutdown_connections 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.506+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/2745005399 wait complete. 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 Processor -- start 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 -- start start 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac103cc0 0x7f7fac1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac10d120 0x7f7fac19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac112de0 con 0x7f7fac1045e0 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac112c60 con 0x7f7fac103cc0 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb3ea0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac112f60 con 0x7f7fac10d120 2024-09-10T22:47:05.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60026/0 (socket says 172.21.15.96:60026) 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 -- 172.21.15.96:0/1442432081 learned_addr learned my addr 172.21.15.96:0/1442432081 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac103cc0 0x7f7fac1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb243d640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac10d120 0x7f7fac19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 -- 172.21.15.96:0/1442432081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac10d120 msgr2=0x7f7fac19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac10d120 0x7f7fac19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 -- 172.21.15.96:0/1442432081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac103cc0 msgr2=0x7f7fac1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac103cc0 0x7f7fac1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.507+0000 7f7fb143b640 1 -- 172.21.15.96:0/1442432081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac19f260 con 0x7f7fac1045e0 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00d740 tx=0x7f7f9c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fa2ffd640 1 -- 172.21.15.96:0/1442432081 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c017020 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac19f550 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fa2ffd640 1 -- 172.21.15.96:0/1442432081 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9c010460 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fa2ffd640 1 -- 172.21.15.96:0/1442432081 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c00f620 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.508+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac1a5c10 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.510+0000 7f7fa2ffd640 1 -- 172.21.15.96:0/1442432081 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7f9c009a20 con 0x7f7fac1045e0 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.510+0000 7f7fb6275740 1 Processor -- start 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 -- start start 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb9d30 0x562ac2fba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac2f31c10 con 0x562ac2fba650 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ac2f21040 con 0x562ac2fb9d30 2024-09-10T22:47:05.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb6275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ac2f24e50 con 0x562ac2fca9f0 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60042/0 (socket says 172.21.15.96:60042) 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2868392166 learned_addr learned my addr 172.21.15.96:0/2868392166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb243d640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb9d30 0x562ac2fba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb143b640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2868392166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 msgr2=0x562ac2fccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2868392166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb9d30 msgr2=0x562ac2fba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb9d30 0x562ac2fba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2868392166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac2c7f540 con 0x562ac2fba650 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb143b640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:05.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fa2ffd640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 0x7f7f980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800ad30 tx=0x7f7fa800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b52baa1fb6c9f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.511+0000 7f7fb243d640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 0x7f7f980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7f97fff640 1 -- 172.21.15.96:0/2868392166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8010840 con 0x562ac2fba650 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fa2ffd640 1 -- 172.21.15.96:0/1442432081 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7f9c0e56d0 con 0x7f7fac1045e0 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7f97fff640 1 -- 172.21.15.96:0/2868392166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa8014070 con 0x562ac2fba650 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7f97fff640 1 -- 172.21.15.96:0/2868392166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8018a80 con 0x562ac2fba650 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 -- 172.21.15.96:0/2868392166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 msgr2=0x562ac2fca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb243d640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 0x7f7f980b77a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fe0 tx=0x7f7fa40093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800ad30 tx=0x7f7fa800f790 comp rx=0 tx=0).stop 2024-09-10T22:47:05.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 -- 172.21.15.96:0/2868392166 shutdown_connections 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fba650 0x562ac2fca4b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2868392166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb9d30 0x562ac2fba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.512+0000 7f7fb6275740 1 -- 172.21.15.96:0/2868392166 >> 172.21.15.96:0/2868392166 conn(0x562ac2fb7b60 msgr2=0x562ac2fb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- 172.21.15.96:0/2868392166 shutdown_connections 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- 172.21.15.96:0/2868392166 wait complete. 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- start start 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 Processor -- start 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fc7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fc83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac2d825f0 con 0x562ac2fb9d30 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ac2ebc010 con 0x562ac2fba650 2024-09-10T22:47:05.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fcd6a0 con 0x562ac2fca9f0 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb6275740 10 client.0 _ll_register_callbacks cb 0x562ac2f1b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60044/0 (socket says 172.21.15.96:60044) 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.513+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/1037024111 learned_addr learned my addr 172.21.15.96:0/1037024111 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb243d640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fc83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fc7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/1037024111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 msgr2=0x562ac2fc83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fc83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 msgr2=0x562ac2fc7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fc7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/1037024111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac2ee9a00 con 0x562ac2fb9d30 2024-09-10T22:47:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8000c00 tx=0x7f7fa800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f2e11e0b691e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80189e0 con 0x562ac2fb9d30 2024-09-10T22:47:05.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7fa8018b80 con 0x562ac2fb9d30 2024-09-10T22:47:05.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa8014070 con 0x562ac2fb9d30 2024-09-10T22:47:05.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fa8005d60 con 0x562ac2fb9d30 2024-09-10T22:47:05.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb6275740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:05.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ac2fd91c0 con 0x562ac2fb9d30 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.514+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa8014070 con 0x562ac2fb9d30 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa8010910 con 0x562ac2fb9d30 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 20 client.15252 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fdb5c0 0x562ac2fdca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 4 client.15252 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ac2f33b60 con 0x562ac2fdb5c0 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fdb5c0 0x562ac2fdca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7f957fa640 10 client.15252 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.515+0000 7f7fb143b640 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fdb5c0 0x562ac2fdca30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=548e09171fb6c2fa server_cookie=55feeaebcde80b07 in_seq=0 out_seq=0 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7f957fa640 1 -- 172.21.15.96:0/1037024111 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ac2f33b60 con 0x562ac2fdb5c0 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7f957fa640 10 client.15252 handle_command_reply: tid=0 2024-09-10T22:47:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 client.15252 shutdown 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 msgr2=0x562ac2fc7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8000c00 tx=0x7f7fa800a800 comp rx=0 tx=0).stop 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 shutdown_connections 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fdb5c0 0x562ac2fdca30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fc7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fca9f0 0x562ac2fc83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 --2- 172.21.15.96:0/1037024111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fc7920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 >> 172.21.15.96:0/1037024111 conn(0x562ac2fb7b60 msgr2=0x562ac2fb6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7f957fa640 0 client.15252 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:47:05.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 shutdown_connections 2024-09-10T22:47:05.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.516+0000 7f7fb6275740 1 -- 172.21.15.96:0/1037024111 wait complete. 2024-09-10T22:47:05.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.517+0000 7f7fb6275740 20 client.15252 trim_cache size 0 max 16384 2024-09-10T22:47:05.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.534+0000 7f7fb6275740 1 Processor -- start 2024-09-10T22:47:05.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.534+0000 7f7fb6275740 1 -- start start 2024-09-10T22:47:05.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 0x562ac2fc5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fba650 0x562ac2fcae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdad70 con 0x562ac2fb9d30 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdb440 con 0x562ac2fb7b60 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb6275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdbb10 con 0x562ac2fba650 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60046/0 (socket says 172.21.15.96:60046) 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 -- 172.21.15.96:0/439191239 learned_addr learned my addr 172.21.15.96:0/439191239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 0x562ac2fc5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb243d640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fba650 0x562ac2fcae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 -- 172.21.15.96:0/439191239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fba650 msgr2=0x562ac2fcae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fba650 0x562ac2fcae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 -- 172.21.15.96:0/439191239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 msgr2=0x562ac2fc5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 0x562ac2fc5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 -- 172.21.15.96:0/439191239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac2fc8bf0 con 0x562ac2fb9d30 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 0x562ac2fc5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7fb143b640 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0a91a0 tx=0x7f7f9c0e3140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e602a29a55fb8c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7f977fe640 1 -- 172.21.15.96:0/439191239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0ac630 con 0x562ac2fb9d30 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7f977fe640 1 -- 172.21.15.96:0/439191239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9c0abf30 con 0x562ac2fb9d30 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.535+0000 7f7f977fe640 1 -- 172.21.15.96:0/439191239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0a81a0 con 0x562ac2fb9d30 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- 172.21.15.96:0/439191239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 msgr2=0x562ac2fb74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0a91a0 tx=0x7f7f9c0e3140 comp rx=0 tx=0).stop 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- 172.21.15.96:0/439191239 shutdown_connections 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fba650 0x562ac2fcae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb9d30 0x562ac2fb74f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 --2- 172.21.15.96:0/439191239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fb7b60 0x562ac2fc5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- 172.21.15.96:0/439191239 >> 172.21.15.96:0/439191239 conn(0x562ac2fda990 msgr2=0x562ac2fcc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- 172.21.15.96:0/439191239 shutdown_connections 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- 172.21.15.96:0/439191239 wait complete. 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 -- start start 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.536+0000 7f7fb6275740 1 Processor -- start 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fb9d30 0x562ac2fe4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fe51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdc8b0 con 0x562ac2fb7b60 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdc1e0 con 0x562ac2fba650 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ac2fdcf80 con 0x562ac2fb9d30 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb6275740 10 client.0 _ll_register_callbacks cb 0x7f7fa4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60048/0 (socket says 172.21.15.96:60048) 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2216467874 learned_addr learned my addr 172.21.15.96:0/2216467874 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2216467874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fb9d30 msgr2=0x562ac2fe4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fb9d30 0x562ac2fe4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 msgr2=0x562ac2fe51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb243d640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fe51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fe51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 -- 172.21.15.96:0/2216467874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac2fe59f0 con 0x562ac2fb7b60 2024-09-10T22:47:05.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7fb1c3c640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8010ef0 tx=0x7f7fa8018d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71df0b1c0d67912 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.537+0000 7f7f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8005350 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7fa8010640 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9c0abf30 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fa800a2d0 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ac3047f60 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9c0abf30 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa803c060 con 0x562ac2fb7b60 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 20 client.15258 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:05.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fca9f0 0x562ac304b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 4 client.15258 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.538+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "status"}) -- 0x562ac2f2f610 con 0x562ac2fca9f0 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb143b640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fca9f0 0x562ac304b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7f957fa640 10 client.15258 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb143b640 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fca9f0 0x562ac304b7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb4f8b82d96630c9 server_cookie=1c09a28488f32636 in_seq=0 out_seq=0 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7f957fa640 1 -- 172.21.15.96:0/2216467874 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562ac2f2f610 con 0x562ac2fca9f0 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7f957fa640 10 client.15258 handle_command_reply: tid=0 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 client.15258 shutdown 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 msgr2=0x562ac2fbc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8010ef0 tx=0x7f7fa8018d00 comp rx=0 tx=0).stop 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 shutdown_connections 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562ac2fca9f0 0x562ac304b7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ac2fba650 0x562ac2fe51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ac2fb9d30 0x562ac2fe4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7f957fa640 0 client.15258 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 --2- 172.21.15.96:0/2216467874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac2fb7b60 0x562ac2fbc380 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 >> 172.21.15.96:0/2216467874 conn(0x562ac2fda990 msgr2=0x562ac2fcc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 shutdown_connections 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.539+0000 7f7fb6275740 1 -- 172.21.15.96:0/2216467874 wait complete. 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb6275740 20 client.15258 trim_cache size 0 max 16384 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:47:05.558 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24542, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 97.518952592999995, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 127.591002207 2024-09-10T22:47:05.559 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 msgr2=0x7f7f980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 0x7f7f980b77a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fe0 tx=0x7f7fa40093c0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 msgr2=0x7f7fac1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00d740 tx=0x7f7f9c00db40 comp rx=0 tx=0).stop 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 shutdown_connections 2024-09-10T22:47:05.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7f980b52e0 0x7f7f980b77a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fac10d120 0x7f7fac19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1045e0 0x7f7fac1996f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 --2- 172.21.15.96:0/1442432081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac103cc0 0x7f7fac1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 >> 172.21.15.96:0/1442432081 conn(0x7f7fac0fd740 msgr2=0x7f7fac0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 shutdown_connections 2024-09-10T22:47:05.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:05.540+0000 7f7fb3ea0640 1 -- 172.21.15.96:0/1442432081 wait complete. 2024-09-10T22:47:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:05 smithi096 ceph-mon[30869]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:05 smithi121 ceph-mon[35119]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:05 smithi117 ceph-mon[35081]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T22:47:06.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T22:47:06.589 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:07.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 -- 172.21.15.96:0/3127413965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c1045c0 msgr2=0x7f7b7c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 --2- 172.21.15.96:0/3127413965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c10cab0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-09-10T22:47:07.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 -- 172.21.15.96:0/3127413965 shutdown_connections 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 --2- 172.21.15.96:0/3127413965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c10d100 0x7f7b7c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 --2- 172.21.15.96:0/3127413965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c10cab0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 --2- 172.21.15.96:0/3127413965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 -- 172.21.15.96:0/3127413965 >> 172.21.15.96:0/3127413965 conn(0x7f7b7c0fd700 msgr2=0x7f7b7c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 -- 172.21.15.96:0/3127413965 shutdown_connections 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.081+0000 7f7b8149f640 1 -- 172.21.15.96:0/3127413965 wait complete. 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 Processor -- start 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 -- start start 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c112d70 con 0x7f7b7c10d100 2024-09-10T22:47:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c112bf0 con 0x7f7b7c103ca0 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.082+0000 7f7b8149f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c112ef0 con 0x7f7b7c1045c0 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54938/0 (socket says 172.21.15.96:54938) 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/702208985 learned_addr learned my addr 172.21.15.96:0/702208985 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7affd640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/702208985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c1045c0 msgr2=0x7f7b7c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/702208985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 msgr2=0x7f7b7c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/702208985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c19f350 con 0x7f7b7c10d100 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7affd640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:07.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d750 tx=0x7f7b7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b73fff640 1 -- 172.21.15.96:0/702208985 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74017070 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b73fff640 1 -- 172.21.15.96:0/702208985 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740054d0 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c19f640 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b73fff640 1 -- 172.21.15.96:0/702208985 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74004c60 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.083+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c1a5d00 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b73fff640 1 -- 172.21.15.96:0/702208985 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7b74004e00 con 0x7f7b7c10d100 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 Processor -- start 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 -- start start 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 0x5581aca40de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581ac9a5c10 con 0x5581aca2e650 2024-09-10T22:47:07.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5581ac995040 con 0x5581aca3e9f0 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b83874740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5581ac998e50 con 0x5581aca2dd30 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b7affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.086+0000 7f7b7affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54954/0 (socket says 172.21.15.96:54954) 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 -- 172.21.15.96:0/1565140361 learned_addr learned my addr 172.21.15.96:0/1565140361 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 0x5581aca40de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 -- 172.21.15.96:0/1565140361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 msgr2=0x5581aca2e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 -- 172.21.15.96:0/1565140361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 msgr2=0x5581aca40de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 0x5581aca40de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 -- 172.21.15.96:0/1565140361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ac6f3540 con 0x5581aca2e650 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 0x5581aca40de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:07.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b73fff640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 0x7f7b680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f20 tx=0x7f7b6c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81909751713c80db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 0x7f7b680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b73fff640 1 -- 172.21.15.96:0/702208985 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7b740e54d0 con 0x7f7b7c10d100 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b717fa640 1 -- 172.21.15.96:0/1565140361 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c003b70 con 0x5581aca2e650 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b717fa640 1 -- 172.21.15.96:0/1565140361 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c003d10 con 0x5581aca2e650 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.087+0000 7f7b717fa640 1 -- 172.21.15.96:0/1565140361 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c011900 con 0x5581aca2e650 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- 172.21.15.96:0/1565140361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 msgr2=0x5581aca3e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 0x7f7b680b77a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f20 tx=0x7f7b6c00f7f0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- 172.21.15.96:0/1565140361 shutdown_connections 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca3e9f0 0x5581aca40de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2e650 0x5581aca3e4b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 --2- 172.21.15.96:0/1565140361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- 172.21.15.96:0/1565140361 >> 172.21.15.96:0/1565140361 conn(0x5581aca2bb60 msgr2=0x5581aca2cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- 172.21.15.96:0/1565140361 shutdown_connections 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- 172.21.15.96:0/1565140361 wait complete. 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.088+0000 7f7b83874740 1 -- start start 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 Processor -- start 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581ac7f65f0 con 0x5581aca3e9f0 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5581ac930010 con 0x5581aca2e650 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5581aca416a0 con 0x5581aca2dd30 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 10 client.0 _ll_register_callbacks cb 0x5581ac98f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:07.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54966/0 (socket says 172.21.15.96:54966) 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/1045909697 learned_addr learned my addr 172.21.15.96:0/1045909697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 msgr2=0x5581aca3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/1045909697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 msgr2=0x5581aca3bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/1045909697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ac95da00 con 0x5581aca3e9f0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7b740a8300 tx=0x7f7b740ab560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faee13755cca93fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b627fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74020070 con 0x5581aca3e9f0 2024-09-10T22:47:07.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7b74005120 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.089+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c003d10 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b740abd30 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581aca4d180 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c003d10 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b740e5110 con 0x5581aca3e9f0 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 20 client.15270 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581aca4f770 0x5581aca50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 4 client.15270 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581ac9a7b60 con 0x5581aca4f770 2024-09-10T22:47:07.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.090+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581aca4f770 0x5581aca50be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.091+0000 7f7b627fc640 10 client.15270 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.091+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581aca4f770 0x5581aca50be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=788ae9cb6dcc7b37 server_cookie=68c1df05fd2c3d6f in_seq=0 out_seq=0 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.091+0000 7f7b627fc640 1 -- 172.21.15.96:0/1045909697 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581ac9a7b60 con 0x5581aca4f770 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.091+0000 7f7b627fc640 10 client.15270 handle_command_reply: tid=0 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 client.15270 shutdown 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 msgr2=0x5581aca3c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7b740a8300 tx=0x7f7b740ab560 comp rx=0 tx=0).stop 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 shutdown_connections 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581aca4f770 0x5581aca50be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3c400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 --2- 172.21.15.96:0/1045909697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b627fc640 0 client.15270 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 >> 172.21.15.96:0/1045909697 conn(0x5581aca2bb60 msgr2=0x5581aca2ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 shutdown_connections 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 1 -- 172.21.15.96:0/1045909697 wait complete. 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.092+0000 7f7b83874740 20 client.15270 trim_cache size 0 max 16384 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 Processor -- start 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 -- start start 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581aca4ef20 con 0x5581aca3e9f0 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5581aca4f5f0 con 0x5581aca2e650 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b83874740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5581aca4fcc0 con 0x5581aca2dd30 2024-09-10T22:47:07.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54982/0 (socket says 172.21.15.96:54982) 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/594715605 learned_addr learned my addr 172.21.15.96:0/594715605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/594715605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 msgr2=0x5581aca2b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/594715605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 msgr2=0x5581aca3c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.110+0000 7f7b7b7fe640 1 -- 172.21.15.96:0/594715605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581aca39e80 con 0x5581aca3e9f0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b740e57f0 tx=0x7f7b7400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26697a6894218668 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b70ff9640 1 -- 172.21.15.96:0/594715605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74020070 con 0x5581aca3e9f0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b70ff9640 1 -- 172.21.15.96:0/594715605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740b52c0 con 0x5581aca3e9f0 2024-09-10T22:47:07.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b70ff9640 1 -- 172.21.15.96:0/594715605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005120 con 0x5581aca3e9f0 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 -- 172.21.15.96:0/594715605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 msgr2=0x5581aca3a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b740e57f0 tx=0x7f7b7400df60 comp rx=0 tx=0).stop 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 -- 172.21.15.96:0/594715605 shutdown_connections 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca3c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca3e9f0 0x5581aca3a200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.111+0000 7f7b83874740 1 --2- 172.21.15.96:0/594715605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca2dd30 0x5581aca2b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- 172.21.15.96:0/594715605 >> 172.21.15.96:0/594715605 conn(0x5581aca2bb60 msgr2=0x5581aca3fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- 172.21.15.96:0/594715605 shutdown_connections 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- 172.21.15.96:0/594715605 wait complete. 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- start start 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 Processor -- start 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca3e9f0 0x5581aca2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581aca50a60 con 0x5581aca2dd30 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5581aca50390 con 0x5581aca2e650 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5581aca51130 con 0x5581aca3e9f0 2024-09-10T22:47:07.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b83874740 10 client.0 _ll_register_callbacks cb 0x7f7b7400de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b7affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b7affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54986/0 (socket says 172.21.15.96:54986) 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b7affd640 1 -- 172.21.15.96:0/1759279747 learned_addr learned my addr 172.21.15.96:0/1759279747 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b7b7fe640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca3e9f0 0x5581aca2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.112+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 -- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca3e9f0 msgr2=0x5581aca2fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca3e9f0 0x5581aca2fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 -- 172.21.15.96:0/1759279747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 msgr2=0x5581aca2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 -- 172.21.15.96:0/1759279747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581aca300a0 con 0x5581aca2dd30 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b7affd640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f20 tx=0x7f7b6c009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7273f62e6cbbadc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c029020 con 0x5581aca2dd30 2024-09-10T22:47:07.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7b6c005040 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b740b52c0 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c004e10 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581acac10e0 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740b52c0 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b6c02e070 con 0x5581aca2dd30 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T22:47:07.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.113+0000 7f7b83874740 20 client.15276 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581acac5980 0x5581acac4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 4 client.15276 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "status"}) -- 0x5581ac9a3610 con 0x5581acac5980 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581acac5980 0x5581acac4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b7a7fc640 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581acac5980 0x5581acac4690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2f8c6c2e9a2b6ac server_cookie=7c6afe1ab6f70cac in_seq=0 out_seq=0 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b627fc640 10 client.15276 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b627fc640 1 -- 172.21.15.96:0/1759279747 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5581ac9a3610 con 0x5581acac5980 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b627fc640 10 client.15276 handle_command_reply: tid=0 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 client.15276 shutdown 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 msgr2=0x5581aca2ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f20 tx=0x7f7b6c009ac0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 shutdown_connections 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.114+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5581acac5980 0x5581acac4690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5581aca2e650 0x5581aca2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5581aca3e9f0 0x5581aca2fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 --2- 172.21.15.96:0/1759279747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581aca2dd30 0x5581aca2ed80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 >> 172.21.15.96:0/1759279747 conn(0x5581aca2bb60 msgr2=0x5581aca2cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b627fc640 0 client.15276 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 shutdown_connections 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 1 -- 172.21.15.96:0/1759279747 wait complete. 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b83874740 20 client.15276 trim_cache size 0 max 16384 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24550, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 100.103803282, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 126.53536268800001 2024-09-10T22:47:07.133 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 msgr2=0x7f7b680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 0x7f7b680b77a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64031400 comp rx=0 tx=0).stop 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 msgr2=0x7f7b7c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d750 tx=0x7f7b7400dc10 comp rx=0 tx=0).stop 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 shutdown_connections 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7b680b52e0 0x7f7b680b77a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b7c10d100 0x7f7b7c19eb80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c1045c0 0x7f7b7c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 --2- 172.21.15.96:0/702208985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b7c103ca0 0x7f7b7c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:07.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 >> 172.21.15.96:0/702208985 conn(0x7f7b7c0fd700 msgr2=0x7f7b7c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:07.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 shutdown_connections 2024-09-10T22:47:07.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:07.115+0000 7f7b8149f640 1 -- 172.21.15.96:0/702208985 wait complete. 2024-09-10T22:47:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:07 smithi096 ceph-mon[30869]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T22:47:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:07 smithi121 ceph-mon[35119]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T22:47:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:07 smithi117 ceph-mon[35081]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T22:47:07.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:47:08.097 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.563+0000 7fa9a865d640 1 -- 172.21.15.96:0/1521126845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 msgr2=0x7fa9a0115f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.563+0000 7fa9a865d640 1 --2- 172.21.15.96:0/1521126845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a0115f30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa998009900 tx=0x7fa99802a410 comp rx=0 tx=0).stop 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 -- 172.21.15.96:0/1521126845 shutdown_connections 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 --2- 172.21.15.96:0/1521126845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075e60 0x7fa9a0113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 --2- 172.21.15.96:0/1521126845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a0115f30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 --2- 172.21.15.96:0/1521126845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075540 0x7fa9a0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 -- 172.21.15.96:0/1521126845 >> 172.21.15.96:0/1521126845 conn(0x7fa9a00fe830 msgr2=0x7fa9a0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 -- 172.21.15.96:0/1521126845 shutdown_connections 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 -- 172.21.15.96:0/1521126845 wait complete. 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 Processor -- start 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.564+0000 7fa9a865d640 1 -- start start 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075540 0x7fa9a019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075e60 0x7fa9a019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119410 con 0x7fa9a0113b40 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119290 con 0x7fa9a0075540 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0119590 con 0x7fa9a0075e60 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55002/0 (socket says 172.21.15.96:55002) 2024-09-10T22:47:08.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 -- 172.21.15.96:0/2276620540 learned_addr learned my addr 172.21.15.96:0/2276620540 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075e60 0x7fa9a019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075540 0x7fa9a019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 -- 172.21.15.96:0/2276620540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075e60 msgr2=0x7fa9a019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075e60 0x7fa9a019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 -- 172.21.15.96:0/2276620540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075540 msgr2=0x7fa9a019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075540 0x7fa9a019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 -- 172.21.15.96:0/2276620540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a5990 con 0x7fa9a0113b40 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a6bfa640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa998005940 tx=0x7fa9980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa99f7fe640 1 -- 172.21.15.96:0/2276620540 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99803f030 con 0x7fa9a0113b40 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa99f7fe640 1 -- 172.21.15.96:0/2276620540 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa998003cb0 con 0x7fa9a0113b40 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a5c20 con 0x7fa9a0113b40 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.565+0000 7fa99f7fe640 1 -- 172.21.15.96:0/2276620540 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998035070 con 0x7fa9a0113b40 2024-09-10T22:47:08.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.566+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01ac340 con 0x7fa9a0113b40 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa99f7fe640 1 -- 172.21.15.96:0/2276620540 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa99803bbb0 con 0x7fa9a0113b40 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 Processor -- start 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 -- start start 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe08fd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fd650 0x560fe09090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe0874c10 con 0x560fe09095e0 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe0864040 con 0x560fe08fcd30 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fe0867e50 con 0x560fe08fd650 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9a5bf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9a5bf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55006/0 (socket says 172.21.15.96:55006) 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.568+0000 7fa9a5bf8640 1 -- 172.21.15.96:0/2737793660 learned_addr learned my addr 172.21.15.96:0/2737793660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:08.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fd650 0x560fe09090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe08fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa99f7fe640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 0x7fa9880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 -- 172.21.15.96:0/2737793660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fd650 msgr2=0x560fe09090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fd650 0x560fe09090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 -- 172.21.15.96:0/2737793660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 msgr2=0x560fe08fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe08fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 -- 172.21.15.96:0/2737793660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe05c2540 con 0x560fe09095e0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a6bfa640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 0x7fa9880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa9940125f0 tx=0x7fa99400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98885121d8ee41d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa99f7fe640 1 -- 172.21.15.96:0/2276620540 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa9981066a0 con 0x7fa9a0113b40 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa99cff9640 1 -- 172.21.15.96:0/2737793660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940106b0 con 0x560fe09095e0 2024-09-10T22:47:08.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa99cff9640 1 -- 172.21.15.96:0/2737793660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa994010d00 con 0x560fe09095e0 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.569+0000 7fa99cff9640 1 -- 172.21.15.96:0/2737793660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99401dc40 con 0x560fe09095e0 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2737793660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 msgr2=0x560fe090b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa9940125f0 tx=0x7fa99400b870 comp rx=0 tx=0).stop 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9a6bfa640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 0x7fa9880b77a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ca220 tx=0x7fa998047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2737793660 shutdown_connections 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe090b9d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fd650 0x560fe09090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2737793660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe08fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2737793660 >> 172.21.15.96:0/2737793660 conn(0x560fe08fab60 msgr2=0x560fe08fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2737793660 shutdown_connections 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.570+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2737793660 wait complete. 2024-09-10T22:47:08.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 -- start start 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 Processor -- start 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe09065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe09095e0 0x560fe0907060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe06c55f0 con 0x560fe08fd650 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe07ff010 con 0x560fe08fcd30 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fe090c290 con 0x560fe09095e0 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9aaa32740 10 client.0 _ll_register_callbacks cb 0x560fe085e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55008/0 (socket says 172.21.15.96:55008) 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 -- 172.21.15.96:0/2269049723 learned_addr learned my addr 172.21.15.96:0/2269049723 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:08.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe09095e0 0x560fe0907060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe09065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 -- 172.21.15.96:0/2269049723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe09095e0 msgr2=0x560fe0907060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe09095e0 0x560fe0907060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 -- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 msgr2=0x560fe09065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe09065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a63f9640 1 -- 172.21.15.96:0/2269049723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe082ca00 con 0x560fe08fd650 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.571+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe09065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa99000ab60 tx=0x7fa990011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7813b780cc9bd1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99001a070 con 0x560fe08fd650 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa990016450 con 0x560fe08fd650 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa994010d00 con 0x560fe08fd650 2024-09-10T22:47:08.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa990026050 con 0x560fe08fd650 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560fe0917f10 con 0x560fe08fd650 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa994010d00 con 0x560fe08fd650 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa99001e6e0 con 0x560fe08fd650 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 20 client.15288 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe091a560 0x560fe091b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 4 client.15288 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.572+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe0876b60 con 0x560fe091a560 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.573+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe091a560 0x560fe091b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.573+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe091a560 0x560fe091b9d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7932db010ec695da server_cookie=49c40ed8264624b3 in_seq=0 out_seq=0 2024-09-10T22:47:08.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.573+0000 7fa985ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa985ffb640 1 -- 172.21.15.96:0/2269049723 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560fe0876b60 con 0x560fe091a560 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa985ffb640 10 client.15288 handle_command_reply: tid=0 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 client.15288 shutdown 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 msgr2=0x560fe0906b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa99000ab60 tx=0x7fa990011580 comp rx=0 tx=0).stop 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 shutdown_connections 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe091a560 0x560fe091b9d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe09095e0 0x560fe0907060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0906b20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa985ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:08.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/2269049723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe08fcd30 0x560fe09065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 >> 172.21.15.96:0/2269049723 conn(0x560fe08fab60 msgr2=0x560fe0900700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 shutdown_connections 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 1 -- 172.21.15.96:0/2269049723 wait complete. 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.574+0000 7fa9aaa32740 20 client.15288 trim_cache size 0 max 16384 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 Processor -- start 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 -- start start 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe08f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe09095e0 0x560fe0904e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe0919d10 con 0x560fe08fd650 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe091a3e0 con 0x560fe09095e0 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fe091aab0 con 0x560fe08fcd30 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55024/0 (socket says 172.21.15.96:55024) 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.592+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1481519045 learned_addr learned my addr 172.21.15.96:0/1481519045 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe08f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1481519045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 msgr2=0x560fe08f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe08f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1481519045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe09095e0 msgr2=0x560fe0904e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe09095e0 0x560fe0904e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1481519045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe08fa990 con 0x560fe08fd650 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa990000f80 tx=0x7fa990009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30d1805d5167687 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa987fff640 1 -- 172.21.15.96:0/1481519045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99002f070 con 0x560fe08fd650 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa987fff640 1 -- 172.21.15.96:0/1481519045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99001e860 con 0x560fe08fd650 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.593+0000 7fa987fff640 1 -- 172.21.15.96:0/1481519045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99001a040 con 0x560fe08fd650 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1481519045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 msgr2=0x560fe0900da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa990000f80 tx=0x7fa990009740 comp rx=0 tx=0).stop 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1481519045 shutdown_connections 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe09095e0 0x560fe0904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe08fd650 0x560fe0900da0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1481519045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe08f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1481519045 >> 172.21.15.96:0/1481519045 conn(0x560fe08fab60 msgr2=0x560fe0901880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1481519045 shutdown_connections 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1481519045 wait complete. 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 -- start start 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.594+0000 7fa9aaa32740 1 Processor -- start 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe0923c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe091ebe0 0x560fe091f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fe091b850 con 0x560fe09095e0 2024-09-10T22:47:08.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fe091b180 con 0x560fe091ebe0 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fe091bf20 con 0x560fe08fcd30 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9aaa32740 10 client.0 _ll_register_callbacks cb 0x7fa990009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55032/0 (socket says 172.21.15.96:55032) 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1721575359 learned_addr learned my addr 172.21.15.96:0/1721575359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe0923c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe091ebe0 0x560fe091f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1721575359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 msgr2=0x560fe0923c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe0923c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe091ebe0 msgr2=0x560fe091f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:08.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe091ebe0 0x560fe091f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 -- 172.21.15.96:0/1721575359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe08fb5e0 con 0x560fe09095e0 2024-09-10T22:47:08.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa9a63f9640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa990002230 tx=0x7fa99001ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d063d389523437f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:08.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.595+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99003a060 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa99001e3b0 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99001e860 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa99002f030 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560fe0987840 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99001e860 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa99003f070 con 0x560fe09095e0 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:47:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 20 client.15294 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe098bf50 0x560fe098ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 4 client.15294 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:47:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.596+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "status"}) -- 0x560fe0872610 con 0x560fe098bf50 2024-09-10T22:47:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe098bf50 0x560fe098ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9a5bf8640 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe098bf50 0x560fe098ac60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b390f68c9613ef7e server_cookie=d6bdf3d0c6afda2d in_seq=0 out_seq=0 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa985ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa985ffb640 1 -- 172.21.15.96:0/1721575359 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x560fe0872610 con 0x560fe098bf50 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa985ffb640 10 client.15294 handle_command_reply: tid=0 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9aaa32740 1 client.15294 shutdown 2024-09-10T22:47:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 msgr2=0x560fe0924170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa990002230 tx=0x7fa99001ef50 comp rx=0 tx=0).stop 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 shutdown_connections 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.597+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560fe098bf50 0x560fe098ac60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fe091ebe0 0x560fe091f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fe09095e0 0x560fe0924170 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 --2- 172.21.15.96:0/1721575359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fe08fcd30 0x560fe0923c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 >> 172.21.15.96:0/1721575359 conn(0x560fe08fab60 msgr2=0x560fe0901880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa985ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 shutdown_connections 2024-09-10T22:47:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 1 -- 172.21.15.96:0/1721575359 wait complete. 2024-09-10T22:47:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9aaa32740 20 client.15294 trim_cache size 0 max 16384 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "14afb286-6fc5-11ef-bcea-c7b262605968", 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24566, 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-10T22:47:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 108.641084959, 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 27, 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 77, 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 119.35511216 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 msgr2=0x7fa9880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 0x7fa9880b77a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ca220 tx=0x7fa998047000 comp rx=0 tx=0).stop 2024-09-10T22:47:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 msgr2=0x7fa9a01a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa998005940 tx=0x7fa9980042f0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 shutdown_connections 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9880b52e0 0x7fa9880b77a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9a0113b40 0x7fa9a01a51c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9a0075e60 0x7fa9a019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 --2- 172.21.15.96:0/2276620540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0075540 0x7fa9a019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 >> 172.21.15.96:0/2276620540 conn(0x7fa9a00fe830 msgr2=0x7fa9a00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 shutdown_connections 2024-09-10T22:47:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:08.598+0000 7fa9a865d640 1 -- 172.21.15.96:0/2276620540 wait complete. 2024-09-10T22:47:09.420 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T22:47:09.510 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T22:47:09.510 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T22:47:09.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:09.511 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-10T22:47:09.512 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:09 smithi096 ceph-mon[30869]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T22:47:09.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:09.527 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-10T22:47:09.595 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-09-10T22:47:09.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:47:09.596 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-10T22:47:09.597 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:47:09.597 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-10T22:47:09.597 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-10T22:47:09.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:47:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:09 smithi121 ceph-mon[35119]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T22:47:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:09 smithi117 ceph-mon[35081]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T22:47:09.891 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:10.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 -- 172.21.15.96:0/2740850399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec109150 msgr2=0x7f90ec10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 --2- 172.21.15.96:0/2740850399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec109150 0x7f90ec10f950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009ca0 tx=0x7f90e402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 -- 172.21.15.96:0/2740850399 shutdown_connections 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 --2- 172.21.15.96:0/2740850399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90ec1087d0 0x7f90ec108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 --2- 172.21.15.96:0/2740850399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec109150 0x7f90ec10f950 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 --2- 172.21.15.96:0/2740850399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ec102dc0 0x7f90ec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 -- 172.21.15.96:0/2740850399 >> 172.21.15.96:0/2740850399 conn(0x7f90ec0fe830 msgr2=0x7f90ec100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 -- 172.21.15.96:0/2740850399 shutdown_connections 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.395+0000 7f90f453f640 1 -- 172.21.15.96:0/2740850399 wait complete. 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 Processor -- start 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 -- start start 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ec1087d0 0x7f90ec199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90ec109150 0x7f90ec19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec112e10 con 0x7f90ec102dc0 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec112c90 con 0x7f90ec109150 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f453f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec112f90 con 0x7f90ec1087d0 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55050/0 (socket says 172.21.15.96:55050) 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 -- 172.21.15.96:0/3815070015 learned_addr learned my addr 172.21.15.96:0/3815070015 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 -- 172.21.15.96:0/3815070015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ec1087d0 msgr2=0x7f90ec199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ec1087d0 0x7f90ec199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 -- 172.21.15.96:0/3815070015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90ec109150 msgr2=0x7f90ec19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:10.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90ec109150 0x7f90ec19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 -- 172.21.15.96:0/3815070015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec19f270 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.396+0000 7f90f22db640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c890 tx=0x7f90e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.397+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0004220 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.397+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec19f560 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.397+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e00043c0 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.397+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec1a5df0 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.397+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0004a00 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.398+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ec1031a0 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.399+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f90e00079b0 con 0x7f90ec102dc0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.400+0000 7f90df7fe640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 0x7f90d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.400+0000 7f90f1ada640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 0x7f90d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:10.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.400+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f90e00e5d50 con 0x7f90ec102dc0 2024-09-10T22:47:10.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.401+0000 7f90f1ada640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 0x7f90d80b77a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90ec100f60 tx=0x7f90e8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:10.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.403+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f90e0014030 con 0x7f90ec102dc0 2024-09-10T22:47:10.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.614+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90ec1a6170 con 0x7f90ec102dc0 2024-09-10T22:47:10.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.615+0000 7f90df7fe640 1 -- 172.21.15.96:0/3815070015 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f90e00accb0 con 0x7f90ec102dc0 2024-09-10T22:47:10.616 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:47:10.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 msgr2=0x7f90d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 0x7f90d80b77a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90ec100f60 tx=0x7f90e8005c70 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 msgr2=0x7f90ec1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c890 tx=0x7f90e000cd50 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 shutdown_connections 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90ec109150 0x7f90ec19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90d80b52e0 0x7f90d80b77a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ec1087d0 0x7f90ec199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 --2- 172.21.15.96:0/3815070015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ec102dc0 0x7f90ec1991c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 >> 172.21.15.96:0/3815070015 conn(0x7f90ec0fe830 msgr2=0x7f90ec101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 shutdown_connections 2024-09-10T22:47:10.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:10.617+0000 7f90f453f640 1 -- 172.21.15.96:0/3815070015 wait complete. 2024-09-10T22:47:10.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:10 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3815070015' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:47:11.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:10 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3815070015' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:47:11.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:10 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3815070015' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:47:11.411 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:47:11.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:47:11.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-10T22:47:11.411 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:47:11.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-10T22:47:11.412 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:47:11.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:47:11.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:47:11.412 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-10T22:47:11.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:11.412 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:47:11.429 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42179sec preferred_lft 42179sec 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link noprefixroute 2024-09-10T22:47:11.430 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:47:11.431 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:47:11.431 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-10T22:47:11.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:11.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:11.648 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:47:11.673 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-10T22:47:11.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:11.675 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-10T22:47:11.752 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-10T22:47:11.752 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:47:11.753 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:47:11.753 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:47:11.753 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:47:11.753 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:47:11.753 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-10T22:47:11.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:47:11.754 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:47:11.754 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:47:11.754 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-10T22:47:11.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:11.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:11 smithi096 ceph-mon[30869]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:12.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:11 smithi121 ceph-mon[35119]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:12.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:11 smithi117 ceph-mon[35081]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:12.071 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-10T22:47:12.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:12.087 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-10T22:47:12.143 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:47:12.143 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-10T22:47:12.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-10T22:47:12.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-10T22:47:12.143 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-10T22:47:12.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.261 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-10T22:47:12.261 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-10T22:47:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:12.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.386 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:47:12.386 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-10T22:47:12.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-10T22:47:12.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:47:12.387 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-10T22:47:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:47:12.463 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /tmp/cephfs 2024-09-10T22:47:12.477 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-10T22:47:12.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:47:12.478 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /tmp/cephfs 2024-09-10T22:47:12.533 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T22:47:12.534 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T22:47:12.534 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/83316ff1-820e-4d07-b614-9a4f7ce8988b 2024-09-10T22:47:12.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:12.534 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/83316ff1-820e-4d07-b614-9a4f7ce8988b /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-09-10T22:47:12.629 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-10T22:47:12.629 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-10T22:47:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-10T22:47:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-10T22:47:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.96:3300,172.21.15.117:3300,172.21.15.121:3300" 2024-09-10T22:47:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with new device syntax: 0@14afb286-6fc5-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/83316ff1-820e-4d07-b614-9a4f7ce8988b 2024-09-10T22:47:12.750 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.96:3300/172.21.15.117:3300/172.21.15.121:3300" will pass to kernel 2024-09-10T22:47:12.753 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:47:12.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:47:12.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /tmp/cephfs 2024-09-10T22:47:12.779 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:47:12.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:47:12.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.875 INFO:teuthology.orchestra.run.smithi096.stdout:client24919 2024-09-10T22:47:12.875 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T22:47:12.876 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:47:12.893 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:47:12.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:47:12.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:12.989 INFO:teuthology.orchestra.run.smithi096.stdout:client24919 2024-09-10T22:47:12.990 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:47:13.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:47:13.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:13.045 INFO:teuthology.orchestra.run.smithi096.stdout:client24919 2024-09-10T22:47:13.046 DEBUG:teuthology.orchestra.run.smithi096:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/14afb286-6fc5-11ef-bcea-c7b262605968.client24919/status') 2024-09-10T22:47:13.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:47:13.104 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.24919 (3)172.21.15.96:0/437480600 2024-09-10T22:47:13.104 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-09-10T22:47:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-09-10T22:47:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-09-10T22:47:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:70 bytes copied, 2.5115e-05 s, 2.8 MB/s 2024-09-10T22:47:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:47:13.112 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:47:13.123 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:47:13.123 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:47:13.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 -- 172.21.15.96:0/3893170343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8103370 msgr2=0x7f05e8109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3893170343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8103370 0x7f05e8109cf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009cd0 tx=0x7f05d802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 -- 172.21.15.96:0/3893170343 shutdown_connections 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3893170343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3893170343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8103370 0x7f05e8109cf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3893170343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8101860 0x7f05e8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 -- 172.21.15.96:0/3893170343 >> 172.21.15.96:0/3893170343 conn(0x7f05e80fd140 msgr2=0x7f05e80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 -- 172.21.15.96:0/3893170343 shutdown_connections 2024-09-10T22:47:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.222+0000 7f05ecf82640 1 -- 172.21.15.96:0/3893170343 wait complete. 2024-09-10T22:47:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 Processor -- start 2024-09-10T22:47:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 -- start start 2024-09-10T22:47:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8103370 0x7f05e819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810e680 con 0x7f05e8101860 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810e500 con 0x7f05e81029b0 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05ecf82640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810e800 con 0x7f05e8103370 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55074/0 (socket says 172.21.15.96:55074) 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.223+0000 7f05e659c640 1 -- 172.21.15.96:0/3057133502 learned_addr learned my addr 172.21.15.96:0/3057133502 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e6d9d640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8103370 0x7f05e819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e5d9b640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 -- 172.21.15.96:0/3057133502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8103370 msgr2=0x7f05e819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8103370 0x7f05e819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 -- 172.21.15.96:0/3057133502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 msgr2=0x7f05e81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 -- 172.21.15.96:0/3057133502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e819a8c0 con 0x7f05e8101860 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e5d9b640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05e659c640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b4a0 tx=0x7f05e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0014070 con 0x7f05e8101860 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e819abb0 con 0x7f05e8101860 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0010ac0 con 0x7f05e8101860 2024-09-10T22:47:13.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0017c30 con 0x7f05e8101860 2024-09-10T22:47:13.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.224+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e81a1280 con 0x7f05e8101860 2024-09-10T22:47:13.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.225+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e8108b00 con 0x7f05e8101860 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.226+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f05e0010610 con 0x7f05e8101860 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.227+0000 7f05df7fe640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 0x7f05d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.227+0000 7f05e5d9b640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 0x7f05d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.227+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f05e00e54b0 con 0x7f05e8101860 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.228+0000 7f05e5d9b640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 0x7f05d40b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f05d00058c0 tx=0x7f05d0005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.230+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f05e00ac410 con 0x7f05e8101860 2024-09-10T22:47:13.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:13.444+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f05e802e5c0 con 0x7f05e8101860 2024-09-10T22:47:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:13 smithi096 ceph-mon[30869]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:13 smithi121 ceph-mon[35119]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:13 smithi117 ceph-mon[35081]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:47:14.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.399+0000 7f05df7fe640 1 -- 172.21.15.96:0/3057133502 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f05e00abd10 con 0x7f05e8101860 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 msgr2=0x7f05d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 0x7f05d40b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f05d00058c0 tx=0x7f05d0005d30 comp rx=0 tx=0).stop 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 msgr2=0x7f05e8198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b4a0 tx=0x7f05e000b960 comp rx=0 tx=0).stop 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 shutdown_connections 2024-09-10T22:47:14.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05e81029b0 0x7f05e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f05d40b52e0 0x7f05d40b77a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05e8103370 0x7f05e819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 --2- 172.21.15.96:0/3057133502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e8101860 0x7f05e8198d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 >> 172.21.15.96:0/3057133502 conn(0x7f05e80fd140 msgr2=0x7f05e8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 shutdown_connections 2024-09-10T22:47:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.402+0000 7f05ecf82640 1 -- 172.21.15.96:0/3057133502 wait complete. 2024-09-10T22:47:14.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 -- 172.21.15.96:0/448756560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680103380 msgr2=0x7f2680109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 --2- 172.21.15.96:0/448756560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680103380 0x7f2680109ec0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2678009ca0 tx=0x7f267802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 -- 172.21.15.96:0/448756560 shutdown_connections 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 --2- 172.21.15.96:0/448756560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680103380 0x7f2680109ec0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 --2- 172.21.15.96:0/448756560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680102810 0x7f2680102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 --2- 172.21.15.96:0/448756560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680101ed0 0x7f26801022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 -- 172.21.15.96:0/448756560 >> 172.21.15.96:0/448756560 conn(0x7f26800fc8c0 msgr2=0x7f26800fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 -- 172.21.15.96:0/448756560 shutdown_connections 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.524+0000 7f26877bb640 1 -- 172.21.15.96:0/448756560 wait complete. 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 Processor -- start 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 -- start start 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680102810 0x7f268019fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 0x7f26801a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f268010d500 con 0x7f2680101ed0 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f268010d380 con 0x7f2680102810 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f26877bb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f268010d680 con 0x7f2680103380 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55096/0 (socket says 172.21.15.96:55096) 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 -- 172.21.15.96:0/3280195923 learned_addr learned my addr 172.21.15.96:0/3280195923 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2684d56640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680102810 0x7f268019fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 -- 172.21.15.96:0/3280195923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 msgr2=0x7f26801a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685d58640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 0x7f26801a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 0x7f26801a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 -- 172.21.15.96:0/3280195923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680102810 msgr2=0x7f268019fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680102810 0x7f268019fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 -- 172.21.15.96:0/3280195923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26801a2b60 con 0x7f2680101ed0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685d58640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 0x7f26801a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.525+0000 7f2685557640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c890 tx=0x7f267c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.526+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c004220 con 0x7f2680101ed0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.526+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c0043c0 con 0x7f2680101ed0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.526+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26801a2e50 con 0x7f2680101ed0 2024-09-10T22:47:14.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.526+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c004a00 con 0x7f2680101ed0 2024-09-10T22:47:14.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.526+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a32b0 con 0x7f2680101ed0 2024-09-10T22:47:14.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.527+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26801a35d0 con 0x7f2680101ed0 2024-09-10T22:47:14.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.528+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f267c020060 con 0x7f2680101ed0 2024-09-10T22:47:14.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.529+0000 7f26767fc640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 0x7f266c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.529+0000 7f2684d56640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 0x7f266c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.529+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f267c0e5950 con 0x7f2680101ed0 2024-09-10T22:47:14.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.530+0000 7f2684d56640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 0x7f266c0b77a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2670005fd0 tx=0x7f2670005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:14.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.531+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f267c014030 con 0x7f2680101ed0 2024-09-10T22:47:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.732+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f26801045a0 con 0x7f266c0b52e0 2024-09-10T22:47:14.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-10T22:47:14.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.751+0000 7f26767fc640 1 -- 172.21.15.96:0/3280195923 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f26801045a0 con 0x7f266c0b52e0 2024-09-10T22:47:14.753 INFO:teuthology.orchestra.run.smithi096.stdout:{} 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 msgr2=0x7f266c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 0x7f266c0b77a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2670005fd0 tx=0x7f2670005850 comp rx=0 tx=0).stop 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 msgr2=0x7f2680101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c890 tx=0x7f267c00cd50 comp rx=0 tx=0).stop 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 shutdown_connections 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2680102810 0x7f268019fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f266c0b52e0 0x7f266c0b77a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2680103380 0x7f26801a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 --2- 172.21.15.96:0/3280195923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2680101ed0 0x7f2680101400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 >> 172.21.15.96:0/3280195923 conn(0x7f26800fc8c0 msgr2=0x7f2680103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 shutdown_connections 2024-09-10T22:47:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.753+0000 7f26877bb640 1 -- 172.21.15.96:0/3280195923 wait complete. 2024-09-10T22:47:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:14 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-10T22:47:14.766 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T22:47:14.776 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T22:47:14.802 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:47:14.802 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:14 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 -- 172.21.15.96:0/1096998458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec06bee0 msgr2=0x7fc8ec105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 --2- 172.21.15.96:0/1096998458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec105560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009900 tx=0x7fc8dc02a410 comp rx=0 tx=0).stop 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 -- 172.21.15.96:0/1096998458 shutdown_connections 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 --2- 172.21.15.96:0/1096998458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec1086a0 0x7fc8ec10ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 --2- 172.21.15.96:0/1096998458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec105af0 0x7fc8ec107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 --2- 172.21.15.96:0/1096998458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec105560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 -- 172.21.15.96:0/1096998458 >> 172.21.15.96:0/1096998458 conn(0x7fc8ec0fd150 msgr2=0x7fc8ec0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 -- 172.21.15.96:0/1096998458 shutdown_connections 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.900+0000 7fc8f093d640 1 -- 172.21.15.96:0/1096998458 wait complete. 2024-09-10T22:47:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 Processor -- start 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 -- start start 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec1086a0 0x7fc8ec199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10e5d0 con 0x7fc8ec105af0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10e450 con 0x7fc8ec06bee0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8f093d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10e750 con 0x7fc8ec1086a0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55134/0 (socket says 172.21.15.96:55134) 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8e3fff640 1 -- 172.21.15.96:0/2180355984 learned_addr learned my addr 172.21.15.96:0/2180355984 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8ead9d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec1086a0 0x7fc8ec199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.901+0000 7fc8ea59c640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 -- 172.21.15.96:0/2180355984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec1086a0 msgr2=0x7fc8ec199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec1086a0 0x7fc8ec199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 -- 172.21.15.96:0/2180355984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec06bee0 msgr2=0x7fc8ec198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 -- 172.21.15.96:0/2180355984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec19a830 con 0x7fc8ec105af0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e3fff640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c920 tx=0x7fc8d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4007c70 con 0x7fc8ec105af0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4007e10 con 0x7fc8ec105af0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec19ab20 con 0x7fc8ec105af0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4018820 con 0x7fc8ec105af0 2024-09-10T22:47:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.902+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a11f0 con 0x7fc8ec105af0 2024-09-10T22:47:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.903+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec0fec40 con 0x7fc8ec105af0 2024-09-10T22:47:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.904+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc8d400f450 con 0x7fc8ec105af0 2024-09-10T22:47:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.905+0000 7fc8e37fe640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.905+0000 7fc8ea59c640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.905+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc8d40e5f10 con 0x7fc8ec105af0 2024-09-10T22:47:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.905+0000 7fc8ea59c640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:14.908+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc8d4014030 con 0x7fc8ec105af0 2024-09-10T22:47:15.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:47:14.736+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:47:15.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:47:14.736+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:47:15.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:47:14.736+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:47:15.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:47:14.736+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:47:15.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 22:47:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T22:47:14.736+0000 7f5cee731640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:47:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.087+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8ec063d50 con 0x7fc8ec105af0 2024-09-10T22:47:15.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.089+0000 7fc8e37fe640 1 -- 172.21.15.96:0/2180355984 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7fc8d40acf00 con 0x7fc8ec105af0 2024-09-10T22:47:15.090 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:15.091 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:47:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 msgr2=0x7fc8d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc0314a0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 msgr2=0x7fc8ec199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c920 tx=0x7fc8d400cde0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 shutdown_connections 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8ec1086a0 0x7fc8ec199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8ec105af0 0x7fc8ec199340 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 --2- 172.21.15.96:0/2180355984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec06bee0 0x7fc8ec198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 >> 172.21.15.96:0/2180355984 conn(0x7fc8ec0fd150 msgr2=0x7fc8ec103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 shutdown_connections 2024-09-10T22:47:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.091+0000 7fc8f093d640 1 -- 172.21.15.96:0/2180355984 wait complete. 2024-09-10T22:47:15.104 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:40:37.540323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:45:15.242749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:45:17.587866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:47:15.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 -- 172.21.15.96:0/2812237647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 msgr2=0x7f98c0103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 --2- 172.21.15.96:0/2812237647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c0103f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009930 tx=0x7f98b402a420 comp rx=0 tx=0).stop 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 -- 172.21.15.96:0/2812237647 shutdown_connections 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 --2- 172.21.15.96:0/2812237647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0104470 0x7f98c010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 --2- 172.21.15.96:0/2812237647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c0103f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 --2- 172.21.15.96:0/2812237647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0102960 0x7f98c0102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 -- 172.21.15.96:0/2812237647 >> 172.21.15.96:0/2812237647 conn(0x7f98c00fe220 msgr2=0x7f98c0100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 -- 172.21.15.96:0/2812237647 shutdown_connections 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.206+0000 7f98c49df640 1 -- 172.21.15.96:0/2812237647 wait complete. 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 Processor -- start 2024-09-10T22:47:15.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 -- start start 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0102960 0x7f98c0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0104470 0x7f98c019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010e5d0 con 0x7f98c0103ab0 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010e450 con 0x7f98c0102960 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98c49df640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010e750 con 0x7f98c0104470 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55146/0 (socket says 172.21.15.96:55146) 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 -- 172.21.15.96:0/3805260725 learned_addr learned my addr 172.21.15.96:0/3805260725 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98be59c640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0102960 0x7f98c0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bed9d640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0104470 0x7f98c019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 -- 172.21.15.96:0/3805260725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0104470 msgr2=0x7f98c019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0104470 0x7f98c019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 -- 172.21.15.96:0/3805260725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0102960 msgr2=0x7f98c0198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0102960 0x7f98c0198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 -- 172.21.15.96:0/3805260725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c019a8c0 con 0x7f98c0103ab0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.207+0000 7f98bdd9b640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009a50 tx=0x7f98b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.208+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b403f030 con 0x7f98c0103ab0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.208+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b4003cb0 con 0x7f98c0103ab0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.208+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4035070 con 0x7f98c0103ab0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.208+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c019ab50 con 0x7f98c0103ab0 2024-09-10T22:47:15.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.208+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01a10a0 con 0x7f98c0103ab0 2024-09-10T22:47:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.209+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c0109cb0 con 0x7f98c0103ab0 2024-09-10T22:47:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.211+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f98b403bb90 con 0x7f98c0103ab0 2024-09-10T22:47:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.211+0000 7f98af7fe640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 0x7f98a80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.212+0000 7f98be59c640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 0x7f98a80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.212+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98b41067e0 con 0x7f98c0103ab0 2024-09-10T22:47:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.212+0000 7f98be59c640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 0x7f98a80b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009bd0 tx=0x7f98b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.215+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f98b40cd740 con 0x7f98c0103ab0 2024-09-10T22:47:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.391+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98c002e810 con 0x7f98c0103ab0 2024-09-10T22:47:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.392+0000 7f98af7fe640 1 -- 172.21.15.96:0/3805260725 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f98b40cd040 con 0x7f98c0103ab0 2024-09-10T22:47:15.393 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-10T22:47:15.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 msgr2=0x7f98a80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 0x7f98a80b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009bd0 tx=0x7f98b0005f90 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 msgr2=0x7f98c01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009a50 tx=0x7f98b40042f0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 shutdown_connections 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a80b5360 0x7f98a80b7820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98c0104470 0x7f98c019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0103ab0 0x7f98c01993d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 --2- 172.21.15.96:0/3805260725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98c0102960 0x7f98c0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 >> 172.21.15.96:0/3805260725 conn(0x7f98c00fe220 msgr2=0x7f98c0106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 shutdown_connections 2024-09-10T22:47:15.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.394+0000 7f98c49df640 1 -- 172.21.15.96:0/3805260725 wait complete. 2024-09-10T22:47:15.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- 172.21.15.96:0/1534720278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c1029b0 msgr2=0x7fda5c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- 172.21.15.96:0/1534720278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c1029b0 0x7fda5c102e30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda50009930 tx=0x7fda5002a420 comp rx=0 tx=0).stop 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- 172.21.15.96:0/1534720278 shutdown_connections 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- 172.21.15.96:0/1534720278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda5c103370 0x7fda5c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- 172.21.15.96:0/1534720278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c1029b0 0x7fda5c102e30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- 172.21.15.96:0/1534720278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c101860 0x7fda5c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- 172.21.15.96:0/1534720278 >> 172.21.15.96:0/1534720278 conn(0x7fda5c0fd140 msgr2=0x7fda5c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- 172.21.15.96:0/1534720278 shutdown_connections 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- 172.21.15.96:0/1534720278 wait complete. 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 Processor -- start 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- start start 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c1029b0 0x7fda5c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda5c103370 0x7fda5c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d4d0 con 0x7fda5c101860 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d350 con 0x7fda5c103370 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.508+0000 7fda643c6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d650 con 0x7fda5c1029b0 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55168/0 (socket says 172.21.15.96:55168) 2024-09-10T22:47:15.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 -- 172.21.15.96:0/3396004320 learned_addr learned my addr 172.21.15.96:0/3396004320 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda61961640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c1029b0 0x7fda5c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 -- 172.21.15.96:0/3396004320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c1029b0 msgr2=0x7fda5c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c1029b0 0x7fda5c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 -- 172.21.15.96:0/3396004320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda5c103370 msgr2=0x7fda5c1a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda5c103370 0x7fda5c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 -- 172.21.15.96:0/3396004320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c1a09d0 con 0x7fda5c101860 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda62162640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda5400e9f0 tx=0x7fda5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5400cd80 con 0x7fda5c101860 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c1a0c60 con 0x7fda5c101860 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda540054d0 con 0x7fda5c101860 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda54010690 con 0x7fda5c101860 2024-09-10T22:47:15.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.509+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1a11b0 con 0x7fda5c101860 2024-09-10T22:47:15.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.510+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c1a1380 con 0x7fda5c101860 2024-09-10T22:47:15.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.511+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fda54004e90 con 0x7fda5c101860 2024-09-10T22:47:15.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.512+0000 7fda5b7fe640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 0x7fda440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.512+0000 7fda61961640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 0x7fda440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.512+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fda540e6070 con 0x7fda5c101860 2024-09-10T22:47:15.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.513+0000 7fda61961640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 0x7fda440b77a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda500314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.516+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fda540acfd0 con 0x7fda5c101860 2024-09-10T22:47:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.690+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda5c1a1550 con 0x7fda5c101860 2024-09-10T22:47:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.690+0000 7fda5b7fe640 1 -- 172.21.15.96:0/3396004320 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fda540ac8d0 con 0x7fda5c101860 2024-09-10T22:47:15.692 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 msgr2=0x7fda440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 0x7fda440b77a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda500314a0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 msgr2=0x7fda5c06acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda5400e9f0 tx=0x7fda5400edf0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 shutdown_connections 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda5c103370 0x7fda5c1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda440b52e0 0x7fda440b77a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda5c1029b0 0x7fda5c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 --2- 172.21.15.96:0/3396004320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda5c101860 0x7fda5c06acf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 >> 172.21.15.96:0/3396004320 conn(0x7fda5c0fd140 msgr2=0x7fda5c103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 shutdown_connections 2024-09-10T22:47:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.692+0000 7fda643c6640 1 -- 172.21.15.96:0/3396004320 wait complete. 2024-09-10T22:47:15.703 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T22:47:15.703 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:15.703 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:15.703 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2180355984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:15.704 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:47:15.704 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:15 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3805260725' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:47:15.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T22:47:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T22:47:15.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:15.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:15.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2180355984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:15.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:47:15.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:15 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3805260725' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:47:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3057133502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T22:47:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:47:15.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2180355984' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:15.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:47:15.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:15 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3805260725' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.807+0000 7fd7791f8640 1 -- 172.21.15.96:0/1645118477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 msgr2=0x7fd774101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.807+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1645118477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774101c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd764009930 tx=0x7fd76402a420 comp rx=0 tx=0).stop 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 -- 172.21.15.96:0/1645118477 shutdown_connections 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1645118477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1645118477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 0x7fd774102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1645118477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774101c80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 -- 172.21.15.96:0/1645118477 >> 172.21.15.96:0/1645118477 conn(0x7fd7740fd120 msgr2=0x7fd7740ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 -- 172.21.15.96:0/1645118477 shutdown_connections 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 -- 172.21.15.96:0/1645118477 wait complete. 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 Processor -- start 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 -- start start 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.808+0000 7fd7791f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 0x7fd774198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410d4f0 con 0x7fd774101880 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410d370 con 0x7fd774103390 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410d670 con 0x7fd7741029d0 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55182/0 (socket says 172.21.15.96:55182) 2024-09-10T22:47:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 -- 172.21.15.96:0/1279409338 learned_addr learned my addr 172.21.15.96:0/1279409338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd76bfff640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 0x7fd774198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd77359e640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 -- 172.21.15.96:0/1279409338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 msgr2=0x7fd774198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 0x7fd774198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 -- 172.21.15.96:0/1279409338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 msgr2=0x7fd774198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 -- 172.21.15.96:0/1279409338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd774199810 con 0x7fd774101880 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd77359e640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd772d9d640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd764009a50 tx=0x7fd7640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76403f030 con 0x7fd774101880 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764003cb0 con 0x7fd774101880 2024-09-10T22:47:15.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd774199aa0 con 0x7fd774101880 2024-09-10T22:47:15.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764035070 con 0x7fd774101880 2024-09-10T22:47:15.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.809+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7741a01d0 con 0x7fd774101880 2024-09-10T22:47:15.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.810+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd774101c80 con 0x7fd774101880 2024-09-10T22:47:15.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.812+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd76403bb90 con 0x7fd774101880 2024-09-10T22:47:15.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.812+0000 7fd770d99640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 0x7fd7600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:15.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.812+0000 7fd76bfff640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 0x7fd7600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:15.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.812+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd7641068e0 con 0x7fd774101880 2024-09-10T22:47:15.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.813+0000 7fd76bfff640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 0x7fd7600b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009b60 tx=0x7fd75c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:15.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.815+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7640cd840 con 0x7fd774101880 2024-09-10T22:47:15.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.988+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd774063d50 con 0x7fd774101880 2024-09-10T22:47:15.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.989+0000 7fd770d99640 1 -- 172.21.15.96:0/1279409338 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7fd7640cd140 con 0x7fd774101880 2024-09-10T22:47:15.991 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 msgr2=0x7fd7600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 0x7fd7600b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009b60 tx=0x7fd75c005f90 comp rx=0 tx=0).stop 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 msgr2=0x7fd774197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd764009a50 tx=0x7fd7640042f0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 shutdown_connections 2024-09-10T22:47:15.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd774103390 0x7fd774198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd7600b52e0 0x7fd7600b77a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7741029d0 0x7fd774198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 --2- 172.21.15.96:0/1279409338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd774101880 0x7fd774197cd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 >> 172.21.15.96:0/1279409338 conn(0x7fd7740fd120 msgr2=0x7fd774105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 shutdown_connections 2024-09-10T22:47:15.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:15.991+0000 7fd7791f8640 1 -- 172.21.15.96:0/1279409338 wait complete. 2024-09-10T22:47:16.004 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T22:47:16.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:47:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:16.637 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:16 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3396004320' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.637 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:16 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1279409338' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 -- 172.21.15.96:0/3122516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c1076a0 msgr2=0x7f952c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 --2- 172.21.15.96:0/3122516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c1076a0 0x7f952c107ae0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9528009900 tx=0x7f952802a410 comp rx=0 tx=0).stop 2024-09-10T22:47:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 -- 172.21.15.96:0/3122516773 shutdown_connections 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 --2- 172.21.15.96:0/3122516773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c108020 0x7f952c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 --2- 172.21.15.96:0/3122516773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c1076a0 0x7f952c107ae0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 --2- 172.21.15.96:0/3122516773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f952c101c90 0x7f952c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 -- 172.21.15.96:0/3122516773 >> 172.21.15.96:0/3122516773 conn(0x7f952c0fd700 msgr2=0x7f952c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 -- 172.21.15.96:0/3122516773 shutdown_connections 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.716+0000 7f9533acf640 1 -- 172.21.15.96:0/3122516773 wait complete. 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 Processor -- start 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 -- start start 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c101c90 0x7f952c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f952c1076a0 0x7f952c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f952c112e00 con 0x7f952c108020 2024-09-10T22:47:16.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f952c112c80 con 0x7f952c1076a0 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f9533acf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f952c112f80 con 0x7f952c101c90 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34584/0 (socket says 172.21.15.96:34584) 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 -- 172.21.15.96:0/2266120237 learned_addr learned my addr 172.21.15.96:0/2266120237 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953186b640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c101c90 0x7f952c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 -- 172.21.15.96:0/2266120237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c101c90 msgr2=0x7f952c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c101c90 0x7f952c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 -- 172.21.15.96:0/2266120237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f952c1076a0 msgr2=0x7f952c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f952c1076a0 0x7f952c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.717+0000 7f953206c640 1 -- 172.21.15.96:0/2266120237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c19f370 con 0x7f952c108020 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f953206c640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f952400eaa0 tx=0x7f952400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524004d20 con 0x7f952c108020 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c19f660 con 0x7f952c108020 2024-09-10T22:47:16.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9524005020 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240116d0 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.718+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c1a5ef0 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.719+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952c102070 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.720+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f952400f040 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.721+0000 7f951effd640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 0x7f95180b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.721+0000 7f953186b640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 0x7f95180b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.721+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f95240e5a90 con 0x7f952c108020 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.721+0000 7f953186b640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 0x7f95180b7830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9520009990 tx=0x7f9520009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.725+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9524016030 con 0x7f952c108020 2024-09-10T22:47:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:16 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3396004320' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:16 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1279409338' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:16 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3396004320' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:16 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1279409338' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:47:16.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.939+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f952c02e5c0 con 0x7f952c108020 2024-09-10T22:47:16.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.940+0000 7f951effd640 1 -- 172.21.15.96:0/2266120237 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f95240ac8f0 con 0x7f952c108020 2024-09-10T22:47:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:16.942 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":28,"btime":"2024-09-10T22:47:14:394570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:47:13.445561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:47:16.942 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 28 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 msgr2=0x7f95180b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 0x7f95180b7830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9520009990 tx=0x7f9520009110 comp rx=0 tx=0).stop 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 msgr2=0x7f952c19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f952400eaa0 tx=0x7f952400eea0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 shutdown_connections 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f952c1076a0 0x7f952c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95180b5370 0x7f95180b7830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f952c108020 0x7f952c19eba0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 --2- 172.21.15.96:0/2266120237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f952c101c90 0x7f952c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:16.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 >> 172.21.15.96:0/2266120237 conn(0x7f952c0fd700 msgr2=0x7f952c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:16.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 shutdown_connections 2024-09-10T22:47:16.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:16.942+0000 7f9533acf640 1 -- 172.21.15.96:0/2266120237 wait complete. 2024-09-10T22:47:17.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:17 smithi096 ceph-mon[30869]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T22:47:17.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:17 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2266120237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:17.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:17 smithi096 ceph-mon[30869]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T22:47:17.733 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T22:47:17.733 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T22:47:17.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:47:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:17 smithi121 ceph-mon[35119]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T22:47:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:17 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2266120237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:17 smithi121 ceph-mon[35119]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T22:47:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:17 smithi117 ceph-mon[35081]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T22:47:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:17 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2266120237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:17 smithi117 ceph-mon[35081]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T22:47:17.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 -- 172.21.15.96:0/1242724616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301018a0 msgr2=0x7fdd30101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 --2- 172.21.15.96:0/1242724616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301018a0 0x7fdd30101ca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009930 tx=0x7fdd2002a420 comp rx=0 tx=0).stop 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 -- 172.21.15.96:0/1242724616 shutdown_connections 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 --2- 172.21.15.96:0/1242724616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301033b0 0x7fdd30109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 --2- 172.21.15.96:0/1242724616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301029f0 0x7fdd30102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 --2- 172.21.15.96:0/1242724616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301018a0 0x7fdd30101ca0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 -- 172.21.15.96:0/1242724616 >> 172.21.15.96:0/1242724616 conn(0x7fdd300fd180 msgr2=0x7fdd300ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 -- 172.21.15.96:0/1242724616 shutdown_connections 2024-09-10T22:47:17.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.839+0000 7fdd3516b640 1 -- 172.21.15.96:0/1242724616 wait complete. 2024-09-10T22:47:17.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 Processor -- start 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 -- start start 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 0x7fdd30198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 0x7fdd3019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3010e640 con 0x7fdd301029f0 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3010e4c0 con 0x7fdd301018a0 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd3516b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3010e7c0 con 0x7fdd301033b0 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34606/0 (socket says 172.21.15.96:34606) 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 -- 172.21.15.96:0/727453682 learned_addr learned my addr 172.21.15.96:0/727453682 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:17.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd2f59e640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 0x7fdd3019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd2ed9d640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 0x7fdd30198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 -- 172.21.15.96:0/727453682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 msgr2=0x7fdd3019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 0x7fdd3019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 -- 172.21.15.96:0/727453682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 msgr2=0x7fdd30198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 0x7fdd30198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 -- 172.21.15.96:0/727453682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3019a8f0 con 0x7fdd301029f0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd2f59e640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 0x7fdd3019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd2ed9d640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 0x7fdd30198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.840+0000 7fdd27fff640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800d770 tx=0x7fdd1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.841+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18017070 con 0x7fdd301029f0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.841+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd180054d0 con 0x7fdd301029f0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.841+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3019abe0 con 0x7fdd301029f0 2024-09-10T22:47:17.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.841+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18004c30 con 0x7fdd301029f0 2024-09-10T22:47:17.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.841+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd301a12b0 con 0x7fdd301029f0 2024-09-10T22:47:17.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.842+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd30108c90 con 0x7fdd301029f0 2024-09-10T22:47:17.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.843+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fdd18004dd0 con 0x7fdd301029f0 2024-09-10T22:47:17.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.844+0000 7fdd2cd99640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 0x7fdd1c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:17.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.844+0000 7fdd2ed9d640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 0x7fdd1c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:17.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.844+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdd180e5bf0 con 0x7fdd301029f0 2024-09-10T22:47:17.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.845+0000 7fdd2ed9d640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 0x7fdd1c0b7830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:17.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:17.847+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdd180aca50 con 0x7fdd301029f0 2024-09-10T22:47:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.019+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd30063590 con 0x7fdd1c0b5370 2024-09-10T22:47:18.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.023+0000 7fdd2cd99640 1 -- 172.21.15.96:0/727453682 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259005 (secure 0 0 0) 0x7fdd30063590 con 0x7fdd1c0b5370 2024-09-10T22:47:18.028 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:47:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 msgr2=0x7fdd1c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 0x7fdd1c0b7830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20000bc0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 msgr2=0x7fdd30199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800d770 tx=0x7fdd1800dc30 comp rx=0 tx=0).stop 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 shutdown_connections 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdd1c0b5370 0x7fdd1c0b7830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd301033b0 0x7fdd3019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd301029f0 0x7fdd30199400 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 --2- 172.21.15.96:0/727453682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd301018a0 0x7fdd30198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 >> 172.21.15.96:0/727453682 conn(0x7fdd300fd180 msgr2=0x7fdd30105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 shutdown_connections 2024-09-10T22:47:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.027+0000 7fdd3516b640 1 -- 172.21.15.96:0/727453682 wait complete. 2024-09-10T22:47:18.040 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T22:47:17.326794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657651,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1147,"num_read_kb":674,"num_write":376,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":809,"ondisk_log_size":809,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363824,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578000,"statfs":{"total":1151940427776,"available":1151567872000,"internally_reserved":0,"allocated":10665984,"data_stored":6830859,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553587+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:46:57.553587+0000","last_peered":"2024-09-10T22:46:57.553587+0000","last_clean":"2024-09-10T22:46:57.553587+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:46:57.553587+0000","last_undegraded":"2024-09-10T22:46:57.553587+0000","last_fullsized":"2024-09-10T22:46:57.553587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538919+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:46:57.538919+0000","last_peered":"2024-09-10T22:46:57.538919+0000","last_clean":"2024-09-10T22:46:57.538919+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:46:57.538919+0000","last_undegraded":"2024-09-10T22:46:57.538919+0000","last_fullsized":"2024-09-10T22:46:57.538919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550143+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:46:57.550143+0000","last_peered":"2024-09-10T22:46:57.550143+0000","last_clean":"2024-09-10T22:46:57.550143+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:46:57.550143+0000","last_undegraded":"2024-09-10T22:46:57.550143+0000","last_fullsized":"2024-09-10T22:46:57.550143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544008+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:46:57.544008+0000","last_peered":"2024-09-10T22:46:57.544008+0000","last_clean":"2024-09-10T22:46:57.544008+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:46:57.544008+0000","last_undegraded":"2024-09-10T22:46:57.544008+0000","last_fullsized":"2024-09-10T22:46:57.544008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.535399+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:46:57.535399+0000","last_peered":"2024-09-10T22:46:57.535399+0000","last_clean":"2024-09-10T22:46:57.535399+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:46:57.535399+0000","last_undegraded":"2024-09-10T22:46:57.535399+0000","last_fullsized":"2024-09-10T22:46:57.535399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549118+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:46:57.549118+0000","last_peered":"2024-09-10T22:46:57.549118+0000","last_clean":"2024-09-10T22:46:57.549118+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:46:57.549118+0000","last_undegraded":"2024-09-10T22:46:57.549118+0000","last_fullsized":"2024-09-10T22:46:57.549118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558179+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:46:57.558179+0000","last_peered":"2024-09-10T22:46:57.558179+0000","last_clean":"2024-09-10T22:46:57.558179+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:46:57.558179+0000","last_undegraded":"2024-09-10T22:46:57.558179+0000","last_fullsized":"2024-09-10T22:46:57.558179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542594+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:46:57.542594+0000","last_peered":"2024-09-10T22:46:57.542594+0000","last_clean":"2024-09-10T22:46:57.542594+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:46:57.542594+0000","last_undegraded":"2024-09-10T22:46:57.542594+0000","last_fullsized":"2024-09-10T22:46:57.542594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557493+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:46:57.557493+0000","last_peered":"2024-09-10T22:46:57.557493+0000","last_clean":"2024-09-10T22:46:57.557493+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:46:57.557493+0000","last_undegraded":"2024-09-10T22:46:57.557493+0000","last_fullsized":"2024-09-10T22:46:57.557493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542611+0000","last_change":"2024-09-10T22:45:58.304922+0000","last_active":"2024-09-10T22:46:57.542611+0000","last_peered":"2024-09-10T22:46:57.542611+0000","last_clean":"2024-09-10T22:46:57.542611+0000","last_became_active":"2024-09-10T22:45:58.304541+0000","last_became_peered":"2024-09-10T22:45:58.304541+0000","last_unstale":"2024-09-10T22:46:57.542611+0000","last_undegraded":"2024-09-10T22:46:57.542611+0000","last_fullsized":"2024-09-10T22:46:57.542611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:09.043873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557385+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:46:57.557385+0000","last_peered":"2024-09-10T22:46:57.557385+0000","last_clean":"2024-09-10T22:46:57.557385+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:46:57.557385+0000","last_undegraded":"2024-09-10T22:46:57.557385+0000","last_fullsized":"2024-09-10T22:46:57.557385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.536823+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:46:57.536823+0000","last_peered":"2024-09-10T22:46:57.536823+0000","last_clean":"2024-09-10T22:46:57.536823+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:46:57.536823+0000","last_undegraded":"2024-09-10T22:46:57.536823+0000","last_fullsized":"2024-09-10T22:46:57.536823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545373+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:46:57.545373+0000","last_peered":"2024-09-10T22:46:57.545373+0000","last_clean":"2024-09-10T22:46:57.545373+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:46:57.545373+0000","last_undegraded":"2024-09-10T22:46:57.545373+0000","last_fullsized":"2024-09-10T22:46:57.545373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.537087+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:46:57.537087+0000","last_peered":"2024-09-10T22:46:57.537087+0000","last_clean":"2024-09-10T22:46:57.537087+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:46:57.537087+0000","last_undegraded":"2024-09-10T22:46:57.537087+0000","last_fullsized":"2024-09-10T22:46:57.537087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542024+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:46:57.542024+0000","last_peered":"2024-09-10T22:46:57.542024+0000","last_clean":"2024-09-10T22:46:57.542024+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:46:57.542024+0000","last_undegraded":"2024-09-10T22:46:57.542024+0000","last_fullsized":"2024-09-10T22:46:57.542024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555782+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:46:57.555782+0000","last_peered":"2024-09-10T22:46:57.555782+0000","last_clean":"2024-09-10T22:46:57.555782+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:46:57.555782+0000","last_undegraded":"2024-09-10T22:46:57.555782+0000","last_fullsized":"2024-09-10T22:46:57.555782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553633+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:46:57.553633+0000","last_peered":"2024-09-10T22:46:57.553633+0000","last_clean":"2024-09-10T22:46:57.553633+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:46:57.553633+0000","last_undegraded":"2024-09-10T22:46:57.553633+0000","last_fullsized":"2024-09-10T22:46:57.553633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554932+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:46:57.554932+0000","last_peered":"2024-09-10T22:46:57.554932+0000","last_clean":"2024-09-10T22:46:57.554932+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:46:57.554932+0000","last_undegraded":"2024-09-10T22:46:57.554932+0000","last_fullsized":"2024-09-10T22:46:57.554932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541118+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:46:57.541118+0000","last_peered":"2024-09-10T22:46:57.541118+0000","last_clean":"2024-09-10T22:46:57.541118+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:46:57.541118+0000","last_undegraded":"2024-09-10T22:46:57.541118+0000","last_fullsized":"2024-09-10T22:46:57.541118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549379+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:46:57.549379+0000","last_peered":"2024-09-10T22:46:57.549379+0000","last_clean":"2024-09-10T22:46:57.549379+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:46:57.549379+0000","last_undegraded":"2024-09-10T22:46:57.549379+0000","last_fullsized":"2024-09-10T22:46:57.549379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552691+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:46:57.552691+0000","last_peered":"2024-09-10T22:46:57.552691+0000","last_clean":"2024-09-10T22:46:57.552691+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:46:57.552691+0000","last_undegraded":"2024-09-10T22:46:57.552691+0000","last_fullsized":"2024-09-10T22:46:57.552691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556630+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:46:57.556630+0000","last_peered":"2024-09-10T22:46:57.556630+0000","last_clean":"2024-09-10T22:46:57.556630+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:46:57.556630+0000","last_undegraded":"2024-09-10T22:46:57.556630+0000","last_fullsized":"2024-09-10T22:46:57.556630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540681+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:46:57.540681+0000","last_peered":"2024-09-10T22:46:57.540681+0000","last_clean":"2024-09-10T22:46:57.540681+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:46:57.540681+0000","last_undegraded":"2024-09-10T22:46:57.540681+0000","last_fullsized":"2024-09-10T22:46:57.540681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543300+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:46:57.543300+0000","last_peered":"2024-09-10T22:46:57.543300+0000","last_clean":"2024-09-10T22:46:57.543300+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:46:57.543300+0000","last_undegraded":"2024-09-10T22:46:57.543300+0000","last_fullsized":"2024-09-10T22:46:57.543300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538562+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:46:57.538562+0000","last_peered":"2024-09-10T22:46:57.538562+0000","last_clean":"2024-09-10T22:46:57.538562+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:46:57.538562+0000","last_undegraded":"2024-09-10T22:46:57.538562+0000","last_fullsized":"2024-09-10T22:46:57.538562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551195+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:46:57.551195+0000","last_peered":"2024-09-10T22:46:57.551195+0000","last_clean":"2024-09-10T22:46:57.551195+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:46:57.551195+0000","last_undegraded":"2024-09-10T22:46:57.551195+0000","last_fullsized":"2024-09-10T22:46:57.551195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541587+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:46:57.541587+0000","last_peered":"2024-09-10T22:46:57.541587+0000","last_clean":"2024-09-10T22:46:57.541587+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:46:57.541587+0000","last_undegraded":"2024-09-10T22:46:57.541587+0000","last_fullsized":"2024-09-10T22:46:57.541587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543325+0000","last_change":"2024-09-10T22:45:58.301222+0000","last_active":"2024-09-10T22:46:57.543325+0000","last_peered":"2024-09-10T22:46:57.543325+0000","last_clean":"2024-09-10T22:46:57.543325+0000","last_became_active":"2024-09-10T22:45:58.300835+0000","last_became_peered":"2024-09-10T22:45:58.300835+0000","last_unstale":"2024-09-10T22:46:57.543325+0000","last_undegraded":"2024-09-10T22:46:57.543325+0000","last_fullsized":"2024-09-10T22:46:57.543325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.772354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541083+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:46:57.541083+0000","last_peered":"2024-09-10T22:46:57.541083+0000","last_clean":"2024-09-10T22:46:57.541083+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:46:57.541083+0000","last_undegraded":"2024-09-10T22:46:57.541083+0000","last_fullsized":"2024-09-10T22:46:57.541083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538073+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:46:57.538073+0000","last_peered":"2024-09-10T22:46:57.538073+0000","last_clean":"2024-09-10T22:46:57.538073+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:46:57.538073+0000","last_undegraded":"2024-09-10T22:46:57.538073+0000","last_fullsized":"2024-09-10T22:46:57.538073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540443+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:46:57.540443+0000","last_peered":"2024-09-10T22:46:57.540443+0000","last_clean":"2024-09-10T22:46:57.540443+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:46:57.540443+0000","last_undegraded":"2024-09-10T22:46:57.540443+0000","last_fullsized":"2024-09-10T22:46:57.540443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545371+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:46:57.545371+0000","last_peered":"2024-09-10T22:46:57.545371+0000","last_clean":"2024-09-10T22:46:57.545371+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:46:57.545371+0000","last_undegraded":"2024-09-10T22:46:57.545371+0000","last_fullsized":"2024-09-10T22:46:57.545371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542915+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:46:57.542915+0000","last_peered":"2024-09-10T22:46:57.542915+0000","last_clean":"2024-09-10T22:46:57.542915+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:46:57.542915+0000","last_undegraded":"2024-09-10T22:46:57.542915+0000","last_fullsized":"2024-09-10T22:46:57.542915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.197033+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:47:11.197033+0000","last_peered":"2024-09-10T22:47:11.197033+0000","last_clean":"2024-09-10T22:47:11.197033+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:47:11.197033+0000","last_undegraded":"2024-09-10T22:47:11.197033+0000","last_fullsized":"2024-09-10T22:47:11.197033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549257+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:46:57.549257+0000","last_peered":"2024-09-10T22:46:57.549257+0000","last_clean":"2024-09-10T22:46:57.549257+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:46:57.549257+0000","last_undegraded":"2024-09-10T22:46:57.549257+0000","last_fullsized":"2024-09-10T22:46:57.549257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.306152+0000","last_change":"2024-09-10T22:45:58.306070+0000","last_active":"2024-09-10T22:45:58.306152+0000","last_peered":"2024-09-10T22:45:58.306152+0000","last_clean":"2024-09-10T22:45:58.306152+0000","last_became_active":"2024-09-10T22:45:58.300281+0000","last_became_peered":"2024-09-10T22:45:58.300281+0000","last_unstale":"2024-09-10T22:45:58.306152+0000","last_undegraded":"2024-09-10T22:45:58.306152+0000","last_fullsized":"2024-09-10T22:45:58.306152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:01.832411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297848+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:58.297848+0000","last_peered":"2024-09-10T22:45:58.297848+0000","last_clean":"2024-09-10T22:45:58.297848+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:58.297848+0000","last_undegraded":"2024-09-10T22:45:58.297848+0000","last_fullsized":"2024-09-10T22:45:58.297848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543952+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:46:57.543952+0000","last_peered":"2024-09-10T22:46:57.543952+0000","last_clean":"2024-09-10T22:46:57.543952+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:46:57.543952+0000","last_undegraded":"2024-09-10T22:46:57.543952+0000","last_fullsized":"2024-09-10T22:46:57.543952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.546120+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:46:57.546120+0000","last_peered":"2024-09-10T22:46:57.546120+0000","last_clean":"2024-09-10T22:46:57.546120+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:46:57.546120+0000","last_undegraded":"2024-09-10T22:46:57.546120+0000","last_fullsized":"2024-09-10T22:46:57.546120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":233,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.198974+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:47:13.198974+0000","last_peered":"2024-09-10T22:47:13.198974+0000","last_clean":"2024-09-10T22:47:13.198974+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:47:13.198974+0000","last_undegraded":"2024-09-10T22:47:13.198974+0000","last_fullsized":"2024-09-10T22:47:13.198974+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551796+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:46:57.551796+0000","last_peered":"2024-09-10T22:46:57.551796+0000","last_clean":"2024-09-10T22:46:57.551796+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:46:57.551796+0000","last_undegraded":"2024-09-10T22:46:57.551796+0000","last_fullsized":"2024-09-10T22:46:57.551796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680757+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:58.680757+0000","last_peered":"2024-09-10T22:45:58.680757+0000","last_clean":"2024-09-10T22:45:58.680757+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:58.680757+0000","last_undegraded":"2024-09-10T22:45:58.680757+0000","last_fullsized":"2024-09-10T22:45:58.680757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541828+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:46:57.541828+0000","last_peered":"2024-09-10T22:46:57.541828+0000","last_clean":"2024-09-10T22:46:57.541828+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:46:57.541828+0000","last_undegraded":"2024-09-10T22:46:57.541828+0000","last_fullsized":"2024-09-10T22:46:57.541828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672340+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:58.672340+0000","last_peered":"2024-09-10T22:45:58.672340+0000","last_clean":"2024-09-10T22:45:58.672340+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:58.672340+0000","last_undegraded":"2024-09-10T22:45:58.672340+0000","last_fullsized":"2024-09-10T22:45:58.672340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556911+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:46:57.556911+0000","last_peered":"2024-09-10T22:46:57.556911+0000","last_clean":"2024-09-10T22:46:57.556911+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:46:57.556911+0000","last_undegraded":"2024-09-10T22:46:57.556911+0000","last_fullsized":"2024-09-10T22:46:57.556911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199739+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:47:13.199739+0000","last_peered":"2024-09-10T22:47:13.199739+0000","last_clean":"2024-09-10T22:47:13.199739+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:47:13.199739+0000","last_undegraded":"2024-09-10T22:47:13.199739+0000","last_fullsized":"2024-09-10T22:47:13.199739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60319,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":217,"num_read_kb":167,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555331+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:46:57.555331+0000","last_peered":"2024-09-10T22:46:57.555331+0000","last_clean":"2024-09-10T22:46:57.555331+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:46:57.555331+0000","last_undegraded":"2024-09-10T22:46:57.555331+0000","last_fullsized":"2024-09-10T22:46:57.555331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303539+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:58.303539+0000","last_peered":"2024-09-10T22:45:58.303539+0000","last_clean":"2024-09-10T22:45:58.303539+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:58.303539+0000","last_undegraded":"2024-09-10T22:45:58.303539+0000","last_fullsized":"2024-09-10T22:45:58.303539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552322+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:46:57.552322+0000","last_peered":"2024-09-10T22:46:57.552322+0000","last_clean":"2024-09-10T22:46:57.552322+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:46:57.552322+0000","last_undegraded":"2024-09-10T22:46:57.552322+0000","last_fullsized":"2024-09-10T22:46:57.552322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556243+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:46:57.556243+0000","last_peered":"2024-09-10T22:46:57.556243+0000","last_clean":"2024-09-10T22:46:57.556243+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:46:57.556243+0000","last_undegraded":"2024-09-10T22:46:57.556243+0000","last_fullsized":"2024-09-10T22:46:57.556243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:12.208490+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:47:12.208490+0000","last_peered":"2024-09-10T22:47:12.208490+0000","last_clean":"2024-09-10T22:47:12.208490+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:47:12.208490+0000","last_undegraded":"2024-09-10T22:47:12.208490+0000","last_fullsized":"2024-09-10T22:47:12.208490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558382+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:46:57.558382+0000","last_peered":"2024-09-10T22:46:57.558382+0000","last_clean":"2024-09-10T22:46:57.558382+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:46:57.558382+0000","last_undegraded":"2024-09-10T22:46:57.558382+0000","last_fullsized":"2024-09-10T22:46:57.558382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298597+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:58.298597+0000","last_peered":"2024-09-10T22:45:58.298597+0000","last_clean":"2024-09-10T22:45:58.298597+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:58.298597+0000","last_undegraded":"2024-09-10T22:45:58.298597+0000","last_fullsized":"2024-09-10T22:45:58.298597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556870+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:46:57.556870+0000","last_peered":"2024-09-10T22:46:57.556870+0000","last_clean":"2024-09-10T22:46:57.556870+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:46:57.556870+0000","last_undegraded":"2024-09-10T22:46:57.556870+0000","last_fullsized":"2024-09-10T22:46:57.556870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299286+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:58.299286+0000","last_peered":"2024-09-10T22:45:58.299286+0000","last_clean":"2024-09-10T22:45:58.299286+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:58.299286+0000","last_undegraded":"2024-09-10T22:45:58.299286+0000","last_fullsized":"2024-09-10T22:45:58.299286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.548879+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:46:57.548879+0000","last_peered":"2024-09-10T22:46:57.548879+0000","last_clean":"2024-09-10T22:46:57.548879+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:46:57.548879+0000","last_undegraded":"2024-09-10T22:46:57.548879+0000","last_fullsized":"2024-09-10T22:46:57.548879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298492+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:58.298492+0000","last_peered":"2024-09-10T22:45:58.298492+0000","last_clean":"2024-09-10T22:45:58.298492+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:58.298492+0000","last_undegraded":"2024-09-10T22:45:58.298492+0000","last_fullsized":"2024-09-10T22:45:58.298492+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558447+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:46:57.558447+0000","last_peered":"2024-09-10T22:46:57.558447+0000","last_clean":"2024-09-10T22:46:57.558447+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:46:57.558447+0000","last_undegraded":"2024-09-10T22:46:57.558447+0000","last_fullsized":"2024-09-10T22:46:57.558447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672395+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:58.672395+0000","last_peered":"2024-09-10T22:45:58.672395+0000","last_clean":"2024-09-10T22:45:58.672395+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:58.672395+0000","last_undegraded":"2024-09-10T22:45:58.672395+0000","last_fullsized":"2024-09-10T22:45:58.672395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554711+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:46:57.554711+0000","last_peered":"2024-09-10T22:46:57.554711+0000","last_clean":"2024-09-10T22:46:57.554711+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:46:57.554711+0000","last_undegraded":"2024-09-10T22:46:57.554711+0000","last_fullsized":"2024-09-10T22:46:57.554711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.196964+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:47:11.196964+0000","last_peered":"2024-09-10T22:47:11.196964+0000","last_clean":"2024-09-10T22:47:11.196964+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:47:11.196964+0000","last_undegraded":"2024-09-10T22:47:11.196964+0000","last_fullsized":"2024-09-10T22:47:11.196964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557894+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:46:57.557894+0000","last_peered":"2024-09-10T22:46:57.557894+0000","last_clean":"2024-09-10T22:46:57.557894+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:46:57.557894+0000","last_undegraded":"2024-09-10T22:46:57.557894+0000","last_fullsized":"2024-09-10T22:46:57.557894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680641+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:58.680641+0000","last_peered":"2024-09-10T22:45:58.680641+0000","last_clean":"2024-09-10T22:45:58.680641+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:58.680641+0000","last_undegraded":"2024-09-10T22:45:58.680641+0000","last_fullsized":"2024-09-10T22:45:58.680641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296623+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:58.296623+0000","last_peered":"2024-09-10T22:45:58.296623+0000","last_clean":"2024-09-10T22:45:58.296623+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:58.296623+0000","last_undegraded":"2024-09-10T22:45:58.296623+0000","last_fullsized":"2024-09-10T22:45:58.296623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550173+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:46:57.550173+0000","last_peered":"2024-09-10T22:46:57.550173+0000","last_clean":"2024-09-10T22:46:57.550173+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:46:57.550173+0000","last_undegraded":"2024-09-10T22:46:57.550173+0000","last_fullsized":"2024-09-10T22:46:57.550173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680509+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:58.680509+0000","last_peered":"2024-09-10T22:45:58.680509+0000","last_clean":"2024-09-10T22:45:58.680509+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:58.680509+0000","last_undegraded":"2024-09-10T22:45:58.680509+0000","last_fullsized":"2024-09-10T22:45:58.680509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544946+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:46:57.544946+0000","last_peered":"2024-09-10T22:46:57.544946+0000","last_clean":"2024-09-10T22:46:57.544946+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:46:57.544946+0000","last_undegraded":"2024-09-10T22:46:57.544946+0000","last_fullsized":"2024-09-10T22:46:57.544946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:14.210466+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:47:14.210466+0000","last_peered":"2024-09-10T22:47:14.210466+0000","last_clean":"2024-09-10T22:47:14.210466+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:47:14.210466+0000","last_undegraded":"2024-09-10T22:47:14.210466+0000","last_fullsized":"2024-09-10T22:47:14.210466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544012+0000","last_change":"2024-09-10T22:45:58.304503+0000","last_active":"2024-09-10T22:46:57.544012+0000","last_peered":"2024-09-10T22:46:57.544012+0000","last_clean":"2024-09-10T22:46:57.544012+0000","last_became_active":"2024-09-10T22:45:58.304243+0000","last_became_peered":"2024-09-10T22:45:58.304243+0000","last_unstale":"2024-09-10T22:46:57.544012+0000","last_undegraded":"2024-09-10T22:46:57.544012+0000","last_fullsized":"2024-09-10T22:46:57.544012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:33.190092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"77'6","reported_seq":135,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199731+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:47:13.199731+0000","last_peered":"2024-09-10T22:47:13.199731+0000","last_clean":"2024-09-10T22:47:13.199731+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:47:13.199731+0000","last_undegraded":"2024-09-10T22:47:13.199731+0000","last_fullsized":"2024-09-10T22:47:13.199731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549451+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:46:57.549451+0000","last_peered":"2024-09-10T22:46:57.549451+0000","last_clean":"2024-09-10T22:46:57.549451+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:46:57.549451+0000","last_undegraded":"2024-09-10T22:46:57.549451+0000","last_fullsized":"2024-09-10T22:46:57.549451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.599710+0000","last_change":"2024-09-10T22:45:58.599612+0000","last_active":"2024-09-10T22:45:58.599710+0000","last_peered":"2024-09-10T22:45:58.599710+0000","last_clean":"2024-09-10T22:45:58.599710+0000","last_became_active":"2024-09-10T22:45:58.599226+0000","last_became_peered":"2024-09-10T22:45:58.599226+0000","last_unstale":"2024-09-10T22:45:58.599710+0000","last_undegraded":"2024-09-10T22:45:58.599710+0000","last_fullsized":"2024-09-10T22:45:58.599710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.112571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544799+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:46:57.544799+0000","last_peered":"2024-09-10T22:46:57.544799+0000","last_clean":"2024-09-10T22:46:57.544799+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:46:57.544799+0000","last_undegraded":"2024-09-10T22:46:57.544799+0000","last_fullsized":"2024-09-10T22:46:57.544799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299088+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:58.299088+0000","last_peered":"2024-09-10T22:45:58.299088+0000","last_clean":"2024-09-10T22:45:58.299088+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:58.299088+0000","last_undegraded":"2024-09-10T22:45:58.299088+0000","last_fullsized":"2024-09-10T22:45:58.299088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542853+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:46:57.542853+0000","last_peered":"2024-09-10T22:46:57.542853+0000","last_clean":"2024-09-10T22:46:57.542853+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:46:57.542853+0000","last_undegraded":"2024-09-10T22:46:57.542853+0000","last_fullsized":"2024-09-10T22:46:57.542853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297683+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:58.297683+0000","last_peered":"2024-09-10T22:45:58.297683+0000","last_clean":"2024-09-10T22:45:58.297683+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:58.297683+0000","last_undegraded":"2024-09-10T22:45:58.297683+0000","last_fullsized":"2024-09-10T22:45:58.297683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556425+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:46:57.556425+0000","last_peered":"2024-09-10T22:46:57.556425+0000","last_clean":"2024-09-10T22:46:57.556425+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:46:57.556425+0000","last_undegraded":"2024-09-10T22:46:57.556425+0000","last_fullsized":"2024-09-10T22:46:57.556425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296692+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:58.296692+0000","last_peered":"2024-09-10T22:45:58.296692+0000","last_clean":"2024-09-10T22:45:58.296692+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:58.296692+0000","last_undegraded":"2024-09-10T22:45:58.296692+0000","last_fullsized":"2024-09-10T22:45:58.296692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544837+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:46:57.544837+0000","last_peered":"2024-09-10T22:46:57.544837+0000","last_clean":"2024-09-10T22:46:57.544837+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:46:57.544837+0000","last_undegraded":"2024-09-10T22:46:57.544837+0000","last_fullsized":"2024-09-10T22:46:57.544837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.602110+0000","last_change":"2024-09-10T22:45:58.602021+0000","last_active":"2024-09-10T22:45:58.602110+0000","last_peered":"2024-09-10T22:45:58.602110+0000","last_clean":"2024-09-10T22:45:58.602110+0000","last_became_active":"2024-09-10T22:45:58.599358+0000","last_became_peered":"2024-09-10T22:45:58.599358+0000","last_unstale":"2024-09-10T22:45:58.602110+0000","last_undegraded":"2024-09-10T22:45:58.602110+0000","last_fullsized":"2024-09-10T22:45:58.602110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:40.510120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.530711+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:46:57.530711+0000","last_peered":"2024-09-10T22:46:57.530711+0000","last_clean":"2024-09-10T22:46:57.530711+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:46:57.530711+0000","last_undegraded":"2024-09-10T22:46:57.530711+0000","last_fullsized":"2024-09-10T22:46:57.530711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680327+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:58.680327+0000","last_peered":"2024-09-10T22:45:58.680327+0000","last_clean":"2024-09-10T22:45:58.680327+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:58.680327+0000","last_undegraded":"2024-09-10T22:45:58.680327+0000","last_fullsized":"2024-09-10T22:45:58.680327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544878+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:46:57.544878+0000","last_peered":"2024-09-10T22:46:57.544878+0000","last_clean":"2024-09-10T22:46:57.544878+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:46:57.544878+0000","last_undegraded":"2024-09-10T22:46:57.544878+0000","last_fullsized":"2024-09-10T22:46:57.544878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303229+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:58.303229+0000","last_peered":"2024-09-10T22:45:58.303229+0000","last_clean":"2024-09-10T22:45:58.303229+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:58.303229+0000","last_undegraded":"2024-09-10T22:45:58.303229+0000","last_fullsized":"2024-09-10T22:45:58.303229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542907+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:46:57.542907+0000","last_peered":"2024-09-10T22:46:57.542907+0000","last_clean":"2024-09-10T22:46:57.542907+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:46:57.542907+0000","last_undegraded":"2024-09-10T22:46:57.542907+0000","last_fullsized":"2024-09-10T22:46:57.542907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298787+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:58.298787+0000","last_peered":"2024-09-10T22:45:58.298787+0000","last_clean":"2024-09-10T22:45:58.298787+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:58.298787+0000","last_undegraded":"2024-09-10T22:45:58.298787+0000","last_fullsized":"2024-09-10T22:45:58.298787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558093+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:46:57.558093+0000","last_peered":"2024-09-10T22:46:57.558093+0000","last_clean":"2024-09-10T22:46:57.558093+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:46:57.558093+0000","last_undegraded":"2024-09-10T22:46:57.558093+0000","last_fullsized":"2024-09-10T22:46:57.558093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557317+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:46:57.557317+0000","last_peered":"2024-09-10T22:46:57.557317+0000","last_clean":"2024-09-10T22:46:57.557317+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:46:57.557317+0000","last_undegraded":"2024-09-10T22:46:57.557317+0000","last_fullsized":"2024-09-10T22:46:57.557317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298925+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:58.298925+0000","last_peered":"2024-09-10T22:45:58.298925+0000","last_clean":"2024-09-10T22:45:58.298925+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:58.298925+0000","last_undegraded":"2024-09-10T22:45:58.298925+0000","last_fullsized":"2024-09-10T22:45:58.298925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558535+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:46:57.558535+0000","last_peered":"2024-09-10T22:46:57.558535+0000","last_clean":"2024-09-10T22:46:57.558535+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:46:57.558535+0000","last_undegraded":"2024-09-10T22:46:57.558535+0000","last_fullsized":"2024-09-10T22:46:57.558535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299045+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:58.299045+0000","last_peered":"2024-09-10T22:45:58.299045+0000","last_clean":"2024-09-10T22:45:58.299045+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:58.299045+0000","last_undegraded":"2024-09-10T22:45:58.299045+0000","last_fullsized":"2024-09-10T22:45:58.299045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550030+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:46:57.550030+0000","last_peered":"2024-09-10T22:46:57.550030+0000","last_clean":"2024-09-10T22:46:57.550030+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:46:57.550030+0000","last_undegraded":"2024-09-10T22:46:57.550030+0000","last_fullsized":"2024-09-10T22:46:57.550030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680612+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:58.680612+0000","last_peered":"2024-09-10T22:45:58.680612+0000","last_clean":"2024-09-10T22:45:58.680612+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:58.680612+0000","last_undegraded":"2024-09-10T22:45:58.680612+0000","last_fullsized":"2024-09-10T22:45:58.680612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538889+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:46:57.538889+0000","last_peered":"2024-09-10T22:46:57.538889+0000","last_clean":"2024-09-10T22:46:57.538889+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:46:57.538889+0000","last_undegraded":"2024-09-10T22:46:57.538889+0000","last_fullsized":"2024-09-10T22:46:57.538889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.195331+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:47:11.195331+0000","last_peered":"2024-09-10T22:47:11.195331+0000","last_clean":"2024-09-10T22:47:11.195331+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:47:11.195331+0000","last_undegraded":"2024-09-10T22:47:11.195331+0000","last_fullsized":"2024-09-10T22:47:11.195331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67049,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1071,"num_read_kb":610,"num_write":251,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":405978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1057326,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":405908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495673,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":999503,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":998091,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":405450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":466769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":467165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":406964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:47:18.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:47:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- 172.21.15.96:0/4123589379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41018a0 msgr2=0x7f76b4101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- 172.21.15.96:0/4123589379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41018a0 0x7f76b4101ca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009930 tx=0x7f76b002a420 comp rx=0 tx=0).stop 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- 172.21.15.96:0/4123589379 shutdown_connections 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- 172.21.15.96:0/4123589379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41033b0 0x7f76b4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- 172.21.15.96:0/4123589379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41029f0 0x7f76b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- 172.21.15.96:0/4123589379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41018a0 0x7f76b4101ca0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- 172.21.15.96:0/4123589379 >> 172.21.15.96:0/4123589379 conn(0x7f76b40fd180 msgr2=0x7f76b40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- 172.21.15.96:0/4123589379 shutdown_connections 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- 172.21.15.96:0/4123589379 wait complete. 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 Processor -- start 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- start start 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 0x7f76b4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41029f0 0x7f76b4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e600 con 0x7f76b41033b0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e480 con 0x7f76b41029f0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bc5da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e780 con 0x7f76b41018a0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.179+0000 7f76bab77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34620/0 (socket says 172.21.15.96:34620) 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 -- 172.21.15.96:0/2670815533 learned_addr learned my addr 172.21.15.96:0/2670815533 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76ba376640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 0x7f76b4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76b9b75640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41029f0 0x7f76b4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 -- 172.21.15.96:0/2670815533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 msgr2=0x7f76b4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 0x7f76b4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 -- 172.21.15.96:0/2670815533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41029f0 msgr2=0x7f76b4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41029f0 0x7f76b4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 -- 172.21.15.96:0/2670815533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b419a860 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76ba376640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 0x7f76b4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bab77640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00cce0 tx=0x7f76ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00f450 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b419ab50 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ac00faa0 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac018b70 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.180+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b41a1220 con 0x7f76b41033b0 2024-09-10T22:47:18.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.181+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b4108bb0 con 0x7f76b41033b0 2024-09-10T22:47:18.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.182+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f76ac00f5f0 con 0x7f76b41033b0 2024-09-10T22:47:18.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.183+0000 7f76a77fe640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 0x7f76a00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.183+0000 7f76ba376640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 0x7f76a00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.183+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f76ac0e6490 con 0x7f76b41033b0 2024-09-10T22:47:18.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.184+0000 7f76ba376640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 0x7f76a00b7830 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:18.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.186+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76ac014030 con 0x7f76b41033b0 2024-09-10T22:47:18.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.361+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76b4063590 con 0x7f76a00b5370 2024-09-10T22:47:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.365+0000 7f76a77fe640 1 -- 172.21.15.96:0/2670815533 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259005 (secure 0 0 0) 0x7f76b4063590 con 0x7f76a00b5370 2024-09-10T22:47:18.369 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:18.370 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 msgr2=0x7f76a00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 0x7f76a00b7830 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b00314a0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 msgr2=0x7f76b4199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00cce0 tx=0x7f76ac00a620 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 shutdown_connections 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b41029f0 0x7f76b4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76a00b5370 0x7f76a00b7830 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76b41033b0 0x7f76b4199fa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 --2- 172.21.15.96:0/2670815533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76b41018a0 0x7f76b4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 >> 172.21.15.96:0/2670815533 conn(0x7f76b40fd180 msgr2=0x7f76b4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 shutdown_connections 2024-09-10T22:47:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.368+0000 7f76bc5da640 1 -- 172.21.15.96:0/2670815533 wait complete. 2024-09-10T22:47:18.380 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T22:47:17.326794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657651,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1147,"num_read_kb":674,"num_write":376,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":809,"ondisk_log_size":809,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363824,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578000,"statfs":{"total":1151940427776,"available":1151567872000,"internally_reserved":0,"allocated":10665984,"data_stored":6830859,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553587+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:46:57.553587+0000","last_peered":"2024-09-10T22:46:57.553587+0000","last_clean":"2024-09-10T22:46:57.553587+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:46:57.553587+0000","last_undegraded":"2024-09-10T22:46:57.553587+0000","last_fullsized":"2024-09-10T22:46:57.553587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538919+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:46:57.538919+0000","last_peered":"2024-09-10T22:46:57.538919+0000","last_clean":"2024-09-10T22:46:57.538919+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:46:57.538919+0000","last_undegraded":"2024-09-10T22:46:57.538919+0000","last_fullsized":"2024-09-10T22:46:57.538919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550143+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:46:57.550143+0000","last_peered":"2024-09-10T22:46:57.550143+0000","last_clean":"2024-09-10T22:46:57.550143+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:46:57.550143+0000","last_undegraded":"2024-09-10T22:46:57.550143+0000","last_fullsized":"2024-09-10T22:46:57.550143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544008+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:46:57.544008+0000","last_peered":"2024-09-10T22:46:57.544008+0000","last_clean":"2024-09-10T22:46:57.544008+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:46:57.544008+0000","last_undegraded":"2024-09-10T22:46:57.544008+0000","last_fullsized":"2024-09-10T22:46:57.544008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.535399+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:46:57.535399+0000","last_peered":"2024-09-10T22:46:57.535399+0000","last_clean":"2024-09-10T22:46:57.535399+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:46:57.535399+0000","last_undegraded":"2024-09-10T22:46:57.535399+0000","last_fullsized":"2024-09-10T22:46:57.535399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549118+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:46:57.549118+0000","last_peered":"2024-09-10T22:46:57.549118+0000","last_clean":"2024-09-10T22:46:57.549118+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:46:57.549118+0000","last_undegraded":"2024-09-10T22:46:57.549118+0000","last_fullsized":"2024-09-10T22:46:57.549118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558179+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:46:57.558179+0000","last_peered":"2024-09-10T22:46:57.558179+0000","last_clean":"2024-09-10T22:46:57.558179+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:46:57.558179+0000","last_undegraded":"2024-09-10T22:46:57.558179+0000","last_fullsized":"2024-09-10T22:46:57.558179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542594+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:46:57.542594+0000","last_peered":"2024-09-10T22:46:57.542594+0000","last_clean":"2024-09-10T22:46:57.542594+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:46:57.542594+0000","last_undegraded":"2024-09-10T22:46:57.542594+0000","last_fullsized":"2024-09-10T22:46:57.542594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557493+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:46:57.557493+0000","last_peered":"2024-09-10T22:46:57.557493+0000","last_clean":"2024-09-10T22:46:57.557493+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:46:57.557493+0000","last_undegraded":"2024-09-10T22:46:57.557493+0000","last_fullsized":"2024-09-10T22:46:57.557493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542611+0000","last_change":"2024-09-10T22:45:58.304922+0000","last_active":"2024-09-10T22:46:57.542611+0000","last_peered":"2024-09-10T22:46:57.542611+0000","last_clean":"2024-09-10T22:46:57.542611+0000","last_became_active":"2024-09-10T22:45:58.304541+0000","last_became_peered":"2024-09-10T22:45:58.304541+0000","last_unstale":"2024-09-10T22:46:57.542611+0000","last_undegraded":"2024-09-10T22:46:57.542611+0000","last_fullsized":"2024-09-10T22:46:57.542611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:09.043873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557385+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:46:57.557385+0000","last_peered":"2024-09-10T22:46:57.557385+0000","last_clean":"2024-09-10T22:46:57.557385+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:46:57.557385+0000","last_undegraded":"2024-09-10T22:46:57.557385+0000","last_fullsized":"2024-09-10T22:46:57.557385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.536823+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:46:57.536823+0000","last_peered":"2024-09-10T22:46:57.536823+0000","last_clean":"2024-09-10T22:46:57.536823+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:46:57.536823+0000","last_undegraded":"2024-09-10T22:46:57.536823+0000","last_fullsized":"2024-09-10T22:46:57.536823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545373+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:46:57.545373+0000","last_peered":"2024-09-10T22:46:57.545373+0000","last_clean":"2024-09-10T22:46:57.545373+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:46:57.545373+0000","last_undegraded":"2024-09-10T22:46:57.545373+0000","last_fullsized":"2024-09-10T22:46:57.545373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.537087+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:46:57.537087+0000","last_peered":"2024-09-10T22:46:57.537087+0000","last_clean":"2024-09-10T22:46:57.537087+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:46:57.537087+0000","last_undegraded":"2024-09-10T22:46:57.537087+0000","last_fullsized":"2024-09-10T22:46:57.537087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542024+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:46:57.542024+0000","last_peered":"2024-09-10T22:46:57.542024+0000","last_clean":"2024-09-10T22:46:57.542024+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:46:57.542024+0000","last_undegraded":"2024-09-10T22:46:57.542024+0000","last_fullsized":"2024-09-10T22:46:57.542024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555782+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:46:57.555782+0000","last_peered":"2024-09-10T22:46:57.555782+0000","last_clean":"2024-09-10T22:46:57.555782+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:46:57.555782+0000","last_undegraded":"2024-09-10T22:46:57.555782+0000","last_fullsized":"2024-09-10T22:46:57.555782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553633+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:46:57.553633+0000","last_peered":"2024-09-10T22:46:57.553633+0000","last_clean":"2024-09-10T22:46:57.553633+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:46:57.553633+0000","last_undegraded":"2024-09-10T22:46:57.553633+0000","last_fullsized":"2024-09-10T22:46:57.553633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554932+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:46:57.554932+0000","last_peered":"2024-09-10T22:46:57.554932+0000","last_clean":"2024-09-10T22:46:57.554932+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:46:57.554932+0000","last_undegraded":"2024-09-10T22:46:57.554932+0000","last_fullsized":"2024-09-10T22:46:57.554932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541118+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:46:57.541118+0000","last_peered":"2024-09-10T22:46:57.541118+0000","last_clean":"2024-09-10T22:46:57.541118+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:46:57.541118+0000","last_undegraded":"2024-09-10T22:46:57.541118+0000","last_fullsized":"2024-09-10T22:46:57.541118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549379+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:46:57.549379+0000","last_peered":"2024-09-10T22:46:57.549379+0000","last_clean":"2024-09-10T22:46:57.549379+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:46:57.549379+0000","last_undegraded":"2024-09-10T22:46:57.549379+0000","last_fullsized":"2024-09-10T22:46:57.549379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552691+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:46:57.552691+0000","last_peered":"2024-09-10T22:46:57.552691+0000","last_clean":"2024-09-10T22:46:57.552691+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:46:57.552691+0000","last_undegraded":"2024-09-10T22:46:57.552691+0000","last_fullsized":"2024-09-10T22:46:57.552691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556630+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:46:57.556630+0000","last_peered":"2024-09-10T22:46:57.556630+0000","last_clean":"2024-09-10T22:46:57.556630+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:46:57.556630+0000","last_undegraded":"2024-09-10T22:46:57.556630+0000","last_fullsized":"2024-09-10T22:46:57.556630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540681+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:46:57.540681+0000","last_peered":"2024-09-10T22:46:57.540681+0000","last_clean":"2024-09-10T22:46:57.540681+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:46:57.540681+0000","last_undegraded":"2024-09-10T22:46:57.540681+0000","last_fullsized":"2024-09-10T22:46:57.540681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543300+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:46:57.543300+0000","last_peered":"2024-09-10T22:46:57.543300+0000","last_clean":"2024-09-10T22:46:57.543300+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:46:57.543300+0000","last_undegraded":"2024-09-10T22:46:57.543300+0000","last_fullsized":"2024-09-10T22:46:57.543300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538562+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:46:57.538562+0000","last_peered":"2024-09-10T22:46:57.538562+0000","last_clean":"2024-09-10T22:46:57.538562+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:46:57.538562+0000","last_undegraded":"2024-09-10T22:46:57.538562+0000","last_fullsized":"2024-09-10T22:46:57.538562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551195+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:46:57.551195+0000","last_peered":"2024-09-10T22:46:57.551195+0000","last_clean":"2024-09-10T22:46:57.551195+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:46:57.551195+0000","last_undegraded":"2024-09-10T22:46:57.551195+0000","last_fullsized":"2024-09-10T22:46:57.551195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541587+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:46:57.541587+0000","last_peered":"2024-09-10T22:46:57.541587+0000","last_clean":"2024-09-10T22:46:57.541587+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:46:57.541587+0000","last_undegraded":"2024-09-10T22:46:57.541587+0000","last_fullsized":"2024-09-10T22:46:57.541587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543325+0000","last_change":"2024-09-10T22:45:58.301222+0000","last_active":"2024-09-10T22:46:57.543325+0000","last_peered":"2024-09-10T22:46:57.543325+0000","last_clean":"2024-09-10T22:46:57.543325+0000","last_became_active":"2024-09-10T22:45:58.300835+0000","last_became_peered":"2024-09-10T22:45:58.300835+0000","last_unstale":"2024-09-10T22:46:57.543325+0000","last_undegraded":"2024-09-10T22:46:57.543325+0000","last_fullsized":"2024-09-10T22:46:57.543325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.772354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541083+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:46:57.541083+0000","last_peered":"2024-09-10T22:46:57.541083+0000","last_clean":"2024-09-10T22:46:57.541083+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:46:57.541083+0000","last_undegraded":"2024-09-10T22:46:57.541083+0000","last_fullsized":"2024-09-10T22:46:57.541083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538073+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:46:57.538073+0000","last_peered":"2024-09-10T22:46:57.538073+0000","last_clean":"2024-09-10T22:46:57.538073+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:46:57.538073+0000","last_undegraded":"2024-09-10T22:46:57.538073+0000","last_fullsized":"2024-09-10T22:46:57.538073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540443+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:46:57.540443+0000","last_peered":"2024-09-10T22:46:57.540443+0000","last_clean":"2024-09-10T22:46:57.540443+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:46:57.540443+0000","last_undegraded":"2024-09-10T22:46:57.540443+0000","last_fullsized":"2024-09-10T22:46:57.540443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545371+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:46:57.545371+0000","last_peered":"2024-09-10T22:46:57.545371+0000","last_clean":"2024-09-10T22:46:57.545371+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:46:57.545371+0000","last_undegraded":"2024-09-10T22:46:57.545371+0000","last_fullsized":"2024-09-10T22:46:57.545371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542915+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:46:57.542915+0000","last_peered":"2024-09-10T22:46:57.542915+0000","last_clean":"2024-09-10T22:46:57.542915+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:46:57.542915+0000","last_undegraded":"2024-09-10T22:46:57.542915+0000","last_fullsized":"2024-09-10T22:46:57.542915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.197033+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:47:11.197033+0000","last_peered":"2024-09-10T22:47:11.197033+0000","last_clean":"2024-09-10T22:47:11.197033+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:47:11.197033+0000","last_undegraded":"2024-09-10T22:47:11.197033+0000","last_fullsized":"2024-09-10T22:47:11.197033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549257+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:46:57.549257+0000","last_peered":"2024-09-10T22:46:57.549257+0000","last_clean":"2024-09-10T22:46:57.549257+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:46:57.549257+0000","last_undegraded":"2024-09-10T22:46:57.549257+0000","last_fullsized":"2024-09-10T22:46:57.549257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.306152+0000","last_change":"2024-09-10T22:45:58.306070+0000","last_active":"2024-09-10T22:45:58.306152+0000","last_peered":"2024-09-10T22:45:58.306152+0000","last_clean":"2024-09-10T22:45:58.306152+0000","last_became_active":"2024-09-10T22:45:58.300281+0000","last_became_peered":"2024-09-10T22:45:58.300281+0000","last_unstale":"2024-09-10T22:45:58.306152+0000","last_undegraded":"2024-09-10T22:45:58.306152+0000","last_fullsized":"2024-09-10T22:45:58.306152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:01.832411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297848+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:58.297848+0000","last_peered":"2024-09-10T22:45:58.297848+0000","last_clean":"2024-09-10T22:45:58.297848+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:58.297848+0000","last_undegraded":"2024-09-10T22:45:58.297848+0000","last_fullsized":"2024-09-10T22:45:58.297848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543952+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:46:57.543952+0000","last_peered":"2024-09-10T22:46:57.543952+0000","last_clean":"2024-09-10T22:46:57.543952+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:46:57.543952+0000","last_undegraded":"2024-09-10T22:46:57.543952+0000","last_fullsized":"2024-09-10T22:46:57.543952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.546120+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:46:57.546120+0000","last_peered":"2024-09-10T22:46:57.546120+0000","last_clean":"2024-09-10T22:46:57.546120+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:46:57.546120+0000","last_undegraded":"2024-09-10T22:46:57.546120+0000","last_fullsized":"2024-09-10T22:46:57.546120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":233,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.198974+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:47:13.198974+0000","last_peered":"2024-09-10T22:47:13.198974+0000","last_clean":"2024-09-10T22:47:13.198974+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:47:13.198974+0000","last_undegraded":"2024-09-10T22:47:13.198974+0000","last_fullsized":"2024-09-10T22:47:13.198974+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551796+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:46:57.551796+0000","last_peered":"2024-09-10T22:46:57.551796+0000","last_clean":"2024-09-10T22:46:57.551796+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:46:57.551796+0000","last_undegraded":"2024-09-10T22:46:57.551796+0000","last_fullsized":"2024-09-10T22:46:57.551796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680757+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:58.680757+0000","last_peered":"2024-09-10T22:45:58.680757+0000","last_clean":"2024-09-10T22:45:58.680757+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:58.680757+0000","last_undegraded":"2024-09-10T22:45:58.680757+0000","last_fullsized":"2024-09-10T22:45:58.680757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541828+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:46:57.541828+0000","last_peered":"2024-09-10T22:46:57.541828+0000","last_clean":"2024-09-10T22:46:57.541828+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:46:57.541828+0000","last_undegraded":"2024-09-10T22:46:57.541828+0000","last_fullsized":"2024-09-10T22:46:57.541828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672340+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:58.672340+0000","last_peered":"2024-09-10T22:45:58.672340+0000","last_clean":"2024-09-10T22:45:58.672340+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:58.672340+0000","last_undegraded":"2024-09-10T22:45:58.672340+0000","last_fullsized":"2024-09-10T22:45:58.672340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556911+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:46:57.556911+0000","last_peered":"2024-09-10T22:46:57.556911+0000","last_clean":"2024-09-10T22:46:57.556911+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:46:57.556911+0000","last_undegraded":"2024-09-10T22:46:57.556911+0000","last_fullsized":"2024-09-10T22:46:57.556911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199739+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:47:13.199739+0000","last_peered":"2024-09-10T22:47:13.199739+0000","last_clean":"2024-09-10T22:47:13.199739+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:47:13.199739+0000","last_undegraded":"2024-09-10T22:47:13.199739+0000","last_fullsized":"2024-09-10T22:47:13.199739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60319,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":217,"num_read_kb":167,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555331+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:46:57.555331+0000","last_peered":"2024-09-10T22:46:57.555331+0000","last_clean":"2024-09-10T22:46:57.555331+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:46:57.555331+0000","last_undegraded":"2024-09-10T22:46:57.555331+0000","last_fullsized":"2024-09-10T22:46:57.555331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303539+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:58.303539+0000","last_peered":"2024-09-10T22:45:58.303539+0000","last_clean":"2024-09-10T22:45:58.303539+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:58.303539+0000","last_undegraded":"2024-09-10T22:45:58.303539+0000","last_fullsized":"2024-09-10T22:45:58.303539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552322+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:46:57.552322+0000","last_peered":"2024-09-10T22:46:57.552322+0000","last_clean":"2024-09-10T22:46:57.552322+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:46:57.552322+0000","last_undegraded":"2024-09-10T22:46:57.552322+0000","last_fullsized":"2024-09-10T22:46:57.552322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556243+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:46:57.556243+0000","last_peered":"2024-09-10T22:46:57.556243+0000","last_clean":"2024-09-10T22:46:57.556243+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:46:57.556243+0000","last_undegraded":"2024-09-10T22:46:57.556243+0000","last_fullsized":"2024-09-10T22:46:57.556243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:12.208490+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:47:12.208490+0000","last_peered":"2024-09-10T22:47:12.208490+0000","last_clean":"2024-09-10T22:47:12.208490+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:47:12.208490+0000","last_undegraded":"2024-09-10T22:47:12.208490+0000","last_fullsized":"2024-09-10T22:47:12.208490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558382+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:46:57.558382+0000","last_peered":"2024-09-10T22:46:57.558382+0000","last_clean":"2024-09-10T22:46:57.558382+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:46:57.558382+0000","last_undegraded":"2024-09-10T22:46:57.558382+0000","last_fullsized":"2024-09-10T22:46:57.558382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298597+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:58.298597+0000","last_peered":"2024-09-10T22:45:58.298597+0000","last_clean":"2024-09-10T22:45:58.298597+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:58.298597+0000","last_undegraded":"2024-09-10T22:45:58.298597+0000","last_fullsized":"2024-09-10T22:45:58.298597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556870+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:46:57.556870+0000","last_peered":"2024-09-10T22:46:57.556870+0000","last_clean":"2024-09-10T22:46:57.556870+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:46:57.556870+0000","last_undegraded":"2024-09-10T22:46:57.556870+0000","last_fullsized":"2024-09-10T22:46:57.556870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299286+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:58.299286+0000","last_peered":"2024-09-10T22:45:58.299286+0000","last_clean":"2024-09-10T22:45:58.299286+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:58.299286+0000","last_undegraded":"2024-09-10T22:45:58.299286+0000","last_fullsized":"2024-09-10T22:45:58.299286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.548879+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:46:57.548879+0000","last_peered":"2024-09-10T22:46:57.548879+0000","last_clean":"2024-09-10T22:46:57.548879+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:46:57.548879+0000","last_undegraded":"2024-09-10T22:46:57.548879+0000","last_fullsized":"2024-09-10T22:46:57.548879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298492+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:58.298492+0000","last_peered":"2024-09-10T22:45:58.298492+0000","last_clean":"2024-09-10T22:45:58.298492+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:58.298492+0000","last_undegraded":"2024-09-10T22:45:58.298492+0000","last_fullsized":"2024-09-10T22:45:58.298492+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558447+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:46:57.558447+0000","last_peered":"2024-09-10T22:46:57.558447+0000","last_clean":"2024-09-10T22:46:57.558447+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:46:57.558447+0000","last_undegraded":"2024-09-10T22:46:57.558447+0000","last_fullsized":"2024-09-10T22:46:57.558447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672395+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:58.672395+0000","last_peered":"2024-09-10T22:45:58.672395+0000","last_clean":"2024-09-10T22:45:58.672395+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:58.672395+0000","last_undegraded":"2024-09-10T22:45:58.672395+0000","last_fullsized":"2024-09-10T22:45:58.672395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554711+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:46:57.554711+0000","last_peered":"2024-09-10T22:46:57.554711+0000","last_clean":"2024-09-10T22:46:57.554711+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:46:57.554711+0000","last_undegraded":"2024-09-10T22:46:57.554711+0000","last_fullsized":"2024-09-10T22:46:57.554711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.196964+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:47:11.196964+0000","last_peered":"2024-09-10T22:47:11.196964+0000","last_clean":"2024-09-10T22:47:11.196964+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:47:11.196964+0000","last_undegraded":"2024-09-10T22:47:11.196964+0000","last_fullsized":"2024-09-10T22:47:11.196964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557894+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:46:57.557894+0000","last_peered":"2024-09-10T22:46:57.557894+0000","last_clean":"2024-09-10T22:46:57.557894+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:46:57.557894+0000","last_undegraded":"2024-09-10T22:46:57.557894+0000","last_fullsized":"2024-09-10T22:46:57.557894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680641+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:58.680641+0000","last_peered":"2024-09-10T22:45:58.680641+0000","last_clean":"2024-09-10T22:45:58.680641+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:58.680641+0000","last_undegraded":"2024-09-10T22:45:58.680641+0000","last_fullsized":"2024-09-10T22:45:58.680641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296623+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:58.296623+0000","last_peered":"2024-09-10T22:45:58.296623+0000","last_clean":"2024-09-10T22:45:58.296623+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:58.296623+0000","last_undegraded":"2024-09-10T22:45:58.296623+0000","last_fullsized":"2024-09-10T22:45:58.296623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550173+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:46:57.550173+0000","last_peered":"2024-09-10T22:46:57.550173+0000","last_clean":"2024-09-10T22:46:57.550173+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:46:57.550173+0000","last_undegraded":"2024-09-10T22:46:57.550173+0000","last_fullsized":"2024-09-10T22:46:57.550173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680509+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:58.680509+0000","last_peered":"2024-09-10T22:45:58.680509+0000","last_clean":"2024-09-10T22:45:58.680509+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:58.680509+0000","last_undegraded":"2024-09-10T22:45:58.680509+0000","last_fullsized":"2024-09-10T22:45:58.680509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544946+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:46:57.544946+0000","last_peered":"2024-09-10T22:46:57.544946+0000","last_clean":"2024-09-10T22:46:57.544946+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:46:57.544946+0000","last_undegraded":"2024-09-10T22:46:57.544946+0000","last_fullsized":"2024-09-10T22:46:57.544946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:14.210466+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:47:14.210466+0000","last_peered":"2024-09-10T22:47:14.210466+0000","last_clean":"2024-09-10T22:47:14.210466+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:47:14.210466+0000","last_undegraded":"2024-09-10T22:47:14.210466+0000","last_fullsized":"2024-09-10T22:47:14.210466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544012+0000","last_change":"2024-09-10T22:45:58.304503+0000","last_active":"2024-09-10T22:46:57.544012+0000","last_peered":"2024-09-10T22:46:57.544012+0000","last_clean":"2024-09-10T22:46:57.544012+0000","last_became_active":"2024-09-10T22:45:58.304243+0000","last_became_peered":"2024-09-10T22:45:58.304243+0000","last_unstale":"2024-09-10T22:46:57.544012+0000","last_undegraded":"2024-09-10T22:46:57.544012+0000","last_fullsized":"2024-09-10T22:46:57.544012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:33.190092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"77'6","reported_seq":135,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199731+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:47:13.199731+0000","last_peered":"2024-09-10T22:47:13.199731+0000","last_clean":"2024-09-10T22:47:13.199731+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:47:13.199731+0000","last_undegraded":"2024-09-10T22:47:13.199731+0000","last_fullsized":"2024-09-10T22:47:13.199731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549451+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:46:57.549451+0000","last_peered":"2024-09-10T22:46:57.549451+0000","last_clean":"2024-09-10T22:46:57.549451+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:46:57.549451+0000","last_undegraded":"2024-09-10T22:46:57.549451+0000","last_fullsized":"2024-09-10T22:46:57.549451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.599710+0000","last_change":"2024-09-10T22:45:58.599612+0000","last_active":"2024-09-10T22:45:58.599710+0000","last_peered":"2024-09-10T22:45:58.599710+0000","last_clean":"2024-09-10T22:45:58.599710+0000","last_became_active":"2024-09-10T22:45:58.599226+0000","last_became_peered":"2024-09-10T22:45:58.599226+0000","last_unstale":"2024-09-10T22:45:58.599710+0000","last_undegraded":"2024-09-10T22:45:58.599710+0000","last_fullsized":"2024-09-10T22:45:58.599710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.112571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544799+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:46:57.544799+0000","last_peered":"2024-09-10T22:46:57.544799+0000","last_clean":"2024-09-10T22:46:57.544799+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:46:57.544799+0000","last_undegraded":"2024-09-10T22:46:57.544799+0000","last_fullsized":"2024-09-10T22:46:57.544799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299088+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:58.299088+0000","last_peered":"2024-09-10T22:45:58.299088+0000","last_clean":"2024-09-10T22:45:58.299088+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:58.299088+0000","last_undegraded":"2024-09-10T22:45:58.299088+0000","last_fullsized":"2024-09-10T22:45:58.299088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542853+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:46:57.542853+0000","last_peered":"2024-09-10T22:46:57.542853+0000","last_clean":"2024-09-10T22:46:57.542853+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:46:57.542853+0000","last_undegraded":"2024-09-10T22:46:57.542853+0000","last_fullsized":"2024-09-10T22:46:57.542853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297683+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:58.297683+0000","last_peered":"2024-09-10T22:45:58.297683+0000","last_clean":"2024-09-10T22:45:58.297683+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:58.297683+0000","last_undegraded":"2024-09-10T22:45:58.297683+0000","last_fullsized":"2024-09-10T22:45:58.297683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556425+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:46:57.556425+0000","last_peered":"2024-09-10T22:46:57.556425+0000","last_clean":"2024-09-10T22:46:57.556425+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:46:57.556425+0000","last_undegraded":"2024-09-10T22:46:57.556425+0000","last_fullsized":"2024-09-10T22:46:57.556425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296692+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:58.296692+0000","last_peered":"2024-09-10T22:45:58.296692+0000","last_clean":"2024-09-10T22:45:58.296692+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:58.296692+0000","last_undegraded":"2024-09-10T22:45:58.296692+0000","last_fullsized":"2024-09-10T22:45:58.296692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544837+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:46:57.544837+0000","last_peered":"2024-09-10T22:46:57.544837+0000","last_clean":"2024-09-10T22:46:57.544837+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:46:57.544837+0000","last_undegraded":"2024-09-10T22:46:57.544837+0000","last_fullsized":"2024-09-10T22:46:57.544837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.602110+0000","last_change":"2024-09-10T22:45:58.602021+0000","last_active":"2024-09-10T22:45:58.602110+0000","last_peered":"2024-09-10T22:45:58.602110+0000","last_clean":"2024-09-10T22:45:58.602110+0000","last_became_active":"2024-09-10T22:45:58.599358+0000","last_became_peered":"2024-09-10T22:45:58.599358+0000","last_unstale":"2024-09-10T22:45:58.602110+0000","last_undegraded":"2024-09-10T22:45:58.602110+0000","last_fullsized":"2024-09-10T22:45:58.602110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:40.510120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.530711+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:46:57.530711+0000","last_peered":"2024-09-10T22:46:57.530711+0000","last_clean":"2024-09-10T22:46:57.530711+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:46:57.530711+0000","last_undegraded":"2024-09-10T22:46:57.530711+0000","last_fullsized":"2024-09-10T22:46:57.530711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680327+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:58.680327+0000","last_peered":"2024-09-10T22:45:58.680327+0000","last_clean":"2024-09-10T22:45:58.680327+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:58.680327+0000","last_undegraded":"2024-09-10T22:45:58.680327+0000","last_fullsized":"2024-09-10T22:45:58.680327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544878+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:46:57.544878+0000","last_peered":"2024-09-10T22:46:57.544878+0000","last_clean":"2024-09-10T22:46:57.544878+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:46:57.544878+0000","last_undegraded":"2024-09-10T22:46:57.544878+0000","last_fullsized":"2024-09-10T22:46:57.544878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303229+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:58.303229+0000","last_peered":"2024-09-10T22:45:58.303229+0000","last_clean":"2024-09-10T22:45:58.303229+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:58.303229+0000","last_undegraded":"2024-09-10T22:45:58.303229+0000","last_fullsized":"2024-09-10T22:45:58.303229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542907+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:46:57.542907+0000","last_peered":"2024-09-10T22:46:57.542907+0000","last_clean":"2024-09-10T22:46:57.542907+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:46:57.542907+0000","last_undegraded":"2024-09-10T22:46:57.542907+0000","last_fullsized":"2024-09-10T22:46:57.542907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298787+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:58.298787+0000","last_peered":"2024-09-10T22:45:58.298787+0000","last_clean":"2024-09-10T22:45:58.298787+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:58.298787+0000","last_undegraded":"2024-09-10T22:45:58.298787+0000","last_fullsized":"2024-09-10T22:45:58.298787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558093+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:46:57.558093+0000","last_peered":"2024-09-10T22:46:57.558093+0000","last_clean":"2024-09-10T22:46:57.558093+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:46:57.558093+0000","last_undegraded":"2024-09-10T22:46:57.558093+0000","last_fullsized":"2024-09-10T22:46:57.558093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557317+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:46:57.557317+0000","last_peered":"2024-09-10T22:46:57.557317+0000","last_clean":"2024-09-10T22:46:57.557317+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:46:57.557317+0000","last_undegraded":"2024-09-10T22:46:57.557317+0000","last_fullsized":"2024-09-10T22:46:57.557317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298925+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:58.298925+0000","last_peered":"2024-09-10T22:45:58.298925+0000","last_clean":"2024-09-10T22:45:58.298925+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:58.298925+0000","last_undegraded":"2024-09-10T22:45:58.298925+0000","last_fullsized":"2024-09-10T22:45:58.298925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558535+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:46:57.558535+0000","last_peered":"2024-09-10T22:46:57.558535+0000","last_clean":"2024-09-10T22:46:57.558535+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:46:57.558535+0000","last_undegraded":"2024-09-10T22:46:57.558535+0000","last_fullsized":"2024-09-10T22:46:57.558535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299045+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:58.299045+0000","last_peered":"2024-09-10T22:45:58.299045+0000","last_clean":"2024-09-10T22:45:58.299045+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:58.299045+0000","last_undegraded":"2024-09-10T22:45:58.299045+0000","last_fullsized":"2024-09-10T22:45:58.299045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550030+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:46:57.550030+0000","last_peered":"2024-09-10T22:46:57.550030+0000","last_clean":"2024-09-10T22:46:57.550030+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:46:57.550030+0000","last_undegraded":"2024-09-10T22:46:57.550030+0000","last_fullsized":"2024-09-10T22:46:57.550030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680612+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:58.680612+0000","last_peered":"2024-09-10T22:45:58.680612+0000","last_clean":"2024-09-10T22:45:58.680612+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:58.680612+0000","last_undegraded":"2024-09-10T22:45:58.680612+0000","last_fullsized":"2024-09-10T22:45:58.680612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538889+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:46:57.538889+0000","last_peered":"2024-09-10T22:46:57.538889+0000","last_clean":"2024-09-10T22:46:57.538889+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:46:57.538889+0000","last_undegraded":"2024-09-10T22:46:57.538889+0000","last_fullsized":"2024-09-10T22:46:57.538889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.195331+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:47:11.195331+0000","last_peered":"2024-09-10T22:47:11.195331+0000","last_clean":"2024-09-10T22:47:11.195331+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:47:11.195331+0000","last_undegraded":"2024-09-10T22:47:11.195331+0000","last_fullsized":"2024-09-10T22:47:11.195331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67049,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1071,"num_read_kb":610,"num_write":251,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":405978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1057326,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":405908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495673,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":999503,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":998091,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":405450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":466769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":467165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":406964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:47:18.384 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T22:47:18.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- 172.21.15.96:0/2384060505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102b40 msgr2=0x7f761c1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- 172.21.15.96:0/2384060505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102b40 0x7f761c1094c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7614009cd0 tx=0x7f761402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- 172.21.15.96:0/2384060505 shutdown_connections 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- 172.21.15.96:0/2384060505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102b40 0x7f761c1094c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- 172.21.15.96:0/2384060505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c102180 0x7f761c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- 172.21.15.96:0/2384060505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c101030 0x7f761c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- 172.21.15.96:0/2384060505 >> 172.21.15.96:0/2384060505 conn(0x7f761c0fc8d0 msgr2=0x7f761c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- 172.21.15.96:0/2384060505 shutdown_connections 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- 172.21.15.96:0/2384060505 wait complete. 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 Processor -- start 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- start start 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c101030 0x7f761c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c102b40 0x7f761c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c430 con 0x7f761c102180 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c2b0 con 0x7f761c102b40 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.518+0000 7f762435d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c5b0 con 0x7f761c101030 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34644/0 (socket says 172.21.15.96:34644) 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 -- 172.21.15.96:0/1457924058 learned_addr learned my addr 172.21.15.96:0/1457924058 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76228fa640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c102b40 0x7f761c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76220f9640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c101030 0x7f761c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 -- 172.21.15.96:0/1457924058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c101030 msgr2=0x7f761c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c101030 0x7f761c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 -- 172.21.15.96:0/1457924058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c102b40 msgr2=0x7f761c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c102b40 0x7f761c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 -- 172.21.15.96:0/1457924058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c19a880 con 0x7f761c102180 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f76218f8640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f761800dbc0 tx=0x7f761800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618005220 con 0x7f761c102180 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761800ae50 con 0x7f761c102180 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c19ab70 con 0x7f761c102180 2024-09-10T22:47:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800e7d0 con 0x7f761c102180 2024-09-10T22:47:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.519+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a1240 con 0x7f761c102180 2024-09-10T22:47:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.520+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c108340 con 0x7f761c102180 2024-09-10T22:47:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.522+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f761800b040 con 0x7f761c102180 2024-09-10T22:47:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.523+0000 7f760f7fe640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 0x7f76080b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.523+0000 7f76220f9640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 0x7f76080b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.523+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f76180e5d60 con 0x7f761c102180 2024-09-10T22:47:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.523+0000 7f76220f9640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 0x7f76080b7830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7610009b60 tx=0x7f7610005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:18.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.526+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76180acbc0 con 0x7f761c102180 2024-09-10T22:47:18.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.699+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 --> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f761c063590 con 0x7f76080b5370 2024-09-10T22:47:18.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.703+0000 7f760f7fe640 1 -- 172.21.15.96:0/1457924058 <== mgr.14150 v2:172.21.15.96:6800/3353555679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259005 (secure 0 0 0) 0x7f761c063590 con 0x7f76080b5370 2024-09-10T22:47:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:18.708 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-10T22:47:18.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 msgr2=0x7f76080b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 0x7f76080b7830 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7610009b60 tx=0x7f7610005f90 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 msgr2=0x7f761c199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f761800dbc0 tx=0x7f761800a620 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 shutdown_connections 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f761c102b40 0x7f761c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76080b5370 0x7f76080b7830 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f761c102180 0x7f761c199390 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 --2- 172.21.15.96:0/1457924058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f761c101030 0x7f761c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 >> 172.21.15.96:0/1457924058 conn(0x7f761c0fc8d0 msgr2=0x7f761c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 shutdown_connections 2024-09-10T22:47:18.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:18.706+0000 7f762435d640 1 -- 172.21.15.96:0/1457924058 wait complete. 2024-09-10T22:47:18.717 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:18 smithi096 ceph-mon[30869]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:18.718 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T22:47:17.326794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657651,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1147,"num_read_kb":674,"num_write":376,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":809,"ondisk_log_size":809,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363824,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578000,"statfs":{"total":1151940427776,"available":1151567872000,"internally_reserved":0,"allocated":10665984,"data_stored":6830859,"data_compressed":107496,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553587+0000","last_change":"2024-09-10T22:45:18.952578+0000","last_active":"2024-09-10T22:46:57.553587+0000","last_peered":"2024-09-10T22:46:57.553587+0000","last_clean":"2024-09-10T22:46:57.553587+0000","last_became_active":"2024-09-10T22:45:18.952204+0000","last_became_peered":"2024-09-10T22:45:18.952204+0000","last_unstale":"2024-09-10T22:46:57.553587+0000","last_undegraded":"2024-09-10T22:46:57.553587+0000","last_fullsized":"2024-09-10T22:46:57.553587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:15.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538919+0000","last_change":"2024-09-10T22:45:18.945119+0000","last_active":"2024-09-10T22:46:57.538919+0000","last_peered":"2024-09-10T22:46:57.538919+0000","last_clean":"2024-09-10T22:46:57.538919+0000","last_became_active":"2024-09-10T22:45:18.944830+0000","last_became_peered":"2024-09-10T22:45:18.944830+0000","last_unstale":"2024-09-10T22:46:57.538919+0000","last_undegraded":"2024-09-10T22:46:57.538919+0000","last_fullsized":"2024-09-10T22:46:57.538919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:21.157488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550143+0000","last_change":"2024-09-10T22:45:18.939835+0000","last_active":"2024-09-10T22:46:57.550143+0000","last_peered":"2024-09-10T22:46:57.550143+0000","last_clean":"2024-09-10T22:46:57.550143+0000","last_became_active":"2024-09-10T22:45:18.939527+0000","last_became_peered":"2024-09-10T22:45:18.939527+0000","last_unstale":"2024-09-10T22:46:57.550143+0000","last_undegraded":"2024-09-10T22:46:57.550143+0000","last_fullsized":"2024-09-10T22:46:57.550143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.688738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544008+0000","last_change":"2024-09-10T22:45:18.953715+0000","last_active":"2024-09-10T22:46:57.544008+0000","last_peered":"2024-09-10T22:46:57.544008+0000","last_clean":"2024-09-10T22:46:57.544008+0000","last_became_active":"2024-09-10T22:45:18.953227+0000","last_became_peered":"2024-09-10T22:45:18.953227+0000","last_unstale":"2024-09-10T22:46:57.544008+0000","last_undegraded":"2024-09-10T22:46:57.544008+0000","last_fullsized":"2024-09-10T22:46:57.544008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:21.436644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.535399+0000","last_change":"2024-09-10T22:45:18.944943+0000","last_active":"2024-09-10T22:46:57.535399+0000","last_peered":"2024-09-10T22:46:57.535399+0000","last_clean":"2024-09-10T22:46:57.535399+0000","last_became_active":"2024-09-10T22:45:18.944617+0000","last_became_peered":"2024-09-10T22:45:18.944617+0000","last_unstale":"2024-09-10T22:46:57.535399+0000","last_undegraded":"2024-09-10T22:46:57.535399+0000","last_fullsized":"2024-09-10T22:46:57.535399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:47.104484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549118+0000","last_change":"2024-09-10T22:45:18.948773+0000","last_active":"2024-09-10T22:46:57.549118+0000","last_peered":"2024-09-10T22:46:57.549118+0000","last_clean":"2024-09-10T22:46:57.549118+0000","last_became_active":"2024-09-10T22:45:18.948468+0000","last_became_peered":"2024-09-10T22:45:18.948468+0000","last_unstale":"2024-09-10T22:46:57.549118+0000","last_undegraded":"2024-09-10T22:46:57.549118+0000","last_fullsized":"2024-09-10T22:46:57.549118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:08.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558179+0000","last_change":"2024-09-10T22:45:18.941966+0000","last_active":"2024-09-10T22:46:57.558179+0000","last_peered":"2024-09-10T22:46:57.558179+0000","last_clean":"2024-09-10T22:46:57.558179+0000","last_became_active":"2024-09-10T22:45:18.941654+0000","last_became_peered":"2024-09-10T22:45:18.941654+0000","last_unstale":"2024-09-10T22:46:57.558179+0000","last_undegraded":"2024-09-10T22:46:57.558179+0000","last_fullsized":"2024-09-10T22:46:57.558179+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:10.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542594+0000","last_change":"2024-09-10T22:45:18.947454+0000","last_active":"2024-09-10T22:46:57.542594+0000","last_peered":"2024-09-10T22:46:57.542594+0000","last_clean":"2024-09-10T22:46:57.542594+0000","last_became_active":"2024-09-10T22:45:18.947235+0000","last_became_peered":"2024-09-10T22:45:18.947235+0000","last_unstale":"2024-09-10T22:46:57.542594+0000","last_undegraded":"2024-09-10T22:46:57.542594+0000","last_fullsized":"2024-09-10T22:46:57.542594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:38.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557493+0000","last_change":"2024-09-10T22:45:18.945115+0000","last_active":"2024-09-10T22:46:57.557493+0000","last_peered":"2024-09-10T22:46:57.557493+0000","last_clean":"2024-09-10T22:46:57.557493+0000","last_became_active":"2024-09-10T22:45:18.944789+0000","last_became_peered":"2024-09-10T22:45:18.944789+0000","last_unstale":"2024-09-10T22:46:57.557493+0000","last_undegraded":"2024-09-10T22:46:57.557493+0000","last_fullsized":"2024-09-10T22:46:57.557493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:03.233088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542611+0000","last_change":"2024-09-10T22:45:58.304922+0000","last_active":"2024-09-10T22:46:57.542611+0000","last_peered":"2024-09-10T22:46:57.542611+0000","last_clean":"2024-09-10T22:46:57.542611+0000","last_became_active":"2024-09-10T22:45:58.304541+0000","last_became_peered":"2024-09-10T22:45:58.304541+0000","last_unstale":"2024-09-10T22:46:57.542611+0000","last_undegraded":"2024-09-10T22:46:57.542611+0000","last_fullsized":"2024-09-10T22:46:57.542611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:09.043873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557385+0000","last_change":"2024-09-10T22:45:18.945678+0000","last_active":"2024-09-10T22:46:57.557385+0000","last_peered":"2024-09-10T22:46:57.557385+0000","last_clean":"2024-09-10T22:46:57.557385+0000","last_became_active":"2024-09-10T22:45:18.944924+0000","last_became_peered":"2024-09-10T22:45:18.944924+0000","last_unstale":"2024-09-10T22:46:57.557385+0000","last_undegraded":"2024-09-10T22:46:57.557385+0000","last_fullsized":"2024-09-10T22:46:57.557385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:58.608322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.536823+0000","last_change":"2024-09-10T22:45:18.947254+0000","last_active":"2024-09-10T22:46:57.536823+0000","last_peered":"2024-09-10T22:46:57.536823+0000","last_clean":"2024-09-10T22:46:57.536823+0000","last_became_active":"2024-09-10T22:45:18.946868+0000","last_became_peered":"2024-09-10T22:45:18.946868+0000","last_unstale":"2024-09-10T22:46:57.536823+0000","last_undegraded":"2024-09-10T22:46:57.536823+0000","last_fullsized":"2024-09-10T22:46:57.536823+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:04.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545373+0000","last_change":"2024-09-10T22:45:18.945821+0000","last_active":"2024-09-10T22:46:57.545373+0000","last_peered":"2024-09-10T22:46:57.545373+0000","last_clean":"2024-09-10T22:46:57.545373+0000","last_became_active":"2024-09-10T22:45:18.945475+0000","last_became_peered":"2024-09-10T22:45:18.945475+0000","last_unstale":"2024-09-10T22:46:57.545373+0000","last_undegraded":"2024-09-10T22:46:57.545373+0000","last_fullsized":"2024-09-10T22:46:57.545373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:10.451989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.537087+0000","last_change":"2024-09-10T22:45:18.948284+0000","last_active":"2024-09-10T22:46:57.537087+0000","last_peered":"2024-09-10T22:46:57.537087+0000","last_clean":"2024-09-10T22:46:57.537087+0000","last_became_active":"2024-09-10T22:45:18.948000+0000","last_became_peered":"2024-09-10T22:45:18.948000+0000","last_unstale":"2024-09-10T22:46:57.537087+0000","last_undegraded":"2024-09-10T22:46:57.537087+0000","last_fullsized":"2024-09-10T22:46:57.537087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:28.714290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542024+0000","last_change":"2024-09-10T22:45:18.942624+0000","last_active":"2024-09-10T22:46:57.542024+0000","last_peered":"2024-09-10T22:46:57.542024+0000","last_clean":"2024-09-10T22:46:57.542024+0000","last_became_active":"2024-09-10T22:45:18.941998+0000","last_became_peered":"2024-09-10T22:45:18.941998+0000","last_unstale":"2024-09-10T22:46:57.542024+0000","last_undegraded":"2024-09-10T22:46:57.542024+0000","last_fullsized":"2024-09-10T22:46:57.542024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:36.683700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555782+0000","last_change":"2024-09-10T22:45:18.940496+0000","last_active":"2024-09-10T22:46:57.555782+0000","last_peered":"2024-09-10T22:46:57.555782+0000","last_clean":"2024-09-10T22:46:57.555782+0000","last_became_active":"2024-09-10T22:45:18.940100+0000","last_became_peered":"2024-09-10T22:45:18.940100+0000","last_unstale":"2024-09-10T22:46:57.555782+0000","last_undegraded":"2024-09-10T22:46:57.555782+0000","last_fullsized":"2024-09-10T22:46:57.555782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:54.958561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.553633+0000","last_change":"2024-09-10T22:45:18.952704+0000","last_active":"2024-09-10T22:46:57.553633+0000","last_peered":"2024-09-10T22:46:57.553633+0000","last_clean":"2024-09-10T22:46:57.553633+0000","last_became_active":"2024-09-10T22:45:18.952413+0000","last_became_peered":"2024-09-10T22:45:18.952413+0000","last_unstale":"2024-09-10T22:46:57.553633+0000","last_undegraded":"2024-09-10T22:46:57.553633+0000","last_fullsized":"2024-09-10T22:46:57.553633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:09.637638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554932+0000","last_change":"2024-09-10T22:45:18.941012+0000","last_active":"2024-09-10T22:46:57.554932+0000","last_peered":"2024-09-10T22:46:57.554932+0000","last_clean":"2024-09-10T22:46:57.554932+0000","last_became_active":"2024-09-10T22:45:18.940671+0000","last_became_peered":"2024-09-10T22:45:18.940671+0000","last_unstale":"2024-09-10T22:46:57.554932+0000","last_undegraded":"2024-09-10T22:46:57.554932+0000","last_fullsized":"2024-09-10T22:46:57.554932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:35.698853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541118+0000","last_change":"2024-09-10T22:45:18.937570+0000","last_active":"2024-09-10T22:46:57.541118+0000","last_peered":"2024-09-10T22:46:57.541118+0000","last_clean":"2024-09-10T22:46:57.541118+0000","last_became_active":"2024-09-10T22:45:18.937031+0000","last_became_peered":"2024-09-10T22:45:18.937031+0000","last_unstale":"2024-09-10T22:46:57.541118+0000","last_undegraded":"2024-09-10T22:46:57.541118+0000","last_fullsized":"2024-09-10T22:46:57.541118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:09.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549379+0000","last_change":"2024-09-10T22:45:18.949340+0000","last_active":"2024-09-10T22:46:57.549379+0000","last_peered":"2024-09-10T22:46:57.549379+0000","last_clean":"2024-09-10T22:46:57.549379+0000","last_became_active":"2024-09-10T22:45:18.949018+0000","last_became_peered":"2024-09-10T22:45:18.949018+0000","last_unstale":"2024-09-10T22:46:57.549379+0000","last_undegraded":"2024-09-10T22:46:57.549379+0000","last_fullsized":"2024-09-10T22:46:57.549379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:09.058202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552691+0000","last_change":"2024-09-10T22:45:18.935544+0000","last_active":"2024-09-10T22:46:57.552691+0000","last_peered":"2024-09-10T22:46:57.552691+0000","last_clean":"2024-09-10T22:46:57.552691+0000","last_became_active":"2024-09-10T22:45:18.935261+0000","last_became_peered":"2024-09-10T22:45:18.935261+0000","last_unstale":"2024-09-10T22:46:57.552691+0000","last_undegraded":"2024-09-10T22:46:57.552691+0000","last_fullsized":"2024-09-10T22:46:57.552691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.543242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556630+0000","last_change":"2024-09-10T22:45:18.941483+0000","last_active":"2024-09-10T22:46:57.556630+0000","last_peered":"2024-09-10T22:46:57.556630+0000","last_clean":"2024-09-10T22:46:57.556630+0000","last_became_active":"2024-09-10T22:45:18.941178+0000","last_became_peered":"2024-09-10T22:45:18.941178+0000","last_unstale":"2024-09-10T22:46:57.556630+0000","last_undegraded":"2024-09-10T22:46:57.556630+0000","last_fullsized":"2024-09-10T22:46:57.556630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:13.402567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540681+0000","last_change":"2024-09-10T22:45:18.940841+0000","last_active":"2024-09-10T22:46:57.540681+0000","last_peered":"2024-09-10T22:46:57.540681+0000","last_clean":"2024-09-10T22:46:57.540681+0000","last_became_active":"2024-09-10T22:45:18.939628+0000","last_became_peered":"2024-09-10T22:45:18.939628+0000","last_unstale":"2024-09-10T22:46:57.540681+0000","last_undegraded":"2024-09-10T22:46:57.540681+0000","last_fullsized":"2024-09-10T22:46:57.540681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:02.461495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543300+0000","last_change":"2024-09-10T22:45:18.944109+0000","last_active":"2024-09-10T22:46:57.543300+0000","last_peered":"2024-09-10T22:46:57.543300+0000","last_clean":"2024-09-10T22:46:57.543300+0000","last_became_active":"2024-09-10T22:45:18.939985+0000","last_became_peered":"2024-09-10T22:45:18.939985+0000","last_unstale":"2024-09-10T22:46:57.543300+0000","last_undegraded":"2024-09-10T22:46:57.543300+0000","last_fullsized":"2024-09-10T22:46:57.543300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:17.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538562+0000","last_change":"2024-09-10T22:45:18.944941+0000","last_active":"2024-09-10T22:46:57.538562+0000","last_peered":"2024-09-10T22:46:57.538562+0000","last_clean":"2024-09-10T22:46:57.538562+0000","last_became_active":"2024-09-10T22:45:18.944589+0000","last_became_peered":"2024-09-10T22:45:18.944589+0000","last_unstale":"2024-09-10T22:46:57.538562+0000","last_undegraded":"2024-09-10T22:46:57.538562+0000","last_fullsized":"2024-09-10T22:46:57.538562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:28.364413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551195+0000","last_change":"2024-09-10T22:45:18.943685+0000","last_active":"2024-09-10T22:46:57.551195+0000","last_peered":"2024-09-10T22:46:57.551195+0000","last_clean":"2024-09-10T22:46:57.551195+0000","last_became_active":"2024-09-10T22:45:18.943312+0000","last_became_peered":"2024-09-10T22:45:18.943312+0000","last_unstale":"2024-09-10T22:46:57.551195+0000","last_undegraded":"2024-09-10T22:46:57.551195+0000","last_fullsized":"2024-09-10T22:46:57.551195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:31.998638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541587+0000","last_change":"2024-09-10T22:45:18.948069+0000","last_active":"2024-09-10T22:46:57.541587+0000","last_peered":"2024-09-10T22:46:57.541587+0000","last_clean":"2024-09-10T22:46:57.541587+0000","last_became_active":"2024-09-10T22:45:18.947491+0000","last_became_peered":"2024-09-10T22:45:18.947491+0000","last_unstale":"2024-09-10T22:46:57.541587+0000","last_undegraded":"2024-09-10T22:46:57.541587+0000","last_fullsized":"2024-09-10T22:46:57.541587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:25.646325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543325+0000","last_change":"2024-09-10T22:45:58.301222+0000","last_active":"2024-09-10T22:46:57.543325+0000","last_peered":"2024-09-10T22:46:57.543325+0000","last_clean":"2024-09-10T22:46:57.543325+0000","last_became_active":"2024-09-10T22:45:58.300835+0000","last_became_peered":"2024-09-10T22:45:58.300835+0000","last_unstale":"2024-09-10T22:46:57.543325+0000","last_undegraded":"2024-09-10T22:46:57.543325+0000","last_fullsized":"2024-09-10T22:46:57.543325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:16.772354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541083+0000","last_change":"2024-09-10T22:45:18.949940+0000","last_active":"2024-09-10T22:46:57.541083+0000","last_peered":"2024-09-10T22:46:57.541083+0000","last_clean":"2024-09-10T22:46:57.541083+0000","last_became_active":"2024-09-10T22:45:18.949592+0000","last_became_peered":"2024-09-10T22:45:18.949592+0000","last_unstale":"2024-09-10T22:46:57.541083+0000","last_undegraded":"2024-09-10T22:46:57.541083+0000","last_fullsized":"2024-09-10T22:46:57.541083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:55.350516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538073+0000","last_change":"2024-09-10T22:45:18.938146+0000","last_active":"2024-09-10T22:46:57.538073+0000","last_peered":"2024-09-10T22:46:57.538073+0000","last_clean":"2024-09-10T22:46:57.538073+0000","last_became_active":"2024-09-10T22:45:18.937309+0000","last_became_peered":"2024-09-10T22:45:18.937309+0000","last_unstale":"2024-09-10T22:46:57.538073+0000","last_undegraded":"2024-09-10T22:46:57.538073+0000","last_fullsized":"2024-09-10T22:46:57.538073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:29.991054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.540443+0000","last_change":"2024-09-10T22:45:18.943488+0000","last_active":"2024-09-10T22:46:57.540443+0000","last_peered":"2024-09-10T22:46:57.540443+0000","last_clean":"2024-09-10T22:46:57.540443+0000","last_became_active":"2024-09-10T22:45:18.943051+0000","last_became_peered":"2024-09-10T22:45:18.943051+0000","last_unstale":"2024-09-10T22:46:57.540443+0000","last_undegraded":"2024-09-10T22:46:57.540443+0000","last_fullsized":"2024-09-10T22:46:57.540443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:02.210343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.545371+0000","last_change":"2024-09-10T22:45:18.948379+0000","last_active":"2024-09-10T22:46:57.545371+0000","last_peered":"2024-09-10T22:46:57.545371+0000","last_clean":"2024-09-10T22:46:57.545371+0000","last_became_active":"2024-09-10T22:45:18.947766+0000","last_became_peered":"2024-09-10T22:45:18.947766+0000","last_unstale":"2024-09-10T22:46:57.545371+0000","last_undegraded":"2024-09-10T22:46:57.545371+0000","last_fullsized":"2024-09-10T22:46:57.545371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:47.721759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542915+0000","last_change":"2024-09-10T22:45:18.943206+0000","last_active":"2024-09-10T22:46:57.542915+0000","last_peered":"2024-09-10T22:46:57.542915+0000","last_clean":"2024-09-10T22:46:57.542915+0000","last_became_active":"2024-09-10T22:45:18.942863+0000","last_became_peered":"2024-09-10T22:45:18.942863+0000","last_unstale":"2024-09-10T22:46:57.542915+0000","last_undegraded":"2024-09-10T22:46:57.542915+0000","last_fullsized":"2024-09-10T22:46:57.542915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:07.052771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.197033+0000","last_change":"2024-09-10T22:45:16.931653+0000","last_active":"2024-09-10T22:47:11.197033+0000","last_peered":"2024-09-10T22:47:11.197033+0000","last_clean":"2024-09-10T22:47:11.197033+0000","last_became_active":"2024-09-10T22:45:16.931194+0000","last_became_peered":"2024-09-10T22:45:16.931194+0000","last_unstale":"2024-09-10T22:47:11.197033+0000","last_undegraded":"2024-09-10T22:47:11.197033+0000","last_fullsized":"2024-09-10T22:47:11.197033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:52.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549257+0000","last_change":"2024-09-10T22:45:18.954162+0000","last_active":"2024-09-10T22:46:57.549257+0000","last_peered":"2024-09-10T22:46:57.549257+0000","last_clean":"2024-09-10T22:46:57.549257+0000","last_became_active":"2024-09-10T22:45:18.953804+0000","last_became_peered":"2024-09-10T22:45:18.953804+0000","last_unstale":"2024-09-10T22:46:57.549257+0000","last_undegraded":"2024-09-10T22:46:57.549257+0000","last_fullsized":"2024-09-10T22:46:57.549257+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:27.276940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.306152+0000","last_change":"2024-09-10T22:45:58.306070+0000","last_active":"2024-09-10T22:45:58.306152+0000","last_peered":"2024-09-10T22:45:58.306152+0000","last_clean":"2024-09-10T22:45:58.306152+0000","last_became_active":"2024-09-10T22:45:58.300281+0000","last_became_peered":"2024-09-10T22:45:58.300281+0000","last_unstale":"2024-09-10T22:45:58.306152+0000","last_undegraded":"2024-09-10T22:45:58.306152+0000","last_fullsized":"2024-09-10T22:45:58.306152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:01.832411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297848+0000","last_change":"2024-09-10T22:45:16.927924+0000","last_active":"2024-09-10T22:45:58.297848+0000","last_peered":"2024-09-10T22:45:58.297848+0000","last_clean":"2024-09-10T22:45:58.297848+0000","last_became_active":"2024-09-10T22:45:16.927629+0000","last_became_peered":"2024-09-10T22:45:16.927629+0000","last_unstale":"2024-09-10T22:45:58.297848+0000","last_undegraded":"2024-09-10T22:45:58.297848+0000","last_fullsized":"2024-09-10T22:45:58.297848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:29.137757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.543952+0000","last_change":"2024-09-10T22:45:18.939948+0000","last_active":"2024-09-10T22:46:57.543952+0000","last_peered":"2024-09-10T22:46:57.543952+0000","last_clean":"2024-09-10T22:46:57.543952+0000","last_became_active":"2024-09-10T22:45:18.939506+0000","last_became_peered":"2024-09-10T22:45:18.939506+0000","last_unstale":"2024-09-10T22:46:57.543952+0000","last_undegraded":"2024-09-10T22:46:57.543952+0000","last_fullsized":"2024-09-10T22:46:57.543952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:19.252811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.936015+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.935521+0000","last_became_peered":"2024-09-10T22:45:16.935521+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:17.044936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.546120+0000","last_change":"2024-09-10T22:45:18.941332+0000","last_active":"2024-09-10T22:46:57.546120+0000","last_peered":"2024-09-10T22:46:57.546120+0000","last_clean":"2024-09-10T22:46:57.546120+0000","last_became_active":"2024-09-10T22:45:18.940054+0000","last_became_peered":"2024-09-10T22:45:18.940054+0000","last_unstale":"2024-09-10T22:46:57.546120+0000","last_undegraded":"2024-09-10T22:46:57.546120+0000","last_fullsized":"2024-09-10T22:46:57.546120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:07.802784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":233,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.198974+0000","last_change":"2024-09-10T22:45:16.935414+0000","last_active":"2024-09-10T22:47:13.198974+0000","last_peered":"2024-09-10T22:47:13.198974+0000","last_clean":"2024-09-10T22:47:13.198974+0000","last_became_active":"2024-09-10T22:45:16.935086+0000","last_became_peered":"2024-09-10T22:45:16.935086+0000","last_unstale":"2024-09-10T22:47:13.198974+0000","last_undegraded":"2024-09-10T22:47:13.198974+0000","last_fullsized":"2024-09-10T22:47:13.198974+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:09.009196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.551796+0000","last_change":"2024-09-10T22:45:18.941932+0000","last_active":"2024-09-10T22:46:57.551796+0000","last_peered":"2024-09-10T22:46:57.551796+0000","last_clean":"2024-09-10T22:46:57.551796+0000","last_became_active":"2024-09-10T22:45:18.941497+0000","last_became_peered":"2024-09-10T22:45:18.941497+0000","last_unstale":"2024-09-10T22:46:57.551796+0000","last_undegraded":"2024-09-10T22:46:57.551796+0000","last_fullsized":"2024-09-10T22:46:57.551796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:41.913821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680757+0000","last_change":"2024-09-10T22:45:16.935221+0000","last_active":"2024-09-10T22:45:58.680757+0000","last_peered":"2024-09-10T22:45:58.680757+0000","last_clean":"2024-09-10T22:45:58.680757+0000","last_became_active":"2024-09-10T22:45:16.934861+0000","last_became_peered":"2024-09-10T22:45:16.934861+0000","last_unstale":"2024-09-10T22:45:58.680757+0000","last_undegraded":"2024-09-10T22:45:58.680757+0000","last_fullsized":"2024-09-10T22:45:58.680757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:06.390389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.541828+0000","last_change":"2024-09-10T22:45:18.946155+0000","last_active":"2024-09-10T22:46:57.541828+0000","last_peered":"2024-09-10T22:46:57.541828+0000","last_clean":"2024-09-10T22:46:57.541828+0000","last_became_active":"2024-09-10T22:45:18.943190+0000","last_became_peered":"2024-09-10T22:45:18.943190+0000","last_unstale":"2024-09-10T22:46:57.541828+0000","last_undegraded":"2024-09-10T22:46:57.541828+0000","last_fullsized":"2024-09-10T22:46:57.541828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:51.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672340+0000","last_change":"2024-09-10T22:45:16.990411+0000","last_active":"2024-09-10T22:45:58.672340+0000","last_peered":"2024-09-10T22:45:58.672340+0000","last_clean":"2024-09-10T22:45:58.672340+0000","last_became_active":"2024-09-10T22:45:16.990052+0000","last_became_peered":"2024-09-10T22:45:16.990052+0000","last_unstale":"2024-09-10T22:45:58.672340+0000","last_undegraded":"2024-09-10T22:45:58.672340+0000","last_fullsized":"2024-09-10T22:45:58.672340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:09.637464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556911+0000","last_change":"2024-09-10T22:45:18.952235+0000","last_active":"2024-09-10T22:46:57.556911+0000","last_peered":"2024-09-10T22:46:57.556911+0000","last_clean":"2024-09-10T22:46:57.556911+0000","last_became_active":"2024-09-10T22:45:18.951865+0000","last_became_peered":"2024-09-10T22:45:18.951865+0000","last_unstale":"2024-09-10T22:46:57.556911+0000","last_undegraded":"2024-09-10T22:46:57.556911+0000","last_fullsized":"2024-09-10T22:46:57.556911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:33.141958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199739+0000","last_change":"2024-09-10T22:45:16.936723+0000","last_active":"2024-09-10T22:47:13.199739+0000","last_peered":"2024-09-10T22:47:13.199739+0000","last_clean":"2024-09-10T22:47:13.199739+0000","last_became_active":"2024-09-10T22:45:16.936280+0000","last_became_peered":"2024-09-10T22:45:16.936280+0000","last_unstale":"2024-09-10T22:47:13.199739+0000","last_undegraded":"2024-09-10T22:47:13.199739+0000","last_fullsized":"2024-09-10T22:47:13.199739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:26.155443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60319,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":217,"num_read_kb":167,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.555331+0000","last_change":"2024-09-10T22:45:18.946315+0000","last_active":"2024-09-10T22:46:57.555331+0000","last_peered":"2024-09-10T22:46:57.555331+0000","last_clean":"2024-09-10T22:46:57.555331+0000","last_became_active":"2024-09-10T22:45:18.945952+0000","last_became_peered":"2024-09-10T22:45:18.945952+0000","last_unstale":"2024-09-10T22:46:57.555331+0000","last_undegraded":"2024-09-10T22:46:57.555331+0000","last_fullsized":"2024-09-10T22:46:57.555331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:29.365766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303539+0000","last_change":"2024-09-10T22:45:16.937218+0000","last_active":"2024-09-10T22:45:58.303539+0000","last_peered":"2024-09-10T22:45:58.303539+0000","last_clean":"2024-09-10T22:45:58.303539+0000","last_became_active":"2024-09-10T22:45:16.936869+0000","last_became_peered":"2024-09-10T22:45:16.936869+0000","last_unstale":"2024-09-10T22:45:58.303539+0000","last_undegraded":"2024-09-10T22:45:58.303539+0000","last_fullsized":"2024-09-10T22:45:58.303539+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:25.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.552322+0000","last_change":"2024-09-10T22:45:18.937409+0000","last_active":"2024-09-10T22:46:57.552322+0000","last_peered":"2024-09-10T22:46:57.552322+0000","last_clean":"2024-09-10T22:46:57.552322+0000","last_became_active":"2024-09-10T22:45:18.937070+0000","last_became_peered":"2024-09-10T22:45:18.937070+0000","last_unstale":"2024-09-10T22:46:57.552322+0000","last_undegraded":"2024-09-10T22:46:57.552322+0000","last_fullsized":"2024-09-10T22:46:57.552322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:34.197931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672276+0000","last_change":"2024-09-10T22:45:16.932801+0000","last_active":"2024-09-10T22:45:58.672276+0000","last_peered":"2024-09-10T22:45:58.672276+0000","last_clean":"2024-09-10T22:45:58.672276+0000","last_became_active":"2024-09-10T22:45:16.932480+0000","last_became_peered":"2024-09-10T22:45:16.932480+0000","last_unstale":"2024-09-10T22:45:58.672276+0000","last_undegraded":"2024-09-10T22:45:58.672276+0000","last_fullsized":"2024-09-10T22:45:58.672276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:02.227425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556243+0000","last_change":"2024-09-10T22:45:18.950689+0000","last_active":"2024-09-10T22:46:57.556243+0000","last_peered":"2024-09-10T22:46:57.556243+0000","last_clean":"2024-09-10T22:46:57.556243+0000","last_became_active":"2024-09-10T22:45:18.950322+0000","last_became_peered":"2024-09-10T22:45:18.950322+0000","last_unstale":"2024-09-10T22:46:57.556243+0000","last_undegraded":"2024-09-10T22:46:57.556243+0000","last_fullsized":"2024-09-10T22:46:57.556243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:11.844601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:12.208490+0000","last_change":"2024-09-10T22:45:16.932872+0000","last_active":"2024-09-10T22:47:12.208490+0000","last_peered":"2024-09-10T22:47:12.208490+0000","last_clean":"2024-09-10T22:47:12.208490+0000","last_became_active":"2024-09-10T22:45:16.932494+0000","last_became_peered":"2024-09-10T22:45:16.932494+0000","last_unstale":"2024-09-10T22:47:12.208490+0000","last_undegraded":"2024-09-10T22:47:12.208490+0000","last_fullsized":"2024-09-10T22:47:12.208490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:00.625799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558382+0000","last_change":"2024-09-10T22:45:18.947537+0000","last_active":"2024-09-10T22:46:57.558382+0000","last_peered":"2024-09-10T22:46:57.558382+0000","last_clean":"2024-09-10T22:46:57.558382+0000","last_became_active":"2024-09-10T22:45:18.947163+0000","last_became_peered":"2024-09-10T22:45:18.947163+0000","last_unstale":"2024-09-10T22:46:57.558382+0000","last_undegraded":"2024-09-10T22:46:57.558382+0000","last_fullsized":"2024-09-10T22:46:57.558382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:32.744822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298597+0000","last_change":"2024-09-10T22:45:16.990506+0000","last_active":"2024-09-10T22:45:58.298597+0000","last_peered":"2024-09-10T22:45:58.298597+0000","last_clean":"2024-09-10T22:45:58.298597+0000","last_became_active":"2024-09-10T22:45:16.990064+0000","last_became_peered":"2024-09-10T22:45:16.990064+0000","last_unstale":"2024-09-10T22:45:58.298597+0000","last_undegraded":"2024-09-10T22:45:58.298597+0000","last_fullsized":"2024-09-10T22:45:58.298597+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:48.353374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556870+0000","last_change":"2024-09-10T22:45:18.950749+0000","last_active":"2024-09-10T22:46:57.556870+0000","last_peered":"2024-09-10T22:46:57.556870+0000","last_clean":"2024-09-10T22:46:57.556870+0000","last_became_active":"2024-09-10T22:45:18.950409+0000","last_became_peered":"2024-09-10T22:45:18.950409+0000","last_unstale":"2024-09-10T22:46:57.556870+0000","last_undegraded":"2024-09-10T22:46:57.556870+0000","last_fullsized":"2024-09-10T22:46:57.556870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:35.913642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299286+0000","last_change":"2024-09-10T22:45:16.934245+0000","last_active":"2024-09-10T22:45:58.299286+0000","last_peered":"2024-09-10T22:45:58.299286+0000","last_clean":"2024-09-10T22:45:58.299286+0000","last_became_active":"2024-09-10T22:45:16.933785+0000","last_became_peered":"2024-09-10T22:45:16.933785+0000","last_unstale":"2024-09-10T22:45:58.299286+0000","last_undegraded":"2024-09-10T22:45:58.299286+0000","last_fullsized":"2024-09-10T22:45:58.299286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:07.793852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.548879+0000","last_change":"2024-09-10T22:45:18.949864+0000","last_active":"2024-09-10T22:46:57.548879+0000","last_peered":"2024-09-10T22:46:57.548879+0000","last_clean":"2024-09-10T22:46:57.548879+0000","last_became_active":"2024-09-10T22:45:18.949554+0000","last_became_peered":"2024-09-10T22:45:18.949554+0000","last_unstale":"2024-09-10T22:46:57.548879+0000","last_undegraded":"2024-09-10T22:46:57.548879+0000","last_fullsized":"2024-09-10T22:46:57.548879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:08.259447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298492+0000","last_change":"2024-09-10T22:45:16.928246+0000","last_active":"2024-09-10T22:45:58.298492+0000","last_peered":"2024-09-10T22:45:58.298492+0000","last_clean":"2024-09-10T22:45:58.298492+0000","last_became_active":"2024-09-10T22:45:16.927305+0000","last_became_peered":"2024-09-10T22:45:16.927305+0000","last_unstale":"2024-09-10T22:45:58.298492+0000","last_undegraded":"2024-09-10T22:45:58.298492+0000","last_fullsized":"2024-09-10T22:45:58.298492+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:36.261002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558447+0000","last_change":"2024-09-10T22:45:18.942167+0000","last_active":"2024-09-10T22:46:57.558447+0000","last_peered":"2024-09-10T22:46:57.558447+0000","last_clean":"2024-09-10T22:46:57.558447+0000","last_became_active":"2024-09-10T22:45:18.941057+0000","last_became_peered":"2024-09-10T22:45:18.941057+0000","last_unstale":"2024-09-10T22:46:57.558447+0000","last_undegraded":"2024-09-10T22:46:57.558447+0000","last_fullsized":"2024-09-10T22:46:57.558447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:39.910296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.672395+0000","last_change":"2024-09-10T22:45:16.933641+0000","last_active":"2024-09-10T22:45:58.672395+0000","last_peered":"2024-09-10T22:45:58.672395+0000","last_clean":"2024-09-10T22:45:58.672395+0000","last_became_active":"2024-09-10T22:45:16.933295+0000","last_became_peered":"2024-09-10T22:45:16.933295+0000","last_unstale":"2024-09-10T22:45:58.672395+0000","last_undegraded":"2024-09-10T22:45:58.672395+0000","last_fullsized":"2024-09-10T22:45:58.672395+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:42.889658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.554711+0000","last_change":"2024-09-10T22:45:18.941902+0000","last_active":"2024-09-10T22:46:57.554711+0000","last_peered":"2024-09-10T22:46:57.554711+0000","last_clean":"2024-09-10T22:46:57.554711+0000","last_became_active":"2024-09-10T22:45:18.941581+0000","last_became_peered":"2024-09-10T22:45:18.941581+0000","last_unstale":"2024-09-10T22:46:57.554711+0000","last_undegraded":"2024-09-10T22:46:57.554711+0000","last_fullsized":"2024-09-10T22:46:57.554711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:03.419939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.196964+0000","last_change":"2024-09-10T22:45:17.373066+0000","last_active":"2024-09-10T22:47:11.196964+0000","last_peered":"2024-09-10T22:47:11.196964+0000","last_clean":"2024-09-10T22:47:11.196964+0000","last_became_active":"2024-09-10T22:45:17.372651+0000","last_became_peered":"2024-09-10T22:45:17.372651+0000","last_unstale":"2024-09-10T22:47:11.196964+0000","last_undegraded":"2024-09-10T22:47:11.196964+0000","last_fullsized":"2024-09-10T22:47:11.196964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:25.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557894+0000","last_change":"2024-09-10T22:45:18.946004+0000","last_active":"2024-09-10T22:46:57.557894+0000","last_peered":"2024-09-10T22:46:57.557894+0000","last_clean":"2024-09-10T22:46:57.557894+0000","last_became_active":"2024-09-10T22:45:18.945366+0000","last_became_peered":"2024-09-10T22:45:18.945366+0000","last_unstale":"2024-09-10T22:46:57.557894+0000","last_undegraded":"2024-09-10T22:46:57.557894+0000","last_fullsized":"2024-09-10T22:46:57.557894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:22.450821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680641+0000","last_change":"2024-09-10T22:44:04.533019+0000","last_active":"2024-09-10T22:45:58.680641+0000","last_peered":"2024-09-10T22:45:58.680641+0000","last_clean":"2024-09-10T22:45:58.680641+0000","last_became_active":"2024-09-10T22:44:04.504099+0000","last_became_peered":"2024-09-10T22:44:04.504099+0000","last_unstale":"2024-09-10T22:45:58.680641+0000","last_undegraded":"2024-09-10T22:45:58.680641+0000","last_fullsized":"2024-09-10T22:45:58.680641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:40:37.957730+0000","last_clean_scrub_stamp":"2024-09-10T22:40:37.957730+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:13.162214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296623+0000","last_change":"2024-09-10T22:45:16.934065+0000","last_active":"2024-09-10T22:45:58.296623+0000","last_peered":"2024-09-10T22:45:58.296623+0000","last_clean":"2024-09-10T22:45:58.296623+0000","last_became_active":"2024-09-10T22:45:16.933601+0000","last_became_peered":"2024-09-10T22:45:16.933601+0000","last_unstale":"2024-09-10T22:45:58.296623+0000","last_undegraded":"2024-09-10T22:45:58.296623+0000","last_fullsized":"2024-09-10T22:45:58.296623+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:42.228702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550173+0000","last_change":"2024-09-10T22:45:18.943582+0000","last_active":"2024-09-10T22:46:57.550173+0000","last_peered":"2024-09-10T22:46:57.550173+0000","last_clean":"2024-09-10T22:46:57.550173+0000","last_became_active":"2024-09-10T22:45:18.943307+0000","last_became_peered":"2024-09-10T22:45:18.943307+0000","last_unstale":"2024-09-10T22:46:57.550173+0000","last_undegraded":"2024-09-10T22:46:57.550173+0000","last_fullsized":"2024-09-10T22:46:57.550173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:17.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680509+0000","last_change":"2024-09-10T22:45:16.932666+0000","last_active":"2024-09-10T22:45:58.680509+0000","last_peered":"2024-09-10T22:45:58.680509+0000","last_clean":"2024-09-10T22:45:58.680509+0000","last_became_active":"2024-09-10T22:45:16.932371+0000","last_became_peered":"2024-09-10T22:45:16.932371+0000","last_unstale":"2024-09-10T22:45:58.680509+0000","last_undegraded":"2024-09-10T22:45:58.680509+0000","last_fullsized":"2024-09-10T22:45:58.680509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:12.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544946+0000","last_change":"2024-09-10T22:45:18.942080+0000","last_active":"2024-09-10T22:46:57.544946+0000","last_peered":"2024-09-10T22:46:57.544946+0000","last_clean":"2024-09-10T22:46:57.544946+0000","last_became_active":"2024-09-10T22:45:18.941764+0000","last_became_peered":"2024-09-10T22:45:18.941764+0000","last_unstale":"2024-09-10T22:46:57.544946+0000","last_undegraded":"2024-09-10T22:46:57.544946+0000","last_fullsized":"2024-09-10T22:46:57.544946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:09.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:14.210466+0000","last_change":"2024-09-10T22:45:16.990535+0000","last_active":"2024-09-10T22:47:14.210466+0000","last_peered":"2024-09-10T22:47:14.210466+0000","last_clean":"2024-09-10T22:47:14.210466+0000","last_became_active":"2024-09-10T22:45:16.989782+0000","last_became_peered":"2024-09-10T22:45:16.989782+0000","last_unstale":"2024-09-10T22:47:14.210466+0000","last_undegraded":"2024-09-10T22:47:14.210466+0000","last_fullsized":"2024-09-10T22:47:14.210466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:21.329405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544012+0000","last_change":"2024-09-10T22:45:58.304503+0000","last_active":"2024-09-10T22:46:57.544012+0000","last_peered":"2024-09-10T22:46:57.544012+0000","last_clean":"2024-09-10T22:46:57.544012+0000","last_became_active":"2024-09-10T22:45:58.304243+0000","last_became_peered":"2024-09-10T22:45:58.304243+0000","last_unstale":"2024-09-10T22:46:57.544012+0000","last_undegraded":"2024-09-10T22:46:57.544012+0000","last_fullsized":"2024-09-10T22:46:57.544012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:33.190092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"77'6","reported_seq":135,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:13.199731+0000","last_change":"2024-09-10T22:45:16.937410+0000","last_active":"2024-09-10T22:47:13.199731+0000","last_peered":"2024-09-10T22:47:13.199731+0000","last_clean":"2024-09-10T22:47:13.199731+0000","last_became_active":"2024-09-10T22:45:16.935794+0000","last_became_peered":"2024-09-10T22:45:16.935794+0000","last_unstale":"2024-09-10T22:47:13.199731+0000","last_undegraded":"2024-09-10T22:47:13.199731+0000","last_fullsized":"2024-09-10T22:47:13.199731+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:11.198008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.549451+0000","last_change":"2024-09-10T22:45:18.950482+0000","last_active":"2024-09-10T22:46:57.549451+0000","last_peered":"2024-09-10T22:46:57.549451+0000","last_clean":"2024-09-10T22:46:57.549451+0000","last_became_active":"2024-09-10T22:45:18.950190+0000","last_became_peered":"2024-09-10T22:45:18.950190+0000","last_unstale":"2024-09-10T22:46:57.549451+0000","last_undegraded":"2024-09-10T22:46:57.549451+0000","last_fullsized":"2024-09-10T22:46:57.549451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.285007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.599710+0000","last_change":"2024-09-10T22:45:58.599612+0000","last_active":"2024-09-10T22:45:58.599710+0000","last_peered":"2024-09-10T22:45:58.599710+0000","last_clean":"2024-09-10T22:45:58.599710+0000","last_became_active":"2024-09-10T22:45:58.599226+0000","last_became_peered":"2024-09-10T22:45:58.599226+0000","last_unstale":"2024-09-10T22:45:58.599710+0000","last_undegraded":"2024-09-10T22:45:58.599710+0000","last_fullsized":"2024-09-10T22:45:58.599710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:20.112571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544799+0000","last_change":"2024-09-10T22:45:18.945181+0000","last_active":"2024-09-10T22:46:57.544799+0000","last_peered":"2024-09-10T22:46:57.544799+0000","last_clean":"2024-09-10T22:46:57.544799+0000","last_became_active":"2024-09-10T22:45:18.944868+0000","last_became_peered":"2024-09-10T22:45:18.944868+0000","last_unstale":"2024-09-10T22:46:57.544799+0000","last_undegraded":"2024-09-10T22:46:57.544799+0000","last_fullsized":"2024-09-10T22:46:57.544799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:06.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299088+0000","last_change":"2024-09-10T22:45:16.990725+0000","last_active":"2024-09-10T22:45:58.299088+0000","last_peered":"2024-09-10T22:45:58.299088+0000","last_clean":"2024-09-10T22:45:58.299088+0000","last_became_active":"2024-09-10T22:45:16.989761+0000","last_became_peered":"2024-09-10T22:45:16.989761+0000","last_unstale":"2024-09-10T22:45:58.299088+0000","last_undegraded":"2024-09-10T22:45:58.299088+0000","last_fullsized":"2024-09-10T22:45:58.299088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:52.498907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542853+0000","last_change":"2024-09-10T22:45:18.932761+0000","last_active":"2024-09-10T22:46:57.542853+0000","last_peered":"2024-09-10T22:46:57.542853+0000","last_clean":"2024-09-10T22:46:57.542853+0000","last_became_active":"2024-09-10T22:45:18.932011+0000","last_became_peered":"2024-09-10T22:45:18.932011+0000","last_unstale":"2024-09-10T22:46:57.542853+0000","last_undegraded":"2024-09-10T22:46:57.542853+0000","last_fullsized":"2024-09-10T22:46:57.542853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:26.317694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'15","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.297683+0000","last_change":"2024-09-10T22:45:16.929726+0000","last_active":"2024-09-10T22:45:58.297683+0000","last_peered":"2024-09-10T22:45:58.297683+0000","last_clean":"2024-09-10T22:45:58.297683+0000","last_became_active":"2024-09-10T22:45:16.929266+0000","last_became_peered":"2024-09-10T22:45:16.929266+0000","last_unstale":"2024-09-10T22:45:58.297683+0000","last_undegraded":"2024-09-10T22:45:58.297683+0000","last_fullsized":"2024-09-10T22:45:58.297683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:02.204254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.556425+0000","last_change":"2024-09-10T22:45:18.942886+0000","last_active":"2024-09-10T22:46:57.556425+0000","last_peered":"2024-09-10T22:46:57.556425+0000","last_clean":"2024-09-10T22:46:57.556425+0000","last_became_active":"2024-09-10T22:45:18.942595+0000","last_became_peered":"2024-09-10T22:45:18.942595+0000","last_unstale":"2024-09-10T22:46:57.556425+0000","last_undegraded":"2024-09-10T22:46:57.556425+0000","last_fullsized":"2024-09-10T22:46:57.556425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:05.904929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.296692+0000","last_change":"2024-09-10T22:45:17.372933+0000","last_active":"2024-09-10T22:45:58.296692+0000","last_peered":"2024-09-10T22:45:58.296692+0000","last_clean":"2024-09-10T22:45:58.296692+0000","last_became_active":"2024-09-10T22:45:17.372597+0000","last_became_peered":"2024-09-10T22:45:17.372597+0000","last_unstale":"2024-09-10T22:45:58.296692+0000","last_undegraded":"2024-09-10T22:45:58.296692+0000","last_fullsized":"2024-09-10T22:45:58.296692+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:06.152441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544837+0000","last_change":"2024-09-10T22:45:18.944739+0000","last_active":"2024-09-10T22:46:57.544837+0000","last_peered":"2024-09-10T22:46:57.544837+0000","last_clean":"2024-09-10T22:46:57.544837+0000","last_became_active":"2024-09-10T22:45:18.943901+0000","last_became_peered":"2024-09-10T22:45:18.943901+0000","last_unstale":"2024-09-10T22:46:57.544837+0000","last_undegraded":"2024-09-10T22:46:57.544837+0000","last_fullsized":"2024-09-10T22:46:57.544837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:13.721963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.602110+0000","last_change":"2024-09-10T22:45:58.602021+0000","last_active":"2024-09-10T22:45:58.602110+0000","last_peered":"2024-09-10T22:45:58.602110+0000","last_clean":"2024-09-10T22:45:58.602110+0000","last_became_active":"2024-09-10T22:45:58.599358+0000","last_became_peered":"2024-09-10T22:45:58.599358+0000","last_unstale":"2024-09-10T22:45:58.602110+0000","last_undegraded":"2024-09-10T22:45:58.602110+0000","last_fullsized":"2024-09-10T22:45:58.602110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:40.510120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.530711+0000","last_change":"2024-09-10T22:45:18.947884+0000","last_active":"2024-09-10T22:46:57.530711+0000","last_peered":"2024-09-10T22:46:57.530711+0000","last_clean":"2024-09-10T22:46:57.530711+0000","last_became_active":"2024-09-10T22:45:18.947544+0000","last_became_peered":"2024-09-10T22:45:18.947544+0000","last_unstale":"2024-09-10T22:46:57.530711+0000","last_undegraded":"2024-09-10T22:46:57.530711+0000","last_fullsized":"2024-09-10T22:46:57.530711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:02.670859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680327+0000","last_change":"2024-09-10T22:45:16.931198+0000","last_active":"2024-09-10T22:45:58.680327+0000","last_peered":"2024-09-10T22:45:58.680327+0000","last_clean":"2024-09-10T22:45:58.680327+0000","last_became_active":"2024-09-10T22:45:16.930892+0000","last_became_peered":"2024-09-10T22:45:16.930892+0000","last_unstale":"2024-09-10T22:45:58.680327+0000","last_undegraded":"2024-09-10T22:45:58.680327+0000","last_fullsized":"2024-09-10T22:45:58.680327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:14.416673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.544878+0000","last_change":"2024-09-10T22:45:18.945647+0000","last_active":"2024-09-10T22:46:57.544878+0000","last_peered":"2024-09-10T22:46:57.544878+0000","last_clean":"2024-09-10T22:46:57.544878+0000","last_became_active":"2024-09-10T22:45:18.945313+0000","last_became_peered":"2024-09-10T22:45:18.945313+0000","last_unstale":"2024-09-10T22:46:57.544878+0000","last_undegraded":"2024-09-10T22:46:57.544878+0000","last_fullsized":"2024-09-10T22:46:57.544878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:48.658675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.303229+0000","last_change":"2024-09-10T22:45:16.930047+0000","last_active":"2024-09-10T22:45:58.303229+0000","last_peered":"2024-09-10T22:45:58.303229+0000","last_clean":"2024-09-10T22:45:58.303229+0000","last_became_active":"2024-09-10T22:45:16.929579+0000","last_became_peered":"2024-09-10T22:45:16.929579+0000","last_unstale":"2024-09-10T22:45:58.303229+0000","last_undegraded":"2024-09-10T22:45:58.303229+0000","last_fullsized":"2024-09-10T22:45:58.303229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:32.621131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.542907+0000","last_change":"2024-09-10T22:45:18.944928+0000","last_active":"2024-09-10T22:46:57.542907+0000","last_peered":"2024-09-10T22:46:57.542907+0000","last_clean":"2024-09-10T22:46:57.542907+0000","last_became_active":"2024-09-10T22:45:18.944634+0000","last_became_peered":"2024-09-10T22:45:18.944634+0000","last_unstale":"2024-09-10T22:46:57.542907+0000","last_undegraded":"2024-09-10T22:46:57.542907+0000","last_fullsized":"2024-09-10T22:46:57.542907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:48.989191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298787+0000","last_change":"2024-09-10T22:45:16.934491+0000","last_active":"2024-09-10T22:45:58.298787+0000","last_peered":"2024-09-10T22:45:58.298787+0000","last_clean":"2024-09-10T22:45:58.298787+0000","last_became_active":"2024-09-10T22:45:16.934146+0000","last_became_peered":"2024-09-10T22:45:16.934146+0000","last_unstale":"2024-09-10T22:45:58.298787+0000","last_undegraded":"2024-09-10T22:45:58.298787+0000","last_fullsized":"2024-09-10T22:45:58.298787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:18.060347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558093+0000","last_change":"2024-09-10T22:45:18.937588+0000","last_active":"2024-09-10T22:46:57.558093+0000","last_peered":"2024-09-10T22:46:57.558093+0000","last_clean":"2024-09-10T22:46:57.558093+0000","last_became_active":"2024-09-10T22:45:18.937174+0000","last_became_peered":"2024-09-10T22:45:18.937174+0000","last_unstale":"2024-09-10T22:46:57.558093+0000","last_undegraded":"2024-09-10T22:46:57.558093+0000","last_fullsized":"2024-09-10T22:46:57.558093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:31:52.454804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.557317+0000","last_change":"2024-09-10T22:45:18.951025+0000","last_active":"2024-09-10T22:46:57.557317+0000","last_peered":"2024-09-10T22:46:57.557317+0000","last_clean":"2024-09-10T22:46:57.557317+0000","last_became_active":"2024-09-10T22:45:18.950449+0000","last_became_peered":"2024-09-10T22:45:18.950449+0000","last_unstale":"2024-09-10T22:46:57.557317+0000","last_undegraded":"2024-09-10T22:46:57.557317+0000","last_fullsized":"2024-09-10T22:46:57.557317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:26.695379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.298925+0000","last_change":"2024-09-10T22:45:16.935528+0000","last_active":"2024-09-10T22:45:58.298925+0000","last_peered":"2024-09-10T22:45:58.298925+0000","last_clean":"2024-09-10T22:45:58.298925+0000","last_became_active":"2024-09-10T22:45:16.935204+0000","last_became_peered":"2024-09-10T22:45:16.935204+0000","last_unstale":"2024-09-10T22:45:58.298925+0000","last_undegraded":"2024-09-10T22:45:58.298925+0000","last_fullsized":"2024-09-10T22:45:58.298925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:39.716111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.558535+0000","last_change":"2024-09-10T22:45:18.940839+0000","last_active":"2024-09-10T22:46:57.558535+0000","last_peered":"2024-09-10T22:46:57.558535+0000","last_clean":"2024-09-10T22:46:57.558535+0000","last_became_active":"2024-09-10T22:45:18.940427+0000","last_became_peered":"2024-09-10T22:45:18.940427+0000","last_unstale":"2024-09-10T22:46:57.558535+0000","last_undegraded":"2024-09-10T22:46:57.558535+0000","last_fullsized":"2024-09-10T22:46:57.558535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:00.266762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.299045+0000","last_change":"2024-09-10T22:45:16.932522+0000","last_active":"2024-09-10T22:45:58.299045+0000","last_peered":"2024-09-10T22:45:58.299045+0000","last_clean":"2024-09-10T22:45:58.299045+0000","last_became_active":"2024-09-10T22:45:16.932088+0000","last_became_peered":"2024-09-10T22:45:16.932088+0000","last_unstale":"2024-09-10T22:45:58.299045+0000","last_undegraded":"2024-09-10T22:45:58.299045+0000","last_fullsized":"2024-09-10T22:45:58.299045+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:49.180382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.550030+0000","last_change":"2024-09-10T22:45:18.940563+0000","last_active":"2024-09-10T22:46:57.550030+0000","last_peered":"2024-09-10T22:46:57.550030+0000","last_clean":"2024-09-10T22:46:57.550030+0000","last_became_active":"2024-09-10T22:45:18.940107+0000","last_became_peered":"2024-09-10T22:45:18.940107+0000","last_unstale":"2024-09-10T22:46:57.550030+0000","last_undegraded":"2024-09-10T22:46:57.550030+0000","last_fullsized":"2024-09-10T22:46:57.550030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:25.425331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:45:58.680612+0000","last_change":"2024-09-10T22:45:16.989788+0000","last_active":"2024-09-10T22:45:58.680612+0000","last_peered":"2024-09-10T22:45:58.680612+0000","last_clean":"2024-09-10T22:45:58.680612+0000","last_became_active":"2024-09-10T22:45:16.989431+0000","last_became_peered":"2024-09-10T22:45:16.989431+0000","last_unstale":"2024-09-10T22:45:58.680612+0000","last_undegraded":"2024-09-10T22:45:58.680612+0000","last_fullsized":"2024-09-10T22:45:58.680612+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:55.257590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:46:57.538889+0000","last_change":"2024-09-10T22:45:18.942798+0000","last_active":"2024-09-10T22:46:57.538889+0000","last_peered":"2024-09-10T22:46:57.538889+0000","last_clean":"2024-09-10T22:46:57.538889+0000","last_became_active":"2024-09-10T22:45:18.942485+0000","last_became_peered":"2024-09-10T22:45:18.942485+0000","last_unstale":"2024-09-10T22:46:57.538889+0000","last_undegraded":"2024-09-10T22:46:57.538889+0000","last_fullsized":"2024-09-10T22:46:57.538889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:17.920442+0000","last_clean_scrub_stamp":"2024-09-10T22:45:17.920442+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:33.854322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T22:47:11.195331+0000","last_change":"2024-09-10T22:45:16.934790+0000","last_active":"2024-09-10T22:47:11.195331+0000","last_peered":"2024-09-10T22:47:11.195331+0000","last_clean":"2024-09-10T22:47:11.195331+0000","last_became_active":"2024-09-10T22:45:16.934470+0000","last_became_peered":"2024-09-10T22:45:16.934470+0000","last_unstale":"2024-09-10T22:47:11.195331+0000","last_undegraded":"2024-09-10T22:47:11.195331+0000","last_fullsized":"2024-09-10T22:47:11.195331+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:45:15.912254+0000","last_clean_scrub_stamp":"2024-09-10T22:45:15.912254+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:39.980817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67049,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1071,"num_read_kb":610,"num_write":251,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247584,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":405978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1057326,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":405908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":406286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495673,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":999503,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":998091,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":405486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":405450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":466769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":467165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":406964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34212,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35832,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:47:18.722 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:47:18.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:18 smithi121 ceph-mon[35119]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:18.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:18 smithi117 ceph-mon[35081]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.004 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 -- 172.21.15.96:0/122876552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c108060 msgr2=0x7f7d6c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 --2- 172.21.15.96:0/122876552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c108060 0x7f7d6c10e860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009ca0 tx=0x7f7d6402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 -- 172.21.15.96:0/122876552 shutdown_connections 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 --2- 172.21.15.96:0/122876552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 --2- 172.21.15.96:0/122876552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c108060 0x7f7d6c10e860 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 --2- 172.21.15.96:0/122876552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 0x7f7d6c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 -- 172.21.15.96:0/122876552 >> 172.21.15.96:0/122876552 conn(0x7f7d6c0fd700 msgr2=0x7f7d6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 -- 172.21.15.96:0/122876552 shutdown_connections 2024-09-10T22:47:19.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.473+0000 7f7d7350d640 1 -- 172.21.15.96:0/122876552 wait complete. 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 Processor -- start 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 -- start start 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 0x7f7d6c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c108060 0x7f7d6c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c112e30 con 0x7f7d6c1076e0 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c112cb0 con 0x7f7d6c108060 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d7350d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c112fb0 con 0x7f7d6c101cd0 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34660/0 (socket says 172.21.15.96:34660) 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 -- 172.21.15.96:0/328487928 learned_addr learned my addr 172.21.15.96:0/328487928 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:19.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d712a9640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 0x7f7d6c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d71aaa640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c108060 0x7f7d6c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 -- 172.21.15.96:0/328487928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 msgr2=0x7f7d6c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 0x7f7d6c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 -- 172.21.15.96:0/328487928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c108060 msgr2=0x7f7d6c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c108060 0x7f7d6c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.474+0000 7f7d70aa8640 1 -- 172.21.15.96:0/328487928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c19f360 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d70aa8640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00d950 tx=0x7f7d5c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c017070 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c00de80 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00f620 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c19f650 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.475+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1a5ee0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.476+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c1020b0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.477+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f7d5c005030 con 0x7f7d6c1076e0 2024-09-10T22:47:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.478+0000 7f7d627fc640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 0x7f7d580b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.478+0000 7f7d712a9640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 0x7f7d580b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.478+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7d5c0e58c0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.479+0000 7f7d712a9640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 0x7f7d580b78b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009990 tx=0x7f7d68009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.481+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d5c0ac6e0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.694+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6c19a3e0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.696 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:19 smithi096 ceph-mon[30869]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.696 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:19 smithi096 ceph-mon[30869]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.696 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:19 smithi096 ceph-mon[30869]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-10T22:47:19.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.694+0000 7f7d627fc640 1 -- 172.21.15.96:0/328487928 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11979 (secure 0 0 0) 0x7f7d6c19a3e0 con 0x7f7d6c1076e0 2024-09-10T22:47:19.697 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:19.697 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":28,"btime":"2024-09-10T22:47:14:394570+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:47:13.445561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:47:19.697 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 28 2024-09-10T22:47:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 msgr2=0x7f7d580b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 0x7f7d580b78b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009990 tx=0x7f7d68009110 comp rx=0 tx=0).stop 2024-09-10T22:47:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 msgr2=0x7f7d6c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00d950 tx=0x7f7d5c00dd50 comp rx=0 tx=0).stop 2024-09-10T22:47:19.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 shutdown_connections 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d6c108060 0x7f7d6c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7d580b53f0 0x7f7d580b78b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6c1076e0 0x7f7d6c1997f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 --2- 172.21.15.96:0/328487928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c101cd0 0x7f7d6c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 >> 172.21.15.96:0/328487928 conn(0x7f7d6c0fd700 msgr2=0x7f7d6c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 shutdown_connections 2024-09-10T22:47:19.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:19.697+0000 7f7d7350d640 1 -- 172.21.15.96:0/328487928 wait complete. 2024-09-10T22:47:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:19 smithi121 ceph-mon[35119]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:19 smithi121 ceph-mon[35119]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:19 smithi121 ceph-mon[35119]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-10T22:47:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:19 smithi117 ceph-mon[35081]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:19 smithi117 ceph-mon[35081]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:47:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:19 smithi117 ceph-mon[35081]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-10T22:47:20.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:47:20.658 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:20.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/328487928' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:20.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:20 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/328487928' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:20.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:20 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/328487928' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:47:21.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 -- 172.21.15.96:0/2719381338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8109150 msgr2=0x7f62f810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:21.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/2719381338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8109150 0x7f62f810f950 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009ca0 tx=0x7f62f002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 -- 172.21.15.96:0/2719381338 shutdown_connections 2024-09-10T22:47:21.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/2719381338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 0x7f62f8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/2719381338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8109150 0x7f62f810f950 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/2719381338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8102dc0 0x7f62f81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.105+0000 7f62fcd8d640 1 -- 172.21.15.96:0/2719381338 >> 172.21.15.96:0/2719381338 conn(0x7f62f80fe830 msgr2=0x7f62f8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- 172.21.15.96:0/2719381338 shutdown_connections 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- 172.21.15.96:0/2719381338 wait complete. 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 Processor -- start 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- start start 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 0x7f62f8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8109150 0x7f62f819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112e10 con 0x7f62f8102dc0 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112c90 con 0x7f62f81087d0 2024-09-10T22:47:21.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.106+0000 7f62fcd8d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112f90 con 0x7f62f8109150 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34680/0 (socket says 172.21.15.96:34680) 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 -- 172.21.15.96:0/1153238717 learned_addr learned my addr 172.21.15.96:0/1153238717 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f6d9d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8109150 0x7f62f819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f5d9b640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 0x7f62f8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 -- 172.21.15.96:0/1153238717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8109150 msgr2=0x7f62f819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8109150 0x7f62f819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 -- 172.21.15.96:0/1153238717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 msgr2=0x7f62f8199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 0x7f62f8199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 -- 172.21.15.96:0/1153238717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f819f270 con 0x7f62f8102dc0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62f659c640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62e800d950 tx=0x7f62e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8017070 con 0x7f62f8102dc0 2024-09-10T22:47:21.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f819f560 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e8009c30 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e800f620 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.107+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a5df0 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.108+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f81031a0 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.110+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f62e8010460 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.110+0000 7f62e77fe640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 0x7f62e00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.110+0000 7f62f5d9b640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 0x7f62e00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.110+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f62e80e58a0 con 0x7f62f8102dc0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.111+0000 7f62f5d9b640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 0x7f62e00b7830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005e10 tx=0x7f62ec005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:21.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.113+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f62e80ac700 con 0x7f62f8102dc0 2024-09-10T22:47:21.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.290+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62f81a5fc0 con 0x7f62f8102dc0 2024-09-10T22:47:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.291+0000 7f62e77fe640 1 -- 172.21.15.96:0/1153238717 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7f62e8009de0 con 0x7f62f8102dc0 2024-09-10T22:47:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:21.293 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:47:21.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 msgr2=0x7f62e00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 0x7f62e00b7830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005e10 tx=0x7f62ec005da0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 msgr2=0x7f62f81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62e800d950 tx=0x7f62e800de10 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 shutdown_connections 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62f81087d0 0x7f62f8199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f62e00b5370 0x7f62e00b7830 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f8109150 0x7f62f819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 --2- 172.21.15.96:0/1153238717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62f8102dc0 0x7f62f81991c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 >> 172.21.15.96:0/1153238717 conn(0x7f62f80fe830 msgr2=0x7f62f8101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 shutdown_connections 2024-09-10T22:47:21.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:21.294+0000 7f62fcd8d640 1 -- 172.21.15.96:0/1153238717 wait complete. 2024-09-10T22:47:21.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:21 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1153238717' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:21.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:21 smithi096 ceph-mon[30869]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-10T22:47:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:21 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1153238717' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:21 smithi121 ceph-mon[35119]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-10T22:47:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:21 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1153238717' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:47:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:21 smithi117 ceph-mon[35081]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-10T22:47:22.133 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T22:47:22.133 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T22:47:22.145 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:47:22.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T22:47:22.146 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T22:47:22.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:47:22.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:47:22.148 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:47:22.187 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:47:22.187 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T22:47:22.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-10T22:47:22.223 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:22.238 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-10T22:47:22.238 INFO:teuthology.orchestra.run.smithi096.stdout:0@14afb286-6fc5-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/83316ff1-820e-4d07-b614-9a4f7ce8988b 340G 0 340G 0% /tmp/cephfs 2024-09-10T22:47:22.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-10T22:47:22.333 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-10T22:47:22.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-10T22:47:22.407 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- 172.21.15.96:0/2106031211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070109170 msgr2=0x7fe07010f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 --2- 172.21.15.96:0/2106031211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070109170 0x7fe07010f970 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe068009ca0 tx=0x7fe06802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- 172.21.15.96:0/2106031211 shutdown_connections 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 --2- 172.21.15.96:0/2106031211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070109170 0x7fe07010f970 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 --2- 172.21.15.96:0/2106031211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0701087f0 0x7fe070108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 --2- 172.21.15.96:0/2106031211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe070102de0 0x7fe0701031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- 172.21.15.96:0/2106031211 >> 172.21.15.96:0/2106031211 conn(0x7fe0700fe830 msgr2=0x7fe070100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- 172.21.15.96:0/2106031211 shutdown_connections 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- 172.21.15.96:0/2106031211 wait complete. 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 Processor -- start 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.944+0000 7fe07625e640 1 -- start start 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0701087f0 0x7fe0701997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070109170 0x7fe07019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe070112e30 con 0x7fe070102de0 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe070112cb0 con 0x7fe0701087f0 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe070112fb0 con 0x7fe070109170 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34704/0 (socket says 172.21.15.96:34704) 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 -- 172.21.15.96:0/968974165 learned_addr learned my addr 172.21.15.96:0/968974165 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe066ffd640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0701087f0 0x7fe0701997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06ffff640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070109170 0x7fe07019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 -- 172.21.15.96:0/968974165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070109170 msgr2=0x7fe07019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070109170 0x7fe07019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 -- 172.21.15.96:0/968974165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0701087f0 msgr2=0x7fe0701997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0701087f0 0x7fe0701997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 -- 172.21.15.96:0/968974165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07019f350 con 0x7fe070102de0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe06000b4a0 tx=0x7fe06000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06d7fa640 1 -- 172.21.15.96:0/968974165 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060014070 con 0x7fe070102de0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07019f640 con 0x7fe070102de0 2024-09-10T22:47:22.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06d7fa640 1 -- 172.21.15.96:0/968974165 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060010ac0 con 0x7fe070102de0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.945+0000 7fe06d7fa640 1 -- 172.21.15.96:0/968974165 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060017c30 con 0x7fe070102de0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.946+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe070075400 con 0x7fe070102de0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.948+0000 7fe078633740 1 Processor -- start 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 -- start start 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06d7fa640 1 -- 172.21.15.96:0/968974165 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fe060010610 con 0x7fe070102de0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558fc9b31e60 con 0x558fc9bbaee0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558fc9b21290 con 0x558fc9bbb800 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe078633740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558fc9b250a0 con 0x558fc9bc7790 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34714/0 (socket says 172.21.15.96:34714) 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 -- 172.21.15.96:0/3364053379 learned_addr learned my addr 172.21.15.96:0/3364053379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 -- 172.21.15.96:0/3364053379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 msgr2=0x558fc9bc9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 -- 172.21.15.96:0/3364053379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 msgr2=0x558fc9bc7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 -- 172.21.15.96:0/3364053379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc987f930 con 0x558fc9bbaee0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.949+0000 7fe06ffff640 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe06802f4f0 tx=0x7fe068031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c7b70206623edb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe065ffb640 1 -- 172.21.15.96:0/3364053379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068031650 con 0x558fc9bbaee0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe065ffb640 1 -- 172.21.15.96:0/3364053379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe068036070 con 0x558fc9bbaee0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe065ffb640 1 -- 172.21.15.96:0/3364053379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068004780 con 0x558fc9bbaee0 2024-09-10T22:47:22.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe06d7fa640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 0x7fe05c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 0x7fe05c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe06d7fa640 1 -- 172.21.15.96:0/968974165 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0600e54a0 con 0x7fe070102de0 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe078633740 1 -- 172.21.15.96:0/3364053379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 msgr2=0x558fc9bbb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe06f7fe640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 0x7fe05c0b7830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x558fc9bbe060 tx=0x7fe0600046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.950+0000 7fe078633740 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe06802f4f0 tx=0x7fe068031af0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- 172.21.15.96:0/3364053379 shutdown_connections 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- 172.21.15.96:0/3364053379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bbb2c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- 172.21.15.96:0/3364053379 >> 172.21.15.96:0/3364053379 conn(0x558fc9bb8d10 msgr2=0x558fc9bb9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- 172.21.15.96:0/3364053379 shutdown_connections 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- 172.21.15.96:0/3364053379 wait complete. 2024-09-10T22:47:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- start start 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 Processor -- start 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558fc9982890 con 0x558fc9bbaee0 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558fc9abbd40 con 0x558fc9bbb800 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558fc9bca260 con 0x558fc9bc7790 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.951+0000 7fe078633740 10 client.0 _ll_register_callbacks cb 0x558fc9b1fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34720/0 (socket says 172.21.15.96:34720) 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 -- 172.21.15.96:0/2118504208 learned_addr learned my addr 172.21.15.96:0/2118504208 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 -- 172.21.15.96:0/2118504208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 msgr2=0x558fc9bc5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 -- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 msgr2=0x558fc9bc4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 -- 172.21.15.96:0/2118504208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc9bd3610 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe068004220 tx=0x7fe0680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cfa8004902edf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe04effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06804e040 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe068003d80 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe068036070 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe068031870 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe078633740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.952+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fc9bd5fc0 con 0x558fc9bbaee0 2024-09-10T22:47:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe068036070 con 0x558fc9bbaee0 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe068043020 con 0x558fc9bbaee0 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 20 client.15390 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9bd9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 4 client.15390 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc9b33db0 con 0x558fc9bd7310 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.953+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9bd9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe04effd640 10 client.15390 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9bd9700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7325f93fb7878adc server_cookie=1413c878ed4e3439 in_seq=0 out_seq=0 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe04effd640 1 -- 172.21.15.96:0/2118504208 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558fc9b33db0 con 0x558fc9bd7310 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe04effd640 10 client.15390 handle_command_reply: tid=0 2024-09-10T22:47:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 client.15390 shutdown 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 msgr2=0x558fc9bc4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe068004220 tx=0x7fe0680095c0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 shutdown_connections 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9bd9700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe04effd640 0 client.15390 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbb800 0x558fc9bc4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 --2- 172.21.15.96:0/2118504208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbaee0 0x558fc9bc4700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.954+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 >> 172.21.15.96:0/2118504208 conn(0x558fc9bb8d10 msgr2=0x558fc9bb7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.955+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 shutdown_connections 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.955+0000 7fe078633740 1 -- 172.21.15.96:0/2118504208 wait complete. 2024-09-10T22:47:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.955+0000 7fe078633740 20 client.15390 trim_cache size 0 max 16384 2024-09-10T22:47:22.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.972+0000 7fe078633740 1 Processor -- start 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.972+0000 7fe078633740 1 -- start start 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bbe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b0490 con 0x558fc9bbb800 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b1fd0 con 0x558fc9bbaee0 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe078633740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b0b60 con 0x558fc9bc7790 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34724/0 (socket says 172.21.15.96:34724) 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 -- 172.21.15.96:0/742417629 learned_addr learned my addr 172.21.15.96:0/742417629 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe06ffff640 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 -- 172.21.15.96:0/742417629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 msgr2=0x558fc9bc1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 -- 172.21.15.96:0/742417629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 msgr2=0x558fc9bbe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bbe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 -- 172.21.15.96:0/742417629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc9bb9790 con 0x558fc9bbb800 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe06ffff640 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe066ffd640 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe05800aa50 tx=0x7fe058012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f318b4d9e9b3a082 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.973+0000 7fe0657fa640 1 -- 172.21.15.96:0/742417629 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05801f040 con 0x558fc9bbb800 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe0657fa640 1 -- 172.21.15.96:0/742417629 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580035f0 con 0x558fc9bbb800 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe0657fa640 1 -- 172.21.15.96:0/742417629 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05801d650 con 0x558fc9bbb800 2024-09-10T22:47:22.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 -- 172.21.15.96:0/742417629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 msgr2=0x558fc9bc1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe05800aa50 tx=0x7fe058012820 comp rx=0 tx=0).stop 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 -- 172.21.15.96:0/742417629 shutdown_connections 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bc7790 0x558fc9bc1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bbb800 0x558fc9bc1930 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 --2- 172.21.15.96:0/742417629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bbe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 -- 172.21.15.96:0/742417629 >> 172.21.15.96:0/742417629 conn(0x558fc9bb8d10 msgr2=0x558fc9bbe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 -- 172.21.15.96:0/742417629 shutdown_connections 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.974+0000 7fe078633740 1 -- 172.21.15.96:0/742417629 wait complete. 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 -- start start 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 Processor -- start 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bcfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bbb800 0x558fc9bcc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b1900 con 0x558fc9bc7790 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701b1230 con 0x558fc9bbaee0 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ae470 con 0x558fc9bbb800 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe078633740 10 client.0 _ll_register_callbacks cb 0x7fe0680099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34738/0 (socket says 172.21.15.96:34738) 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.975+0000 7fe06ffff640 1 -- 172.21.15.96:0/2593654357 learned_addr learned my addr 172.21.15.96:0/2593654357 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bbb800 0x558fc9bcc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bcfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 -- 172.21.15.96:0/2593654357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bbb800 msgr2=0x558fc9bcc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bbb800 0x558fc9bcc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 -- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 msgr2=0x558fc9bcfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bcfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 -- 172.21.15.96:0/2593654357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc9bccff0 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe06ffff640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe068000f80 tx=0x7fe068031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91deb08f1eba9781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe04effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068004840 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580035f0 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe068030c90 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe068062050 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe078633740 10 client.15396 fetch_fsmap learned FSMap version 28 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.976+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fc9c44d70 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580035f0 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe068043040 con 0x558fc9bc7790 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:22.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 20 client.15396 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9c47530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 4 client.15396 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558fc9b2f860 con 0x558fc9bd7310 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.977+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9c47530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.978+0000 7fe066ffd640 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9c47530 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68285d32c35d7b2e server_cookie=7d7cb6ba17900a60 in_seq=0 out_seq=0 2024-09-10T22:47:22.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.978+0000 7fe04effd640 10 client.15396 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.979+0000 7fe04effd640 1 -- 172.21.15.96:0/2593654357 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fc9b2f860 con 0x558fc9bd7310 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.979+0000 7fe04effd640 10 client.15396 handle_command_reply: tid=0 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.979+0000 7fe078633740 1 client.15396 shutdown 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 msgr2=0x558fc9bcca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe068000f80 tx=0x7fe068031450 comp rx=0 tx=0).stop 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 shutdown_connections 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558fc9bd7310 0x558fc9c47530 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558fc9bc7790 0x558fc9bcca30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558fc9bbb800 0x558fc9bcc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe04effd640 0 client.15396 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 --2- 172.21.15.96:0/2593654357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558fc9bbaee0 0x558fc9bcfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 >> 172.21.15.96:0/2593654357 conn(0x558fc9bb8d10 msgr2=0x558fc9bd2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 shutdown_connections 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 1 -- 172.21.15.96:0/2593654357 wait complete. 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.980+0000 7fe078633740 20 client.15396 trim_cache size 0 max 16384 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:47:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "51945418-7691-40a2-845b-cad5ed759ad6", 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 msgr2=0x7fe05c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 0x7fe05c0b7830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x558fc9bbe060 tx=0x7fe0600046c0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 msgr2=0x7fe0701992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe06000b4a0 tx=0x7fe06000b960 comp rx=0 tx=0).stop 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 shutdown_connections 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0701087f0 0x7fe0701997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe05c0b5370 0x7fe05c0b7830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070109170 0x7fe07019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 --2- 172.21.15.96:0/968974165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe070102de0 0x7fe0701992a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 >> 172.21.15.96:0/968974165 conn(0x7fe0700fe830 msgr2=0x7fe070100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 shutdown_connections 2024-09-10T22:47:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:22.981+0000 7fe07625e640 1 -- 172.21.15.96:0/968974165 wait complete. 2024-09-10T22:47:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:23 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:47:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:23 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:23 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:47:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:23 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:23 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:47:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:23 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:23 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:47:23.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:23 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:23 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:47:23.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:23 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:23.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:23 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:47:23.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:23 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:47:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:24 smithi096 ceph-mon[30869]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:47:24.637 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-10T22:47:24.642 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:47:24.674 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-10T22:47:24.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-10T22:47:24.754 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T22:47:24.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:24 smithi121 ceph-mon[35119]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:47:24.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:24 smithi117 ceph-mon[35081]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/: 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 data 2024-09-10T22:47:24.831 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:24.832 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data: 2024-09-10T22:47:24.832 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:24.832 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:24.832 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T22:47:24.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-10T22:47:24.923 INFO:teuthology.orchestra.run.smithi096.stdout:[Service] 2024-09-10T22:47:24.923 INFO:teuthology.orchestra.run.smithi096.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-10T22:47:24.923 INFO:teuthology.orchestra.run.smithi096.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-10T22:47:24.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-10T22:47:24.935 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:25.015 INFO:teuthology.orchestra.run.smithi096.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-10T22:47:25.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/3303282494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a8103ca0 msgr2=0x7fe3a8104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/3303282494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8104080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe39802a410 comp rx=0 tx=0).stop 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/3303282494 shutdown_connections 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/3303282494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a810d100 0x7fe3a810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/3303282494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a81045c0 0x7fe3a810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/3303282494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8104080 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.577+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/3303282494 >> 172.21.15.96:0/3303282494 conn(0x7fe3a80fd700 msgr2=0x7fe3a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.578+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/3303282494 shutdown_connections 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.578+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/3303282494 wait complete. 2024-09-10T22:47:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.578+0000 7fe3ad7d5640 1 Processor -- start 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 -- start start 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8112d70 con 0x7fe3a810d100 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8112bf0 con 0x7fe3a81045c0 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3ad7d5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8112ef0 con 0x7fe3a8103ca0 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34760/0 (socket says 172.21.15.96:34760) 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60716/0 (socket says 172.21.15.96:60716) 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 -- 172.21.15.96:0/1265493871 learned_addr learned my addr 172.21.15.96:0/1265493871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 -- 172.21.15.96:0/1265493871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a8103ca0 msgr2=0x7fe3a8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 -- 172.21.15.96:0/1265493871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 msgr2=0x7fe3a81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a77fe640 1 -- 172.21.15.96:0/1265493871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a819f330 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.579+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000eaa0 tx=0x7fe3a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe38ffff640 1 -- 172.21.15.96:0/1265493871 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000cdb0 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe38ffff640 1 -- 172.21.15.96:0/1265493871 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a0005020 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe38ffff640 1 -- 172.21.15.96:0/1265493871 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00116d0 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a819f620 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.580+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a81a5ce0 con 0x7fe3a810d100 2024-09-10T22:47:25.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.582+0000 7fe3afbaa740 1 Processor -- start 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe38ffff640 1 -- 172.21.15.96:0/1265493871 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe3a810d100 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 -- start start 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 0x55c4994b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c499427e50 con 0x55c4994b1da0 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c499417280 con 0x55c4994b1480 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c49941b090 con 0x55c4994c2140 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 0x55c4994b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994c1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34774/0 (socket says 172.21.15.96:34774) 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/3016264476 learned_addr learned my addr 172.21.15.96:0/3016264476 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:25.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3016264476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 msgr2=0x55c4994c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3016264476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 msgr2=0x55c4994c1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.583+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994c1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3016264476 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c499175650 con 0x55c4994b1480 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe38ffff640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 0x7fe3900b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 0x55c4994b1860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3a810f7d0 tx=0x7fe39c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711e5be33bcebd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe38d7fa640 1 -- 172.21.15.96:0/3016264476 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c012250 con 0x55c4994b1480 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe38ffff640 1 -- 172.21.15.96:0/1265493871 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe3a00e5b10 con 0x7fe3a810d100 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 0x7fe3900b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe38d7fa640 1 -- 172.21.15.96:0/3016264476 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c012e50 con 0x55c4994b1480 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.584+0000 7fe38d7fa640 1 -- 172.21.15.96:0/3016264476 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c010ea0 con 0x55c4994b1480 2024-09-10T22:47:25.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3016264476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 msgr2=0x55c4994b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 0x55c4994b1860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3a810f7d0 tx=0x7fe39c005c90 comp rx=0 tx=0).stop 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 0x7fe3900b7830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe398005c40 tx=0x7fe398005e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3016264476 shutdown_connections 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994c1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3016264476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1480 0x55c4994b1860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3016264476 >> 172.21.15.96:0/3016264476 conn(0x55c4994af2b0 msgr2=0x55c4994b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3016264476 shutdown_connections 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3016264476 wait complete. 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 -- start start 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.585+0000 7fe3afbaa740 1 Processor -- start 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 0x55c4994beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c499278700 con 0x55c4994b1480 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4993b2110 con 0x55c4994b1da0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c4994c4e40 con 0x55c4994c2140 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3afbaa740 10 client.0 _ll_register_callbacks cb 0x55c4993cb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34782/0 (socket says 172.21.15.96:34782) 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/3511003825 learned_addr learned my addr 172.21.15.96:0/3511003825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 0x55c4994beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3511003825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 msgr2=0x55c4994bf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994bf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3511003825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 msgr2=0x55c4994c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3511003825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4993dfc40 con 0x55c4994b1da0 2024-09-10T22:47:25.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 0x55c4994beea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe39c005d00 tx=0x7fe39c010af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0ad26e44d0d4dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.586+0000 7fe38aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c004250 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39c012e50 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe39c0043f0 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe39c005030 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 10 client.24986 fetch_fsmap learned FSMap version 28 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4994d0800 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c012e50 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe39c031050 con 0x55c4994b1da0 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 10 client.24986 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 10 client.24986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 20 client.24986 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:25.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c4994d1bb0 0x55c4994d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 4 client.24986 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c499429da0 con 0x55c4994d1bb0 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.587+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c4994d1bb0 0x55c4994d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.588+0000 7fe38aff5640 10 client.24986 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.588+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c4994d1bb0 0x55c4994d3fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d153d712695eb54 server_cookie=96a7d04b6c1bc1c in_seq=0 out_seq=0 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.588+0000 7fe38aff5640 1 -- 172.21.15.96:0/3511003825 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c499429da0 con 0x55c4994d1bb0 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.588+0000 7fe38aff5640 10 client.24986 handle_command_reply: tid=0 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.588+0000 7fe3afbaa740 1 client.24986 shutdown 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 msgr2=0x55c4994beea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 0x55c4994beea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe39c005d00 tx=0x7fe39c010af0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 shutdown_connections 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c4994d1bb0 0x55c4994d3fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994b1da0 0x55c4994beea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994c2140 0x55c4994bf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3511003825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe38aff5640 0 client.24986 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 >> 172.21.15.96:0/3511003825 conn(0x55c4994af2b0 msgr2=0x55c4994ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 shutdown_connections 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3511003825 wait complete. 2024-09-10T22:47:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.589+0000 7fe3afbaa740 20 client.24986 trim_cache size 0 max 16384 2024-09-10T22:47:25.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:25 smithi096 ceph-mon[30869]: pgmap v351: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.607+0000 7fe3afbaa740 1 Processor -- start 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.607+0000 7fe3afbaa740 1 -- start start 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1da0 0x55c4994ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994d9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81ac090 con 0x55c4994b1480 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81adbd0 con 0x55c4994c2140 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81ac760 con 0x55c4994b1da0 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34796/0 (socket says 172.21.15.96:34796) 2024-09-10T22:47:25.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/2898791961 learned_addr learned my addr 172.21.15.96:0/2898791961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994d9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1da0 0x55c4994ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/2898791961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1da0 msgr2=0x55c4994ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1da0 0x55c4994ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/2898791961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 msgr2=0x55c4994d9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994d9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a6ffd640 1 -- 172.21.15.96:0/2898791961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4994bd3c0 con 0x55c4994b1480 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.608+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994d9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe39803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24321fc61cc029a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe38cff9640 1 -- 172.21.15.96:0/2898791961 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398035040 con 0x55c4994b1480 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe38cff9640 1 -- 172.21.15.96:0/2898791961 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe398055070 con 0x55c4994b1480 2024-09-10T22:47:25.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe38cff9640 1 -- 172.21.15.96:0/2898791961 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39805a840 con 0x55c4994b1480 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe3afbaa740 1 -- 172.21.15.96:0/2898791961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 msgr2=0x55c4994aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.609+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe398009900 tx=0x7fe39803e000 comp rx=0 tx=0).stop 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 -- 172.21.15.96:0/2898791961 shutdown_connections 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994d9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1da0 0x55c4994ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/2898791961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1480 0x55c4994aebc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 -- 172.21.15.96:0/2898791961 >> 172.21.15.96:0/2898791961 conn(0x55c4994af2b0 msgr2=0x55c4994c04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 -- 172.21.15.96:0/2898791961 shutdown_connections 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 -- 172.21.15.96:0/2898791961 wait complete. 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 -- start start 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.610+0000 7fe3afbaa740 1 Processor -- start 2024-09-10T22:47:25.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1480 0x55c4994ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81ad500 con 0x55c4994b1da0 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81ace30 con 0x55c4994c2140 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81aa0b0 con 0x55c4994b1480 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3afbaa740 10 client.0 _ll_register_callbacks cb 0x7fe398009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34804/0 (socket says 172.21.15.96:34804) 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3505247392 learned_addr learned my addr 172.21.15.96:0/3505247392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:25.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a77fe640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1480 0x55c4994ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3505247392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1480 msgr2=0x55c4994ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1480 0x55c4994ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 msgr2=0x55c4994c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 -- 172.21.15.96:0/3505247392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4994cad60 con 0x55c4994b1da0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe3a67fc640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe39c012810 tx=0x7fe39c0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71dd94794d4245f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe38aff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.611+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ba30 con 0x55c4994b1da0 2024-09-10T22:47:25.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe39c025040 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe398055070 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe39c004250 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c49953edd0 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c037020 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe39c03c030 con 0x55c4994b1da0 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 20 client.15414 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c49953f4f0 0x55c499541910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 4 client.15414 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:47:25.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.612+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe398055070 con 0x55c49953f4f0 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.613+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c49953f4f0 0x55c499541910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.613+0000 7fe38aff5640 10 client.15414 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.613+0000 7fe3a6ffd640 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c49953f4f0 0x55c499541910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6ef8476122b6c3 server_cookie=4cf3b5f3f48fca10 in_seq=0 out_seq=0 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe38aff5640 1 -- 172.21.15.96:0/3505247392 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x7fe398055070 con 0x55c49953f4f0 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe38aff5640 10 client.15414 handle_command_reply: tid=0 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 client.15414 shutdown 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 msgr2=0x55c4994ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe39c012810 tx=0x7fe39c0059b0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 shutdown_connections 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c49953f4f0 0x55c499541910 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4994c2140 0x55c4994c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4994b1da0 0x55c4994ca670 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 --2- 172.21.15.96:0/3505247392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4994b1480 0x55c4994ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 >> 172.21.15.96:0/3505247392 conn(0x55c4994af2b0 msgr2=0x55c4994affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe38aff5640 0 client.15414 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 shutdown_connections 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 1 -- 172.21.15.96:0/3505247392 wait complete. 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.614+0000 7fe3afbaa740 20 client.15414 trim_cache size 0 max 16384 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:47:25.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "51945418-7691-40a2-845b-cad5ed759ad6", 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:47:25.635 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 msgr2=0x7fe3900b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 0x7fe3900b7830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe398005c40 tx=0x7fe398005e10 comp rx=0 tx=0).stop 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 msgr2=0x7fe3a819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000eaa0 tx=0x7fe3a000ef60 comp rx=0 tx=0).stop 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 shutdown_connections 2024-09-10T22:47:25.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3a81045c0 0x7fe3a81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe3900b5370 0x7fe3900b7830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3a810d100 0x7fe3a819eb60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 --2- 172.21.15.96:0/1265493871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3a8103ca0 0x7fe3a8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 >> 172.21.15.96:0/1265493871 conn(0x7fe3a80fd700 msgr2=0x7fe3a80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 shutdown_connections 2024-09-10T22:47:25.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:25.615+0000 7fe3ad7d5640 1 -- 172.21.15.96:0/1265493871 wait complete. 2024-09-10T22:47:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:25 smithi121 ceph-mon[35119]: pgmap v351: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-10T22:47:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:25 smithi117 ceph-mon[35081]: pgmap v351: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-10T22:47:26.409 INFO:tasks.cephfs.filesystem:scrub status for tag:51945418-7691-40a2-845b-cad5ed759ad6 - {'path': '/', 'tag': '51945418-7691-40a2-845b-cad5ed759ad6', 'options': 'recursive,force'} 2024-09-10T22:47:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:27 smithi096 ceph-mon[30869]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T22:47:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:27 smithi121 ceph-mon[35119]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T22:47:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:27 smithi117 ceph-mon[35081]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T22:47:28.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:28 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:28.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:28 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:28 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:47:29.890 INFO:teuthology.orchestra.run.smithi096.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-10T22:47:29.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T22:47:29.936 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/: 2024-09-10T22:47:29.936 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 data 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data: 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:total 35 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 . 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 base 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 10 22:47 global 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 log 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_commit_ts 2024-09-10T22:47:29.937 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_dynshmem 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 10 22:47 pg_hba.conf 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 22:47 pg_ident.conf 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 pg_logical 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 pg_multixact 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_notify 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_replslot 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_serial 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_snapshots 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_stat 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_stat_tmp 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 pg_subtrans 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_tblspc 2024-09-10T22:47:29.938 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_twophase 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 pg_wal 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 pg_xact 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 22:47 postgresql.auto.conf 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 10 22:47 postgresql.conf 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base: 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 . 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 1 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 13493 2024-09-10T22:47:29.939 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 13494 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:total 7777 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 . 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 .. 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T22:47:29.940 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 22:47 1259 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T22:47:29.941 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T22:47:29.942 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T22:47:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T22:47:29.944 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T22:47:29.945 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T22:47:29.946 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 22:47 2619 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2619_fsm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T22:47:29.947 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T22:47:29.948 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T22:47:29.949 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2696 2024-09-10T22:47:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T22:47:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T22:47:29.952 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T22:47:29.953 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T22:47:29.954 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T22:47:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T22:47:29.956 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T22:47:29.957 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T22:47:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T22:47:29.959 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T22:47:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T22:47:29.963 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T22:47:29.963 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T22:47:29.963 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T22:47:29.964 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:total 7777 2024-09-10T22:47:29.965 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 . 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 .. 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 22:47 1259 2024-09-10T22:47:29.966 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T22:47:29.967 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T22:47:29.968 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T22:47:29.969 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T22:47:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T22:47:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 22:47 2619 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2619_fsm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T22:47:29.972 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T22:47:29.973 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T22:47:29.974 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2696 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T22:47:29.975 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T22:47:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T22:47:29.977 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T22:47:29.978 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T22:47:29.979 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T22:47:29.980 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T22:47:29.981 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T22:47:29.982 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T22:47:29.983 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T22:47:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T22:47:29.985 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:total 7777 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 . 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 .. 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 22:47 1259 2024-09-10T22:47:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T22:47:29.988 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T22:47:29.989 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T22:47:29.990 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T22:47:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T22:47:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 22:47 2619 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2619_fsm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T22:47:29.993 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T22:47:29.994 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T22:47:29.995 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T22:47:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2696 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T22:47:29.997 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T22:47:29.998 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T22:47:29.999 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T22:47:30.000 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T22:47:30.001 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T22:47:30.002 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T22:47:30.003 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T22:47:30.004 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T22:47:30.005 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T22:47:30.006 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T22:47:30.007 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/global: 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:total 537 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 10 22:47 . 2024-09-10T22:47:30.008 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1213 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1213_fsm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1213_vm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1214 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1214_fsm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1214_vm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 1232 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 1233 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1260 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1260_fsm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1260_vm 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1261 2024-09-10T22:47:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1261_fsm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1261_vm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1262 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1262_fsm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1262_vm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2396 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2396_fsm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2396_vm 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2397 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2671 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2672 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2676 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2677 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2694 2024-09-10T22:47:30.010 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2695 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2697 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2698 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2846 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2847 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2964 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2965 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2966 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2967 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3592 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3593 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4060 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4061 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4175 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4176 2024-09-10T22:47:30.011 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4177 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4178 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4181 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4182 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4183 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4184 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4185 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4186 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6000 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6001 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6002 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6100 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6114 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6115 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 pg_control 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T22:47:30.012 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/log: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.013 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:total 1 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 . 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 mappings 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 22:47 replorigin_checkpoint 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 snapshots 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 .. 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-10T22:47:30.014 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 .. 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 . 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 members 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 offsets 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:30.015 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 .. 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 0000 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 .. 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 0000 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.016 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.017 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 0000 2024-09-10T22:47:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:total 16384 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 . 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.019 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:47 000000010000000000000001 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 archive_status 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 .. 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 10 22:47 .. 2024-09-10T22:47:30.020 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 0000 2024-09-10T22:47:30.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: pgmap v353: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.0 MiB/s wr, 36 op/s 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:30.142 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:30.143 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:dropping old tables... 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-10T22:47:30.187 INFO:teuthology.orchestra.run.smithi096.stderr:creating tables... 2024-09-10T22:47:30.197 INFO:teuthology.orchestra.run.smithi096.stderr:generating data (client-side)... 2024-09-10T22:47:30.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: pgmap v353: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.0 MiB/s wr, 36 op/s 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:30.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.268 INFO:teuthology.orchestra.run.smithi096.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.75 s) 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: pgmap v353: 97 pgs: 97 active+clean; 13 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.0 MiB/s wr, 36 op/s 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:47:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:47:30.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:47:30.413 INFO:teuthology.orchestra.run.smithi096.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.20 s, remaining 3.02 s) 2024-09-10T22:47:30.538 INFO:teuthology.orchestra.run.smithi096.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.33 s, remaining 3.16 s) 2024-09-10T22:47:30.619 INFO:teuthology.orchestra.run.smithi096.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.41 s, remaining 2.85 s) 2024-09-10T22:47:30.737 INFO:teuthology.orchestra.run.smithi096.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.53 s, remaining 2.84 s) 2024-09-10T22:47:30.822 INFO:teuthology.orchestra.run.smithi096.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.61 s, remaining 2.64 s) 2024-09-10T22:47:30.945 INFO:teuthology.orchestra.run.smithi096.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.73 s, remaining 2.62 s) 2024-09-10T22:47:31.102 INFO:teuthology.orchestra.run.smithi096.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.89 s, remaining 2.67 s) 2024-09-10T22:47:31.197 INFO:teuthology.orchestra.run.smithi096.stderr:900000 of 3200000 tuples (28%) done (elapsed 0.99 s, remaining 2.52 s) 2024-09-10T22:47:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.12 s, remaining 2.46 s) 2024-09-10T22:47:31.475 INFO:teuthology.orchestra.run.smithi096.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.26 s, remaining 2.41 s) 2024-09-10T22:47:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:31 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:31 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:31 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:31.544 INFO:teuthology.orchestra.run.smithi096.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.33 s, remaining 2.22 s) 2024-09-10T22:47:31.685 INFO:teuthology.orchestra.run.smithi096.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.47 s, remaining 2.15 s) 2024-09-10T22:47:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.63 s, remaining 2.10 s) 2024-09-10T22:47:31.935 INFO:teuthology.orchestra.run.smithi096.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.72 s, remaining 1.95 s) 2024-09-10T22:47:32.057 INFO:teuthology.orchestra.run.smithi096.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.85 s, remaining 1.85 s) 2024-09-10T22:47:32.153 INFO:teuthology.orchestra.run.smithi096.stderr:1700000 of 3200000 tuples (53%) done (elapsed 1.94 s, remaining 1.71 s) 2024-09-10T22:47:32.292 INFO:teuthology.orchestra.run.smithi096.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.08 s, remaining 1.62 s) 2024-09-10T22:47:32.438 INFO:teuthology.orchestra.run.smithi096.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.23 s, remaining 1.52 s) 2024-09-10T22:47:32.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:32 smithi096 ceph-mon[30869]: pgmap v354: 97 pgs: 97 active+clean; 24 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-09-10T22:47:32.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:32 smithi121 ceph-mon[35119]: pgmap v354: 97 pgs: 97 active+clean; 24 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-09-10T22:47:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:32 smithi117 ceph-mon[35081]: pgmap v354: 97 pgs: 97 active+clean; 24 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.0 MiB/s wr, 52 op/s 2024-09-10T22:47:32.576 INFO:teuthology.orchestra.run.smithi096.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.36 s, remaining 1.42 s) 2024-09-10T22:47:32.654 INFO:teuthology.orchestra.run.smithi096.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.44 s, remaining 1.28 s) 2024-09-10T22:47:32.782 INFO:teuthology.orchestra.run.smithi096.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.57 s, remaining 1.17 s) 2024-09-10T22:47:32.868 INFO:teuthology.orchestra.run.smithi096.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.66 s, remaining 1.04 s) 2024-09-10T22:47:32.991 INFO:teuthology.orchestra.run.smithi096.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.78 s, remaining 0.93 s) 2024-09-10T22:47:33.143 INFO:teuthology.orchestra.run.smithi096.stderr:2500000 of 3200000 tuples (78%) done (elapsed 2.93 s, remaining 0.82 s) 2024-09-10T22:47:33.232 INFO:teuthology.orchestra.run.smithi096.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.02 s, remaining 0.70 s) 2024-09-10T22:47:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.15 s, remaining 0.58 s) 2024-09-10T22:47:33.452 INFO:teuthology.orchestra.run.smithi096.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.24 s, remaining 0.46 s) 2024-09-10T22:47:33.579 INFO:teuthology.orchestra.run.smithi096.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.37 s, remaining 0.35 s) 2024-09-10T22:47:33.738 INFO:teuthology.orchestra.run.smithi096.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.53 s, remaining 0.24 s) 2024-09-10T22:47:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:33 smithi096 ceph-mon[30869]: pgmap v355: 97 pgs: 97 active+clean; 132 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 19 MiB/s wr, 84 op/s 2024-09-10T22:47:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:33 smithi121 ceph-mon[35119]: pgmap v355: 97 pgs: 97 active+clean; 132 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 19 MiB/s wr, 84 op/s 2024-09-10T22:47:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:33 smithi117 ceph-mon[35081]: pgmap v355: 97 pgs: 97 active+clean; 132 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 19 MiB/s wr, 84 op/s 2024-09-10T22:47:33.886 INFO:teuthology.orchestra.run.smithi096.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.67 s, remaining 0.12 s) 2024-09-10T22:47:33.965 INFO:teuthology.orchestra.run.smithi096.stderr:3200000 of 3200000 tuples (100%) done (elapsed 3.75 s, remaining 0.00 s) 2024-09-10T22:47:33.996 INFO:teuthology.orchestra.run.smithi096.stderr:vacuuming... 2024-09-10T22:47:34.562 INFO:teuthology.orchestra.run.smithi096.stderr:creating primary keys... 2024-09-10T22:47:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:34 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:47:34.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:34 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:47:34.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:34 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:47:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.280+0000 7f81d68ef640 1 -- 172.21.15.96:0/3550462367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 msgr2=0x7f81d0102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.280+0000 7f81d68ef640 1 --2- 172.21.15.96:0/3550462367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0102d90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009930 tx=0x7f81c002a440 comp rx=0 tx=0).stop 2024-09-10T22:47:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 -- 172.21.15.96:0/3550462367 shutdown_connections 2024-09-10T22:47:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 --2- 172.21.15.96:0/3550462367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d01044a0 0x7f81d010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 --2- 172.21.15.96:0/3550462367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d0103ae0 0x7f81d0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 --2- 172.21.15.96:0/3550462367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0102d90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 -- 172.21.15.96:0/3550462367 >> 172.21.15.96:0/3550462367 conn(0x7f81d00fe250 msgr2=0x7f81d0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 -- 172.21.15.96:0/3550462367 shutdown_connections 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 -- 172.21.15.96:0/3550462367 wait complete. 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.281+0000 7f81d68ef640 1 Processor -- start 2024-09-10T22:47:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 -- start start 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d0103ae0 0x7f81d0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d01044a0 0x7f81d019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81d010e600 con 0x7f81d0102990 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81d010e480 con 0x7f81d0103ae0 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d68ef640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81d010e780 con 0x7f81d01044a0 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39656/0 (socket says 172.21.15.96:39656) 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81cffff640 1 -- 172.21.15.96:0/438429355 learned_addr learned my addr 172.21.15.96:0/438429355 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d01044a0 0x7f81d019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.282+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d0103ae0 0x7f81d0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 -- 172.21.15.96:0/438429355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d01044a0 msgr2=0x7f81d019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d01044a0 0x7f81d019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 -- 172.21.15.96:0/438429355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d0103ae0 msgr2=0x7f81d0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d0103ae0 0x7f81d0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 -- 172.21.15.96:0/438429355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d019a980 con 0x7f81d0102990 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cffff640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81c0031470 tx=0x7f81c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cd7fa640 1 -- 172.21.15.96:0/438429355 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c003f030 con 0x7f81d0102990 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d019ac10 con 0x7f81d0102990 2024-09-10T22:47:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cd7fa640 1 -- 172.21.15.96:0/438429355 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0003aa0 con 0x7f81d0102990 2024-09-10T22:47:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81cd7fa640 1 -- 172.21.15.96:0/438429355 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0035070 con 0x7f81d0102990 2024-09-10T22:47:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.283+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d0075560 con 0x7f81d0102990 2024-09-10T22:47:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.287+0000 7f81d96bf740 1 Processor -- start 2024-09-10T22:47:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.287+0000 7f81d96bf740 1 -- start start 2024-09-10T22:47:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb3860 0x563947bb5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 0x563947bbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563947b256f0 con 0x563947bb6190 2024-09-10T22:47:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563947b14b20 con 0x563947bb3860 2024-09-10T22:47:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d96bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563947b18930 con 0x563947bb8bd0 2024-09-10T22:47:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.288+0000 7f81d4e8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb3860 0x563947bb5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cd7fa640 1 -- 172.21.15.96:0/438429355 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f81c00035f0 con 0x7f81d0102990 2024-09-10T22:47:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39658/0 (socket says 172.21.15.96:39658) 2024-09-10T22:47:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 -- 172.21.15.96:0/391167256 learned_addr learned my addr 172.21.15.96:0/391167256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 0x563947bbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 -- 172.21.15.96:0/391167256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 msgr2=0x563947bbb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 0x563947bbb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 -- 172.21.15.96:0/391167256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb3860 msgr2=0x563947bb5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb3860 0x563947bb5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.289+0000 7f81cffff640 1 -- 172.21.15.96:0/391167256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563947ada530 con 0x563947bb6190 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 0x563947bbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81cffff640 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81c0103db0 tx=0x7f81c010c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334660ee55f2c856 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81cd7fa640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 0x7f81b80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81b67fc640 1 -- 172.21.15.96:0/391167256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0106cd0 con 0x563947bb6190 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81b67fc640 1 -- 172.21.15.96:0/391167256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0113030 con 0x563947bb6190 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81b67fc640 1 -- 172.21.15.96:0/391167256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c00cc520 con 0x563947bb6190 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 0x7f81b80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.290+0000 7f81cd7fa640 1 -- 172.21.15.96:0/438429355 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81c01063c0 con 0x7f81d0102990 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.291+0000 7f81d96bf740 1 -- 172.21.15.96:0/391167256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 msgr2=0x563947bb8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.291+0000 7f81d96bf740 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81c0103db0 tx=0x7f81c010c6f0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 -- 172.21.15.96:0/391167256 shutdown_connections 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb8bd0 0x563947bbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bb8580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 --2- 172.21.15.96:0/391167256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb3860 0x563947bb5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 -- 172.21.15.96:0/391167256 >> 172.21.15.96:0/391167256 conn(0x563947bae9b0 msgr2=0x563947baf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 -- 172.21.15.96:0/391167256 shutdown_connections 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 -- 172.21.15.96:0/391167256 wait complete. 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 0x7f81b80b78b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81d0199bc0 tx=0x7f81c80096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.292+0000 7f81d96bf740 1 -- start start 2024-09-10T22:47:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 Processor -- start 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb6190 0x563947bbd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563947a4cab0 con 0x563947bb3860 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39674/0 (socket says 172.21.15.96:39674) 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81cffff640 1 -- 172.21.15.96:0/319149037 learned_addr learned my addr 172.21.15.96:0/319149037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563947aba730 con 0x563947bb8bd0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563947bbb970 con 0x563947bb6190 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81d96bf740 10 client.0 _ll_register_callbacks cb 0x563947b0e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.293+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb6190 0x563947bbd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 -- 172.21.15.96:0/319149037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb6190 msgr2=0x563947bbd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb6190 0x563947bbd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 -- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 msgr2=0x563947bbdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bbdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563947876790 con 0x563947bb3860 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81cffff640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f81c00c8be0 tx=0x7f81c002a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6749222947a08d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0049070 con 0x563947bb3860 2024-09-10T22:47:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81c00c8e40 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c0113030 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81c00cc340 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81d96bf740 10 client.15426 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.294+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639478ef620 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c00cde50 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f81c00cce30 con 0x563947bb3860 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 10 client.15426 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 20 client.15426 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947bc8280 0x563947bca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 4 client.15426 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563947b27640 con 0x563947bc8280 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.295+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947bc8280 0x563947bca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.296+0000 7f81af7fe640 10 client.15426 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.296+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947bc8280 0x563947bca690 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f01f98fd9482b44 server_cookie=c5d5ba508e226830 in_seq=0 out_seq=0 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.296+0000 7f81af7fe640 1 -- 172.21.15.96:0/319149037 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563947b27640 con 0x563947bc8280 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.296+0000 7f81af7fe640 10 client.15426 handle_command_reply: tid=0 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 client.15426 shutdown 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 msgr2=0x563947bbcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f81c00c8be0 tx=0x7f81c002a440 comp rx=0 tx=0).stop 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 shutdown_connections 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947bc8280 0x563947bca690 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81af7fe640 0 client.15426 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb6190 0x563947bbd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 --2- 172.21.15.96:0/319149037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb3860 0x563947bbcc40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 >> 172.21.15.96:0/319149037 conn(0x563947bae9b0 msgr2=0x563947badbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 shutdown_connections 2024-09-10T22:47:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.297+0000 7f81d96bf740 1 -- 172.21.15.96:0/319149037 wait complete. 2024-09-10T22:47:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.298+0000 7f81d96bf740 20 client.15426 trim_cache size 0 max 16384 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 Processor -- start 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 -- start start 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 0x563947bbe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bb1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01ac000 con 0x563947bb8bd0 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01adc00 con 0x563947bb6190 2024-09-10T22:47:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d96bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01ac6d0 con 0x563947bb3260 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39686/0 (socket says 172.21.15.96:39686) 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81cffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 0x563947bbe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81cf7fe640 1 -- 172.21.15.96:0/900673846 learned_addr learned my addr 172.21.15.96:0/900673846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bb1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.313+0000 7f81cf7fe640 1 -- 172.21.15.96:0/900673846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 msgr2=0x563947bb1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bb1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cf7fe640 1 -- 172.21.15.96:0/900673846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 msgr2=0x563947bbe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 0x563947bbe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cf7fe640 1 -- 172.21.15.96:0/900673846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563947bbe220 con 0x563947bb8bd0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cffff640 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 0x563947bbe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f81c4004490 tx=0x7f81c400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f8743a874d4ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81b5ffb640 1 -- 172.21.15.96:0/900673846 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4003b50 con 0x563947bb8bd0 2024-09-10T22:47:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81b5ffb640 1 -- 172.21.15.96:0/900673846 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c40054d0 con 0x563947bb8bd0 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.314+0000 7f81b5ffb640 1 -- 172.21.15.96:0/900673846 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4004c10 con 0x563947bb8bd0 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- 172.21.15.96:0/900673846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 msgr2=0x563947bb2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f81c4004490 tx=0x7f81c400bbe0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- 172.21.15.96:0/900673846 shutdown_connections 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bb1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb8bd0 0x563947bb2d20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 --2- 172.21.15.96:0/900673846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb6190 0x563947bbe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- 172.21.15.96:0/900673846 >> 172.21.15.96:0/900673846 conn(0x563947bae9b0 msgr2=0x563947bbc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- 172.21.15.96:0/900673846 shutdown_connections 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- 172.21.15.96:0/900673846 wait complete. 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 -- start start 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.315+0000 7f81d96bf740 1 Processor -- start 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bc44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bc3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01ad470 con 0x563947bb6190 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01acda0 con 0x563947bb8bd0 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81d01aa000 con 0x563947bb3260 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 10 client.0 _ll_register_callbacks cb 0x7f81c8009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39694/0 (socket says 172.21.15.96:39694) 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 -- 172.21.15.96:0/45086919 learned_addr learned my addr 172.21.15.96:0/45086919 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cffff640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bc44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 -- 172.21.15.96:0/45086919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 msgr2=0x563947bc44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bc44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 -- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 msgr2=0x563947bc3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bc3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 -- 172.21.15.96:0/45086919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563947bc3cc0 con 0x563947bb6190 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cffff640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d4e8c640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bc3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81cf7fe640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005710 tx=0x7f81c4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfee49628a7203bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4015040 con 0x563947bb6190 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81c4003b50 con 0x563947bb6190 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.316+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c40054d0 con 0x563947bb6190 2024-09-10T22:47:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81c4003d50 con 0x563947bb6190 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 10 client.15432 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563947c351a0 con 0x563947bb6190 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c40054d0 con 0x563947bb6190 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f81c401d970 con 0x563947bb6190 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 10 client.15432 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 20 client.15432 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947c35a70 0x563947c37e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 4 client.15432 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.317+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563947b230f0 con 0x563947c35a70 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.318+0000 7f81cffff640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947c35a70 0x563947c37e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.319+0000 7f81af7fe640 10 client.15432 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.319+0000 7f81cffff640 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947c35a70 0x563947c37e90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dde446c5f43ad27 server_cookie=a3f712ecd41d333d in_seq=0 out_seq=0 2024-09-10T22:47:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81af7fe640 1 -- 172.21.15.96:0/45086919 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563947b230f0 con 0x563947c35a70 2024-09-10T22:47:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81af7fe640 10 client.15432 handle_command_reply: tid=0 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 client.15432 shutdown 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 msgr2=0x563947bc31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005710 tx=0x7f81c4005740 comp rx=0 tx=0).stop 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 shutdown_connections 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563947c35a70 0x563947c37e90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563947bb8bd0 0x563947bc3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563947bb6190 0x563947bc31c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81af7fe640 0 client.15432 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 --2- 172.21.15.96:0/45086919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563947bb3260 0x563947bc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 >> 172.21.15.96:0/45086919 conn(0x563947bae9b0 msgr2=0x563947bb1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 shutdown_connections 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 1 -- 172.21.15.96:0/45086919 wait complete. 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.491+0000 7f81d96bf740 20 client.15432 trim_cache size 0 max 16384 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:47:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:47:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 msgr2=0x7f81b80b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 0x7f81b80b78b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81d0199bc0 tx=0x7f81c80096a0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 msgr2=0x7f81d0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81c0031470 tx=0x7f81c0004270 comp rx=0 tx=0).stop 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 shutdown_connections 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81d0103ae0 0x7f81d0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81b80b53f0 0x7f81b80b78b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d01044a0 0x7f81d019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 --2- 172.21.15.96:0/438429355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81d0102990 0x7f81d0198e40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 >> 172.21.15.96:0/438429355 conn(0x7f81d00fe250 msgr2=0x7f81d0106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 shutdown_connections 2024-09-10T22:47:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:35.492+0000 7f81d68ef640 1 -- 172.21.15.96:0/438429355 wait complete. 2024-09-10T22:47:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:35 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:35 smithi096 ceph-mon[30869]: pgmap v356: 97 pgs: 97 active+clean; 296 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 46 MiB/s wr, 143 op/s 2024-09-10T22:47:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:35 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:35 smithi121 ceph-mon[35119]: pgmap v356: 97 pgs: 97 active+clean; 296 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 46 MiB/s wr, 143 op/s 2024-09-10T22:47:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:35 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:47:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:35 smithi117 ceph-mon[35081]: pgmap v356: 97 pgs: 97 active+clean; 296 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 46 MiB/s wr, 143 op/s 2024-09-10T22:47:35.854 INFO:teuthology.orchestra.run.smithi096.stderr:done in 5.67 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 3.80 s, vacuum 0.57 s, primary keys 1.29 s). 2024-09-10T22:47:35.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-10T22:47:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:starting vacuum...end. 2024-09-10T22:47:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:37 smithi096 ceph-mon[30869]: pgmap v357: 97 pgs: 97 active+clean; 328 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 51 MiB/s wr, 155 op/s 2024-09-10T22:47:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:37 smithi121 ceph-mon[35119]: pgmap v357: 97 pgs: 97 active+clean; 328 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 51 MiB/s wr, 155 op/s 2024-09-10T22:47:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:37 smithi117 ceph-mon[35081]: pgmap v357: 97 pgs: 97 active+clean; 328 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 51 MiB/s wr, 155 op/s 2024-09-10T22:47:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:39 smithi096 ceph-mon[30869]: pgmap v358: 97 pgs: 97 active+clean; 465 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 72 MiB/s wr, 353 op/s 2024-09-10T22:47:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:39 smithi121 ceph-mon[35119]: pgmap v358: 97 pgs: 97 active+clean; 465 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 72 MiB/s wr, 353 op/s 2024-09-10T22:47:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:39 smithi117 ceph-mon[35081]: pgmap v358: 97 pgs: 97 active+clean; 465 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 72 MiB/s wr, 353 op/s 2024-09-10T22:47:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 5.0 s, 3598.2 tps, lat 27.196 ms stddev 28.264 2024-09-10T22:47:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:41 smithi096 ceph-mon[30869]: pgmap v359: 97 pgs: 97 active+clean; 620 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 430 op/s 2024-09-10T22:47:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:41 smithi121 ceph-mon[35119]: pgmap v359: 97 pgs: 97 active+clean; 620 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 430 op/s 2024-09-10T22:47:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:41 smithi117 ceph-mon[35081]: pgmap v359: 97 pgs: 97 active+clean; 620 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 430 op/s 2024-09-10T22:47:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:43 smithi096 ceph-mon[30869]: pgmap v360: 97 pgs: 97 active+clean; 778 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 479 op/s 2024-09-10T22:47:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:43 smithi121 ceph-mon[35119]: pgmap v360: 97 pgs: 97 active+clean; 778 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 479 op/s 2024-09-10T22:47:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:43 smithi117 ceph-mon[35081]: pgmap v360: 97 pgs: 97 active+clean; 778 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 76 MiB/s wr, 479 op/s 2024-09-10T22:47:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:45 smithi096 ceph-mon[30869]: pgmap v361: 97 pgs: 97 active+clean; 894 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 62 MiB/s wr, 656 op/s 2024-09-10T22:47:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:45 smithi121 ceph-mon[35119]: pgmap v361: 97 pgs: 97 active+clean; 894 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 62 MiB/s wr, 656 op/s 2024-09-10T22:47:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:45 smithi117 ceph-mon[35081]: pgmap v361: 97 pgs: 97 active+clean; 894 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 62 MiB/s wr, 656 op/s 2024-09-10T22:47:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 10.0 s, 4015.0 tps, lat 25.014 ms stddev 26.156 2024-09-10T22:47:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:47 smithi096 ceph-mon[30869]: pgmap v362: 97 pgs: 97 active+clean; 899 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 605 op/s 2024-09-10T22:47:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:47 smithi121 ceph-mon[35119]: pgmap v362: 97 pgs: 97 active+clean; 899 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 605 op/s 2024-09-10T22:47:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:47 smithi117 ceph-mon[35081]: pgmap v362: 97 pgs: 97 active+clean; 899 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 605 op/s 2024-09-10T22:47:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.603+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1335706972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4181044a0 msgr2=0x7ff41810ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.603+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1335706972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4181044a0 0x7ff41810ae20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff410009cd0 tx=0x7ff41002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.603+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1335706972 shutdown_connections 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.603+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1335706972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4181044a0 0x7ff41810ae20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.603+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1335706972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff418103ae0 0x7ff418103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.604+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1335706972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418102990 0x7ff418102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.604+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1335706972 >> 172.21.15.96:0/1335706972 conn(0x7ff4180fe250 msgr2=0x7ff418100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.604+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1335706972 shutdown_connections 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.604+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1335706972 wait complete. 2024-09-10T22:47:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.604+0000 7ff41dcbd640 1 Processor -- start 2024-09-10T22:47:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 -- start start 2024-09-10T22:47:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff418102990 0x7ff418198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 0x7ff41819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810e600 con 0x7ff418102990 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810e480 con 0x7ff418103ae0 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff41dcbd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff41810e780 con 0x7ff4181044a0 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff417fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 0x7ff41819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45040/0 (socket says 172.21.15.96:45040) 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.605+0000 7ff416ffd640 1 -- 172.21.15.96:0/1293069472 learned_addr learned my addr 172.21.15.96:0/1293069472 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff4177fe640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff418102990 0x7ff418198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 -- 172.21.15.96:0/1293069472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 msgr2=0x7ff41819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 0x7ff41819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 -- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff418102990 msgr2=0x7ff418198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff418102990 0x7ff418198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 -- 172.21.15.96:0/1293069472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41819a920 con 0x7ff418103ae0 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff417fff640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 0x7ff41819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.606+0000 7ff416ffd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00e9f0 tx=0x7ff40c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.607+0000 7ff414ff9640 1 -- 172.21.15.96:0/1293069472 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00cd80 con 0x7ff418103ae0 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.607+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41819ac10 con 0x7ff418103ae0 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.607+0000 7ff414ff9640 1 -- 172.21.15.96:0/1293069472 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0054d0 con 0x7ff418103ae0 2024-09-10T22:47:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.607+0000 7ff414ff9640 1 -- 172.21.15.96:0/1293069472 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0186d0 con 0x7ff418103ae0 2024-09-10T22:47:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.607+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41806cd50 con 0x7ff418103ae0 2024-09-10T22:47:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff414ff9640 1 -- 172.21.15.96:0/1293069472 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7ff40c020030 con 0x7ff418103ae0 2024-09-10T22:47:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 Processor -- start 2024-09-10T22:47:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 -- start start 2024-09-10T22:47:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e835c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e838580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e83b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56400e7a56f0 con 0x56400e838bd0 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56400e794b20 con 0x56400e836190 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.612+0000 7ff420a8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56400e798930 con 0x56400e833860 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff414ff9640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 0x7ff4000b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff417fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e835c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff417fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e835c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51680/0 (socket says 172.21.15.96:51680) 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff417fff640 1 -- 172.21.15.96:0/3777568025 learned_addr learned my addr 172.21.15.96:0/3777568025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff414ff9640 1 -- 172.21.15.96:0/1293069472 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff40c0e5d40 con 0x7ff418103ae0 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff4177fe640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e838580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e83b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff417fff640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 0x7ff4000b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 -- 172.21.15.96:0/3777568025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 msgr2=0x56400e835c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e835c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 -- 172.21.15.96:0/3777568025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 msgr2=0x56400e838580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e838580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 -- 172.21.15.96:0/3777568025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400e75a530 con 0x56400e838bd0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff4177fe640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e838580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff417fff640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 0x7ff4000b78f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff41002aed0 tx=0x7ff41002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.613+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e83b090 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0e37f0 tx=0x7ff40c0abc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d14f529819d0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.614+0000 7ff3fdffb640 1 -- 172.21.15.96:0/3777568025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0abfb0 con 0x56400e838bd0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.614+0000 7ff3fdffb640 1 -- 172.21.15.96:0/3777568025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0acba0 con 0x56400e838bd0 2024-09-10T22:47:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.614+0000 7ff3fdffb640 1 -- 172.21.15.96:0/3777568025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0d3420 con 0x56400e838bd0 2024-09-10T22:47:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 -- 172.21.15.96:0/3777568025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 msgr2=0x56400e83b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e83b090 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0e37f0 tx=0x7ff40c0abc10 comp rx=0 tx=0).stop 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 -- 172.21.15.96:0/3777568025 shutdown_connections 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e838580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e83b090 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3777568025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e835c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 -- 172.21.15.96:0/3777568025 >> 172.21.15.96:0/3777568025 conn(0x56400e82e9b0 msgr2=0x56400e82f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 -- 172.21.15.96:0/3777568025 shutdown_connections 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.615+0000 7ff420a8f740 1 -- 172.21.15.96:0/3777568025 wait complete. 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.616+0000 7ff420a8f740 1 -- start start 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.616+0000 7ff420a8f740 1 Processor -- start 2024-09-10T22:47:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e83d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e83e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56400e6ccab0 con 0x56400e836190 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56400e73a730 con 0x56400e838bd0 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56400e83bb50 con 0x56400e833860 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff420a8f740 10 client.0 _ll_register_callbacks cb 0x56400e78e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff4177fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51694/0 (socket says 172.21.15.96:51694) 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff4177fe640 1 -- 172.21.15.96:0/761375474 learned_addr learned my addr 172.21.15.96:0/761375474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff417fff640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e83e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.617+0000 7ff4177fe640 1 -- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 msgr2=0x56400e83e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff4177fe640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e83e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff4177fe640 1 -- 172.21.15.96:0/761375474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 msgr2=0x56400e83d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff4177fe640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e83d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff4177fe640 1 -- 172.21.15.96:0/761375474 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400e4f6790 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff4177fe640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff408012700 tx=0x7ff408012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10a6e9a639f78e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff3f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408004260 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff408004400 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff40c0acba0 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4080119a0 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff420a8f740 10 client.25027 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.618+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56400e56f620 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0acba0 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff40800f440 con 0x56400e833860 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 10 client.25027 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 10 client.25027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 20 client.25027 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e848900 0x56400e84ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 4 client.25027 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400e7a7640 con 0x56400e848900 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff416ffd640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e848900 0x56400e84ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff3f6ffd640 10 client.25027 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.619+0000 7ff416ffd640 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e848900 0x56400e84ada0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d319a3fd26bebc server_cookie=32f8bab783729720 in_seq=0 out_seq=0 2024-09-10T22:47:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/761375474 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56400e7a7640 con 0x56400e848900 2024-09-10T22:47:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff3f6ffd640 10 client.25027 handle_command_reply: tid=0 2024-09-10T22:47:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 client.25027 shutdown 2024-09-10T22:47:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 msgr2=0x56400e83cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff408012700 tx=0x7ff408012bc0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 shutdown_connections 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e848900 0x56400e84ada0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e83e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e83d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff3f6ffd640 0 client.25027 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 --2- 172.21.15.96:0/761375474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 >> 172.21.15.96:0/761375474 conn(0x56400e82e9b0 msgr2=0x56400e82dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 shutdown_connections 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 1 -- 172.21.15.96:0/761375474 wait complete. 2024-09-10T22:47:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.620+0000 7ff420a8f740 20 client.25027 trim_cache size 0 max 16384 2024-09-10T22:47:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 Processor -- start 2024-09-10T22:47:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 -- start start 2024-09-10T22:47:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e833320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4180758b0 con 0x56400e836190 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4181b2110 con 0x56400e838bd0 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff420a8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff418075f80 con 0x56400e833860 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57176/0 (socket says 172.21.15.96:57176) 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 -- 172.21.15.96:0/3264812857 learned_addr learned my addr 172.21.15.96:0/3264812857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff4177fe640 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff417fff640 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e833320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 -- 172.21.15.96:0/3264812857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 msgr2=0x56400e83cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 -- 172.21.15.96:0/3264812857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 msgr2=0x56400e833320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e833320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 -- 172.21.15.96:0/3264812857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400e83e4e0 con 0x56400e836190 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.635+0000 7ff416ffd640 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0ac1a0 tx=0x7ff40c009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9cec98e87065db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff3fd7fa640 1 -- 172.21.15.96:0/3264812857 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0c72a0 con 0x56400e836190 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff3fd7fa640 1 -- 172.21.15.96:0/3264812857 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0ac4a0 con 0x56400e836190 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff3fd7fa640 1 -- 172.21.15.96:0/3264812857 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0a6ba0 con 0x56400e836190 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- 172.21.15.96:0/3264812857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 msgr2=0x56400e832de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0ac1a0 tx=0x7ff40c009720 comp rx=0 tx=0).stop 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- 172.21.15.96:0/3264812857 shutdown_connections 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e838bd0 0x56400e833320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e836190 0x56400e832de0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 --2- 172.21.15.96:0/3264812857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e83cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- 172.21.15.96:0/3264812857 >> 172.21.15.96:0/3264812857 conn(0x56400e82e9b0 msgr2=0x56400e829af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- 172.21.15.96:0/3264812857 shutdown_connections 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- 172.21.15.96:0/3264812857 wait complete. 2024-09-10T22:47:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.636+0000 7ff420a8f740 1 -- start start 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 Processor -- start 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e841f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e8425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff418076d20 con 0x56400e838bd0 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff418076650 con 0x56400e836190 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff418073890 con 0x56400e833860 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff420a8f740 10 client.0 _ll_register_callbacks cb 0x7ff408009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57182/0 (socket says 172.21.15.96:57182) 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 -- 172.21.15.96:0/924995706 learned_addr learned my addr 172.21.15.96:0/924995706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff4177fe640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e841f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff416ffd640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e8425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 -- 172.21.15.96:0/924995706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 msgr2=0x56400e841f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e841f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 -- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 msgr2=0x56400e8425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e8425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff417fff640 1 -- 172.21.15.96:0/924995706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400e8439f0 con 0x56400e838bd0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.637+0000 7ff4177fe640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e841f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff417fff640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff410035000 tx=0x7ff4100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a18f64fb1433415 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41005b070 con 0x56400e838bd0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff4100307b0 con 0x56400e838bd0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff40c0ac4a0 con 0x56400e838bd0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff410067050 con 0x56400e838bd0 2024-09-10T22:47:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 10 client.15447 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56400e844e00 con 0x56400e838bd0 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0ac4a0 con 0x56400e838bd0 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff410030070 con 0x56400e838bd0 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 10 client.15447 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 20 client.15447 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e8b5f10 0x56400e8b8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 4 client.15447 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56400e7a30f0 con 0x56400e8b5f10 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.638+0000 7ff4177fe640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e8b5f10 0x56400e8b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.639+0000 7ff3f6ffd640 10 client.15447 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.639+0000 7ff4177fe640 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e8b5f10 0x56400e8b8330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae67a503737f09c server_cookie=ca4124a7cc99aede in_seq=0 out_seq=0 2024-09-10T22:47:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff3f6ffd640 1 -- 172.21.15.96:0/924995706 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56400e7a30f0 con 0x56400e8b5f10 2024-09-10T22:47:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff3f6ffd640 10 client.15447 handle_command_reply: tid=0 2024-09-10T22:47:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 client.15447 shutdown 2024-09-10T22:47:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 msgr2=0x56400e8431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff410035000 tx=0x7ff4100095c0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 shutdown_connections 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56400e8b5f10 0x56400e8b8330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56400e836190 0x56400e8425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400e838bd0 0x56400e8431c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff3f6ffd640 0 client.15447 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 --2- 172.21.15.96:0/924995706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56400e833860 0x56400e841f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 >> 172.21.15.96:0/924995706 conn(0x56400e82e9b0 msgr2=0x56400e832de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 shutdown_connections 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.746+0000 7ff420a8f740 1 -- 172.21.15.96:0/924995706 wait complete. 2024-09-10T22:47:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.747+0000 7ff420a8f740 20 client.15447 trim_cache size 0 max 16384 2024-09-10T22:47:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:47:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:47:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:47:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.747+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 msgr2=0x7ff4000b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.747+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 0x7ff4000b78f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff41002aed0 tx=0x7ff41002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.747+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 msgr2=0x7ff418199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.747+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00e9f0 tx=0x7ff40c00edf0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 shutdown_connections 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff418103ae0 0x7ff418199430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4000b5430 0x7ff4000b78f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4181044a0 0x7ff41819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 --2- 172.21.15.96:0/1293069472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff418102990 0x7ff418198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 >> 172.21.15.96:0/1293069472 conn(0x7ff4180fe250 msgr2=0x7ff418106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 shutdown_connections 2024-09-10T22:47:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:47:48.748+0000 7ff41dcbd640 1 -- 172.21.15.96:0/1293069472 wait complete. 2024-09-10T22:47:49.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:49 smithi096 ceph-mon[30869]: pgmap v363: 97 pgs: 97 active+clean; 922 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 789 op/s 2024-09-10T22:47:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:49 smithi121 ceph-mon[35119]: pgmap v363: 97 pgs: 97 active+clean; 922 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 789 op/s 2024-09-10T22:47:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:49 smithi117 ceph-mon[35081]: pgmap v363: 97 pgs: 97 active+clean; 922 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 789 op/s 2024-09-10T22:47:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 15.0 s, 3661.8 tps, lat 27.322 ms stddev 28.903 2024-09-10T22:47:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:51 smithi096 ceph-mon[30869]: pgmap v364: 97 pgs: 97 active+clean; 927 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T22:47:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:51 smithi121 ceph-mon[35119]: pgmap v364: 97 pgs: 97 active+clean; 927 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T22:47:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:51 smithi117 ceph-mon[35081]: pgmap v364: 97 pgs: 97 active+clean; 927 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T22:47:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:53 smithi096 ceph-mon[30869]: pgmap v365: 97 pgs: 97 active+clean; 930 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 739 op/s 2024-09-10T22:47:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:53 smithi121 ceph-mon[35119]: pgmap v365: 97 pgs: 97 active+clean; 930 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 739 op/s 2024-09-10T22:47:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:53 smithi117 ceph-mon[35081]: pgmap v365: 97 pgs: 97 active+clean; 930 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.9 MiB/s wr, 739 op/s 2024-09-10T22:47:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:55 smithi096 ceph-mon[30869]: pgmap v366: 97 pgs: 97 active+clean; 936 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T22:47:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:55 smithi121 ceph-mon[35119]: pgmap v366: 97 pgs: 97 active+clean; 936 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T22:47:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:55 smithi117 ceph-mon[35081]: pgmap v366: 97 pgs: 97 active+clean; 936 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T22:47:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 20.0 s, 3710.0 tps, lat 27.005 ms stddev 26.596 2024-09-10T22:47:56.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:47:56.758 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:57.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.325+0000 7f2e2fe79640 1 -- 172.21.15.96:0/86092566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 msgr2=0x7f2e2810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.325+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/86092566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e2810a530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009900 tx=0x7f2e2402a490 comp rx=0 tx=0).stop 2024-09-10T22:47:57.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.326+0000 7f2e2fe79640 1 -- 172.21.15.96:0/86092566 shutdown_connections 2024-09-10T22:47:57.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.326+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/86092566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e28115a10 0x7f2e28117e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.326+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/86092566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e2810aa70 0x7f2e281154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.326+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/86092566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e2810a530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.326+0000 7f2e2fe79640 1 -- 172.21.15.96:0/86092566 >> 172.21.15.96:0/86092566 conn(0x7f2e2806dfc0 msgr2=0x7f2e2806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.327+0000 7f2e2fe79640 1 -- 172.21.15.96:0/86092566 shutdown_connections 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.327+0000 7f2e2fe79640 1 -- 172.21.15.96:0/86092566 wait complete. 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 Processor -- start 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 -- start start 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e28115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e2810aa70 0x7f2e28111690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810c060 con 0x7f2e2810aa70 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810bee0 con 0x7f2e2810a150 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2fe79640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810c1e0 con 0x7f2e28115a10 2024-09-10T22:47:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2e416640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2e416640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40762/0 (socket says 172.21.15.96:40762) 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.328+0000 7f2e2e416640 1 -- 172.21.15.96:0/1154047482 learned_addr learned my addr 172.21.15.96:0/1154047482 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2d414640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e2810aa70 0x7f2e28111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e28115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 -- 172.21.15.96:0/1154047482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 msgr2=0x7f2e28115450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e28115450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 -- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e2810aa70 msgr2=0x7f2e28111690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e2810aa70 0x7f2e28111690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 -- 172.21.15.96:0/1154047482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281b4100 con 0x7f2e28115a10 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.329+0000 7f2e2e416640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000eb40 tx=0x7f2e2000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.330+0000 7f2e1effd640 1 -- 172.21.15.96:0/1154047482 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20004b20 con 0x7f2e28115a10 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.330+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281b43f0 con 0x7f2e28115a10 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.330+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281b48d0 con 0x7f2e28115a10 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.330+0000 7f2e1effd640 1 -- 172.21.15.96:0/1154047482 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e20004cc0 con 0x7f2e28115a10 2024-09-10T22:47:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.330+0000 7f2e1effd640 1 -- 172.21.15.96:0/1154047482 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20010690 con 0x7f2e28115a10 2024-09-10T22:47:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.333+0000 7f2e1effd640 1 -- 172.21.15.96:0/1154047482 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f2e20010830 con 0x7f2e28115a10 2024-09-10T22:47:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.334+0000 7f2e1effd640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 0x7f2e140b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.335+0000 7f2e1effd640 1 -- 172.21.15.96:0/1154047482 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2e200e6000 con 0x7f2e28115a10 2024-09-10T22:47:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.336+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 0x7f2e140b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.336+0000 7f2e3224e740 1 Processor -- start 2024-09-10T22:47:57.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.336+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 0x7f2e140b79c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009850 tx=0x7f2e2400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.338+0000 7f2e3224e740 1 -- start start 2024-09-10T22:47:57.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31d480 0x56405a31d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a329d30 0x56405a32c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56405a293e50 con 0x56405a31dda0 2024-09-10T22:47:57.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56405a283280 con 0x56405a31d480 2024-09-10T22:47:57.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e3224e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56405a287090 con 0x56405a329d30 2024-09-10T22:47:57.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41462/0 (socket says 172.21.15.96:41462) 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 -- 172.21.15.96:0/2156343024 learned_addr learned my addr 172.21.15.96:0/2156343024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2e416640 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a329d30 0x56405a32c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 -- 172.21.15.96:0/2156343024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a329d30 msgr2=0x56405a32c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a329d30 0x56405a32c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 -- 172.21.15.96:0/2156343024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31d480 msgr2=0x56405a31d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31d480 0x56405a31d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.339+0000 7f2e2d414640 1 -- 172.21.15.96:0/2156343024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564059fe1650 con 0x56405a31dda0 2024-09-10T22:47:57.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.340+0000 7f2e2d414640 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009c20 tx=0x7f2e180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bdca573d69b852 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.340+0000 7f2e13fff640 1 -- 172.21.15.96:0/2156343024 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18012450 con 0x56405a31dda0 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.340+0000 7f2e13fff640 1 -- 172.21.15.96:0/2156343024 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e18012aa0 con 0x56405a31dda0 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.340+0000 7f2e13fff640 1 -- 172.21.15.96:0/2156343024 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800fa90 con 0x56405a31dda0 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 -- 172.21.15.96:0/2156343024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 msgr2=0x56405a3297f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009c20 tx=0x7f2e180117b0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 -- 172.21.15.96:0/2156343024 shutdown_connections 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a329d30 0x56405a32c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a3297f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 --2- 172.21.15.96:0/2156343024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31d480 0x56405a31d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.341+0000 7f2e3224e740 1 -- 172.21.15.96:0/2156343024 >> 172.21.15.96:0/2156343024 conn(0x56405a31b2b0 msgr2=0x56405a31c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.342+0000 7f2e3224e740 1 -- 172.21.15.96:0/2156343024 shutdown_connections 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.342+0000 7f2e3224e740 1 -- 172.21.15.96:0/2156343024 wait complete. 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.342+0000 7f2e3224e740 1 -- start start 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.342+0000 7f2e3224e740 1 Processor -- start 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.343+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a326a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a327020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56405a0e4700 con 0x56405a31dda0 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56405a21e110 con 0x56405a329d30 2024-09-10T22:47:57.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56405a32c9b0 con 0x56405a31d480 2024-09-10T22:47:57.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e3224e740 10 client.0 _ll_register_callbacks cb 0x56405a237990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:57.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e2dc15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e2dc15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40766/0 (socket says 172.21.15.96:40766) 2024-09-10T22:47:57.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e2dc15640 1 -- 172.21.15.96:0/4034907744 learned_addr learned my addr 172.21.15.96:0/4034907744 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.344+0000 7f2e2d414640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a326a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2e416640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a327020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2dc15640 1 -- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 msgr2=0x56405a327020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a327020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2dc15640 1 -- 172.21.15.96:0/4034907744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 msgr2=0x56405a326a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a326a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.345+0000 7f2e2dc15640 1 -- 172.21.15.96:0/4034907744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405a24bc40 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.346+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e240038a0 tx=0x7f2e24000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b5930abcbc8f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.346+0000 7f2e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.346+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24035050 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.346+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2e24032980 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.346+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e18012aa0 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.347+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e24032b80 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.347+0000 7f2e3224e740 10 client.25045 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.347+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56405a338350 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.348+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e2404e070 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2e24053450 con 0x56405a31d480 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 10 client.25045 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 10 client.25045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 20 client.25045 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a339d10 0x56405a33c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 4 client.25045 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2e18012aa0 con 0x56405a339d10 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.349+0000 7f2e2d414640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a339d10 0x56405a33c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.351+0000 7f2e2d414640 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a339d10 0x56405a33c130 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68cbebd324a0d7bb server_cookie=89119977a0a0db2f in_seq=0 out_seq=0 2024-09-10T22:47:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.351+0000 7f2e117fa640 10 client.25045 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.352+0000 7f2e117fa640 1 -- 172.21.15.96:0/4034907744 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2e18012aa0 con 0x56405a339d10 2024-09-10T22:47:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.352+0000 7f2e117fa640 10 client.25045 handle_command_reply: tid=0 2024-09-10T22:47:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.352+0000 7f2e3224e740 1 client.25045 shutdown 2024-09-10T22:47:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 msgr2=0x56405a3295f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e240038a0 tx=0x7f2e24000bc0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 shutdown_connections 2024-09-10T22:47:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a339d10 0x56405a33c130 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a327020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a326a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 --2- 172.21.15.96:0/4034907744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a3295f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 >> 172.21.15.96:0/4034907744 conn(0x56405a31b2b0 msgr2=0x56405a320e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e117fa640 0 client.25045 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 shutdown_connections 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.356+0000 7f2e3224e740 1 -- 172.21.15.96:0/4034907744 wait complete. 2024-09-10T22:47:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.357+0000 7f2e3224e740 20 client.25045 trim_cache size 0 max 16384 2024-09-10T22:47:57.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 Processor -- start 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 -- start start 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a31abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a31c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00e220 con 0x56405a31dda0 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00fe20 con 0x56405a329d30 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e3224e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00e8f0 con 0x56405a31d480 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e2d414640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e2d414640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41474/0 (socket says 172.21.15.96:41474) 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e2d414640 1 -- 172.21.15.96:0/3576717086 learned_addr learned my addr 172.21.15.96:0/3576717086 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.383+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a31abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2e416640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a31c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 -- 172.21.15.96:0/3576717086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 msgr2=0x56405a31abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a31abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 -- 172.21.15.96:0/3576717086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 msgr2=0x56405a31c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a31c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 -- 172.21.15.96:0/3576717086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405a31dbd0 con 0x56405a31dda0 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a31abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e2d414640 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e180099e0 tx=0x7f2e1800f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e17460d5b94f1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e137fe640 1 -- 172.21.15.96:0/3576717086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18009650 con 0x56405a31dda0 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e137fe640 1 -- 172.21.15.96:0/3576717086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e18009d10 con 0x56405a31dda0 2024-09-10T22:47:57.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.384+0000 7f2e137fe640 1 -- 172.21.15.96:0/3576717086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18012a20 con 0x56405a31dda0 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 -- 172.21.15.96:0/3576717086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 msgr2=0x56405a31c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e180099e0 tx=0x7f2e1800f630 comp rx=0 tx=0).stop 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 -- 172.21.15.96:0/3576717086 shutdown_connections 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a329d30 0x56405a31c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a31dda0 0x56405a31c370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 --2- 172.21.15.96:0/3576717086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a31abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 -- 172.21.15.96:0/3576717086 >> 172.21.15.96:0/3576717086 conn(0x56405a31b2b0 msgr2=0x56405a321ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.385+0000 7f2e3224e740 1 -- 172.21.15.96:0/3576717086 shutdown_connections 2024-09-10T22:47:57.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.386+0000 7f2e3224e740 1 -- 172.21.15.96:0/3576717086 wait complete. 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.386+0000 7f2e3224e740 1 -- start start 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.386+0000 7f2e3224e740 1 Processor -- start 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e3224e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 0x56405a3416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e2dc15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e2dc15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40780/0 (socket says 172.21.15.96:40780) 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e2dc15640 1 -- 172.21.15.96:0/1886077149 learned_addr learned my addr 172.21.15.96:0/1886077149 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a329d30 0x56405a341c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.387+0000 7f2e2d414640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 0x56405a3416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00f690 con 0x56405a329d30 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00efc0 con 0x56405a31dda0 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c00c2b0 con 0x56405a31d480 2024-09-10T22:47:57.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2e416640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a329d30 0x56405a341c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e3224e740 10 client.0 _ll_register_callbacks cb 0x7f2e2403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2d414640 1 -- 172.21.15.96:0/1886077149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 msgr2=0x56405a332520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2d414640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2d414640 1 -- 172.21.15.96:0/1886077149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a329d30 msgr2=0x56405a341c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2d414640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a329d30 0x56405a341c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2d414640 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405a3424b0 con 0x56405a31dda0 2024-09-10T22:47:57.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.388+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.389+0000 7f2e2d414640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 0x56405a3416c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005fd0 tx=0x7f2e18012680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d4346cd22c4b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18009650 con 0x56405a31dda0 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2e180129b0 con 0x56405a31dda0 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e18009d10 con 0x56405a31dda0 2024-09-10T22:47:57.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2e180050b0 con 0x56405a31dda0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e3224e740 10 client.25031 fetch_fsmap learned FSMap version 30 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.390+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56405a3a6cc0 con 0x56405a31dda0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1800f6a0 con 0x56405a31dda0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2e180035f0 con 0x56405a31dda0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 10 client.25031 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 20 client.25031 populate_metadata read hostname 'smithi096' 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a3a71e0 0x56405a3a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 4 client.25031 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.391+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2e18009d10 con 0x56405a3a71e0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a3a71e0 0x56405a3a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e2dc15640 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a3a71e0 0x56405a3a9600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97a24103dc5fbc7 server_cookie=4188a09f5bb114c7 in_seq=0 out_seq=0 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e117fa640 10 client.25031 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:57.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e117fa640 1 -- 172.21.15.96:0/1886077149 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7f2e18009d10 con 0x56405a3a71e0 2024-09-10T22:47:57.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e117fa640 10 client.25031 handle_command_reply: tid=0 2024-09-10T22:47:57.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.392+0000 7f2e3224e740 1 client.25031 shutdown 2024-09-10T22:47:57.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 msgr2=0x56405a3416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 0x56405a3416c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2e18005fd0 tx=0x7f2e18012680 comp rx=0 tx=0).stop 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 shutdown_connections 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56405a3a71e0 0x56405a3a9600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56405a31dda0 0x56405a3416c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56405a329d30 0x56405a341c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 --2- 172.21.15.96:0/1886077149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56405a31d480 0x56405a332520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e117fa640 0 client.25031 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 >> 172.21.15.96:0/1886077149 conn(0x56405a31b2b0 msgr2=0x56405a31c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 shutdown_connections 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 1 -- 172.21.15.96:0/1886077149 wait complete. 2024-09-10T22:47:57.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.394+0000 7f2e3224e740 20 client.25031 trim_cache size 0 max 16384 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:47:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:47:57.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:47:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.395+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 msgr2=0x7f2e140b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.395+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 0x7f2e140b79c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009850 tx=0x7f2e2400b040 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.395+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 msgr2=0x7f2e28111e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.395+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000eb40 tx=0x7f2e2000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 shutdown_connections 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2e140b5500 0x7f2e140b79c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e28115a10 0x7f2e28111e50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e2810aa70 0x7f2e28111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 --2- 172.21.15.96:0/1154047482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e2810a150 0x7f2e28115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.396+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 >> 172.21.15.96:0/1154047482 conn(0x7f2e2806dfc0 msgr2=0x7f2e2811c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.397+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 shutdown_connections 2024-09-10T22:47:57.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:57.397+0000 7f2e2fe79640 1 -- 172.21.15.96:0/1154047482 wait complete. 2024-09-10T22:47:57.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:57 smithi096 ceph-mon[30869]: pgmap v367: 97 pgs: 97 active+clean; 945 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 708 op/s 2024-09-10T22:47:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:57 smithi121 ceph-mon[35119]: pgmap v367: 97 pgs: 97 active+clean; 945 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 708 op/s 2024-09-10T22:47:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:57 smithi117 ceph-mon[35081]: pgmap v367: 97 pgs: 97 active+clean; 945 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 MiB/s wr, 708 op/s 2024-09-10T22:47:58.239 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:47:58.239 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:47:58.590 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:47:59.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.114+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2255086235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 msgr2=0x7f4b18102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.114+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2255086235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b18102070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009900 tx=0x7f4b0802a410 comp rx=0 tx=0).stop 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.114+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2255086235 shutdown_connections 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.114+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2255086235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 0x7f4b18107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.114+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2255086235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 0x7f4b1810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.115+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2255086235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b18102070 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.115+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2255086235 >> 172.21.15.96:0/2255086235 conn(0x7f4b180fd700 msgr2=0x7f4b180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.115+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2255086235 shutdown_connections 2024-09-10T22:47:59.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.115+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2255086235 wait complete. 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 Processor -- start 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 -- start start 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 0x7f4b1819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 0x7f4b1819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18111ca0 con 0x7f4b18101c90 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18111b20 con 0x7f4b181076a0 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.116+0000 7f4b1cca9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18111e20 con 0x7f4b18108020 2024-09-10T22:47:59.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41498/0 (socket says 172.21.15.96:41498) 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 -- 172.21.15.96:0/2618829385 learned_addr learned my addr 172.21.15.96:0/2618829385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b0ffff640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 0x7f4b1819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b16d9d640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 0x7f4b1819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 -- 172.21.15.96:0/2618829385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 msgr2=0x7f4b1819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 0x7f4b1819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 -- 172.21.15.96:0/2618829385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 msgr2=0x7f4b1819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 0x7f4b1819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 -- 172.21.15.96:0/2618829385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b181a5490 con 0x7f4b18101c90 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1659c640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b08006010 tx=0x7f4b080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08040070 con 0x7f4b18101c90 2024-09-10T22:47:59.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b181a5720 con 0x7f4b18101c90 2024-09-10T22:47:59.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181a5c50 con 0x7f4b18101c90 2024-09-10T22:47:59.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b08004c60 con 0x7f4b18101c90 2024-09-10T22:47:59.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.117+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0804a060 con 0x7f4b18101c90 2024-09-10T22:47:59.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.119+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b18102070 con 0x7f4b18101c90 2024-09-10T22:47:59.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.121+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f4b0803a040 con 0x7f4b18101c90 2024-09-10T22:47:59.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.122+0000 7f4b0f7fe640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 0x7f4b040b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:47:59.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.122+0000 7f4b0ffff640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 0x7f4b040b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:47:59.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.122+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4b08106bf0 con 0x7f4b18101c90 2024-09-10T22:47:59.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.122+0000 7f4b0ffff640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 0x7f4b040b7930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b1806b6b0 tx=0x7f4b0000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:47:59.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.126+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b080cd300 con 0x7f4b18101c90 2024-09-10T22:47:59.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.460+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b1806aec0 con 0x7f4b18101c90 2024-09-10T22:47:59.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.461+0000 7f4b0f7fe640 1 -- 172.21.15.96:0/2618829385 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f4b1806aec0 con 0x7f4b18101c90 2024-09-10T22:47:59.462 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:47:59.463 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":30,"btime":"2024-09-10T22:47:34:397648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:47:34.250205+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:47:59.463 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 30 2024-09-10T22:47:59.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 msgr2=0x7f4b040b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 0x7f4b040b7930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b1806b6b0 tx=0x7f4b0000b410 comp rx=0 tx=0).stop 2024-09-10T22:47:59.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 msgr2=0x7f4b1819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:47:59.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b08006010 tx=0x7f4b080045e0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 shutdown_connections 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b181076a0 0x7f4b1819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4b040b5470 0x7f4b040b7930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b18108020 0x7f4b1819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 --2- 172.21.15.96:0/2618829385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b18101c90 0x7f4b1819a0b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:47:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.465+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 >> 172.21.15.96:0/2618829385 conn(0x7f4b180fd700 msgr2=0x7f4b180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:47:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.468+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 shutdown_connections 2024-09-10T22:47:59.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:47:59.469+0000 7f4b1cca9640 1 -- 172.21.15.96:0/2618829385 wait complete. 2024-09-10T22:47:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:47:59 smithi096 ceph-mon[30869]: pgmap v368: 97 pgs: 97 active+clean; 951 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 851 op/s 2024-09-10T22:47:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:47:59 smithi121 ceph-mon[35119]: pgmap v368: 97 pgs: 97 active+clean; 951 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 851 op/s 2024-09-10T22:47:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:47:59 smithi117 ceph-mon[35081]: pgmap v368: 97 pgs: 97 active+clean; 951 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 851 op/s 2024-09-10T22:48:00.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:00 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2618829385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:00.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:48:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:00 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2618829385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:00 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2618829385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 25.0 s, 3630.0 tps, lat 27.466 ms stddev 27.101 2024-09-10T22:48:01.003 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:01.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.565+0000 7f1ba88dd640 1 -- 172.21.15.96:0/1253371870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410a3b0 msgr2=0x7f1ba410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.565+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/1253371870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410a3b0 0x7f1ba410a790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009860 tx=0x7f1b9402a340 comp rx=0 tx=0).stop 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.566+0000 7f1ba88dd640 1 -- 172.21.15.96:0/1253371870 shutdown_connections 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.566+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/1253371870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410f210 0x7f1ba410f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.566+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/1253371870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410fb90 0x7f1ba4116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.566+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/1253371870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410a3b0 0x7f1ba410a790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.566+0000 7f1ba88dd640 1 -- 172.21.15.96:0/1253371870 >> 172.21.15.96:0/1253371870 conn(0x7f1ba406e2e0 msgr2=0x7f1ba406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.569+0000 7f1ba88dd640 1 -- 172.21.15.96:0/1253371870 shutdown_connections 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.569+0000 7f1ba88dd640 1 -- 172.21.15.96:0/1253371870 wait complete. 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 Processor -- start 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 -- start start 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410f210 0x7f1ba41a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410fb90 0x7f1ba41a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba411a460 con 0x7f1ba410a3b0 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba411a2e0 con 0x7f1ba410fb90 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.571+0000 7f1ba88dd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba411a5e0 con 0x7f1ba410f210 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41510/0 (socket says 172.21.15.96:41510) 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 -- 172.21.15.96:0/2850701809 learned_addr learned my addr 172.21.15.96:0/2850701809 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410fb90 0x7f1ba41a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 -- 172.21.15.96:0/2850701809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410f210 msgr2=0x7f1ba41a21f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410f210 0x7f1ba41a21f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 -- 172.21.15.96:0/2850701809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410fb90 msgr2=0x7f1ba41a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410fb90 0x7f1ba41a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 -- 172.21.15.96:0/2850701809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41a4850 con 0x7f1ba410a3b0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.572+0000 7f1ba259c640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b94035040 tx=0x7f1b94032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.574+0000 7f1b937fe640 1 -- 172.21.15.96:0/2850701809 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94037040 con 0x7f1ba410a3b0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.574+0000 7f1b937fe640 1 -- 172.21.15.96:0/2850701809 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b94041070 con 0x7f1ba410a3b0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.574+0000 7f1b937fe640 1 -- 172.21.15.96:0/2850701809 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940035f0 con 0x7f1ba410a3b0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.574+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41a4ae0 con 0x7f1ba410a3b0 2024-09-10T22:48:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.575+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba411abc0 con 0x7f1ba410a3b0 2024-09-10T22:48:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.579+0000 7f1b937fe640 1 -- 172.21.15.96:0/2850701809 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f1b94004050 con 0x7f1ba410a3b0 2024-09-10T22:48:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.581+0000 7f1baacb2740 1 Processor -- start 2024-09-10T22:48:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.582+0000 7f1b937fe640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 0x7f1b8c0b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.582+0000 7f1baacb2740 1 -- start start 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.582+0000 7f1b937fe640 1 -- 172.21.15.96:0/2850701809 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1b94105570 con 0x7f1ba410a3b0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513075840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513078170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556512fdfea0 con 0x5565130694d0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556512fcf2d0 con 0x556513069df0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1baacb2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556512fd30e0 con 0x556513075d80 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 0x7f1b8c0b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41514/0 (socket says 172.21.15.96:41514) 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba1d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 -- 172.21.15.96:0/3250970720 learned_addr learned my addr 172.21.15.96:0/3250970720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513078170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 -- 172.21.15.96:0/3250970720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 msgr2=0x556513078170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513078170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 -- 172.21.15.96:0/3250970720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 msgr2=0x556513075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 -- 172.21.15.96:0/3250970720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556512d2d650 con 0x5565130694d0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 0x7f1b8c0b7a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005fd0 tx=0x7f1b9c00fab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.583+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c81a0 tx=0x7f1b940303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471aad0734457fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.584+0000 7f1b90ff9640 1 -- 172.21.15.96:0/3250970720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940d0020 con 0x5565130694d0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.584+0000 7f1b90ff9640 1 -- 172.21.15.96:0/3250970720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b940328f0 con 0x5565130694d0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.584+0000 7f1b90ff9640 1 -- 172.21.15.96:0/3250970720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94104040 con 0x5565130694d0 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.585+0000 7f1baacb2740 1 -- 172.21.15.96:0/3250970720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 msgr2=0x5565130698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.585+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c81a0 tx=0x7f1b940303b0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 -- 172.21.15.96:0/3250970720 shutdown_connections 2024-09-10T22:48:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513078170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3250970720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x5565130698b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 -- 172.21.15.96:0/3250970720 >> 172.21.15.96:0/3250970720 conn(0x556513067300 msgr2=0x5565130684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 -- 172.21.15.96:0/3250970720 shutdown_connections 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.586+0000 7f1baacb2740 1 -- 172.21.15.96:0/3250970720 wait complete. 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 -- start start 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 Processor -- start 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513069df0 0x556513072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 0x556513073220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556512e30700 con 0x556513075d80 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556512f6a160 con 0x5565130694d0 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1baacb2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565130787c0 con 0x556513069df0 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46438/0 (socket says 172.21.15.96:46438) 2024-09-10T22:48:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1ba259c640 1 -- 172.21.15.96:0/665285857 learned_addr learned my addr 172.21.15.96:0/665285857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 0x556513073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.587+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513069df0 0x556513072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1baacb2740 10 client.0 _ll_register_callbacks cb 0x556512f839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 -- 172.21.15.96:0/665285857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513069df0 msgr2=0x556513072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513069df0 0x556513072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 -- 172.21.15.96:0/665285857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 msgr2=0x556513073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 0x556513073220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 -- 172.21.15.96:0/665285857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556512f97c90 con 0x5565130694d0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 0x556513073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1ba259c640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c8390 tx=0x7f1b9402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc226d593ff2c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940e6060 con 0x5565130694d0 2024-09-10T22:48:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.588+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1b940d4ee0 con 0x5565130694d0 2024-09-10T22:48:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b940328f0 con 0x5565130694d0 2024-09-10T22:48:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b940eb050 con 0x5565130694d0 2024-09-10T22:48:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 10 client.25049 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556513084020 con 0x5565130694d0 2024-09-10T22:48:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b94030450 con 0x5565130694d0 2024-09-10T22:48:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1b940d01e0 con 0x5565130694d0 2024-09-10T22:48:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 10 client.25049 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 10 client.25049 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 20 client.25049 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.589+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x556513085370 0x556513087760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.590+0000 7f1baacb2740 4 client.25049 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.590+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b940328f0 con 0x556513085370 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.590+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x556513085370 0x556513087760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.590+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x556513085370 0x556513087760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78365cc0112ed30d server_cookie=6d9a6ecdb558e892 in_seq=0 out_seq=0 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.590+0000 7f1b89ffb640 10 client.25049 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1b89ffb640 1 -- 172.21.15.96:0/665285857 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1b940328f0 con 0x556513085370 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1b89ffb640 10 client.25049 handle_command_reply: tid=0 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 client.25049 shutdown 2024-09-10T22:48:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 msgr2=0x556513075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c8390 tx=0x7f1b9402a340 comp rx=0 tx=0).stop 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 shutdown_connections 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x556513085370 0x556513087760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556513075d80 0x556513073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513069df0 0x556513072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 --2- 172.21.15.96:0/665285857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565130694d0 0x556513075700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 >> 172.21.15.96:0/665285857 conn(0x556513067300 msgr2=0x55651306cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1b89ffb640 0 client.25049 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 shutdown_connections 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 1 -- 172.21.15.96:0/665285857 wait complete. 2024-09-10T22:48:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.591+0000 7f1baacb2740 20 client.25049 trim_cache size 0 max 16384 2024-09-10T22:48:01.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 Processor -- start 2024-09-10T22:48:01.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 -- start start 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513066c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513074890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bd0a0 con 0x5565130694d0 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41beca0 con 0x556513069df0 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1baacb2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bd770 con 0x556513075d80 2024-09-10T22:48:01.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41536/0 (socket says 172.21.15.96:41536) 2024-09-10T22:48:01.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1ba259c640 1 -- 172.21.15.96:0/3761981821 learned_addr learned my addr 172.21.15.96:0/3761981821 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513066c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.609+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.610+0000 7f1ba259c640 1 -- 172.21.15.96:0/3761981821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 msgr2=0x556513074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.610+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.610+0000 7f1ba259c640 1 -- 172.21.15.96:0/3761981821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 msgr2=0x556513066c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.610+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513066c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.610+0000 7f1ba259c640 1 -- 172.21.15.96:0/3761981821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556513075330 con 0x5565130694d0 2024-09-10T22:48:01.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.611+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c8e40 tx=0x7f1b940303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73bb0df62c6fcc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.611+0000 7f1b8bfff640 1 -- 172.21.15.96:0/3761981821 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94104050 con 0x5565130694d0 2024-09-10T22:48:01.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.611+0000 7f1b8bfff640 1 -- 172.21.15.96:0/3761981821 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b940328f0 con 0x5565130694d0 2024-09-10T22:48:01.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.612+0000 7f1b8bfff640 1 -- 172.21.15.96:0/3761981821 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940d0040 con 0x5565130694d0 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.612+0000 7f1baacb2740 1 -- 172.21.15.96:0/3761981821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 msgr2=0x556513068210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.612+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b940c8e40 tx=0x7f1b940303b0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 -- 172.21.15.96:0/3761981821 shutdown_connections 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x556513066c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x556513074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3761981821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x556513068210 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 -- 172.21.15.96:0/3761981821 >> 172.21.15.96:0/3761981821 conn(0x556513067300 msgr2=0x556513074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 -- 172.21.15.96:0/3761981821 shutdown_connections 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 -- 172.21.15.96:0/3761981821 wait complete. 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.613+0000 7f1baacb2740 1 -- start start 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 Processor -- start 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x55651307e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x5565130788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41be510 con 0x5565130694d0 2024-09-10T22:48:01.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bde40 con 0x556513069df0 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41bb0e0 con 0x556513075d80 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41544/0 (socket says 172.21.15.96:41544) 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba259c640 1 -- 172.21.15.96:0/3991909388 learned_addr learned my addr 172.21.15.96:0/3991909388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x55651307e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1baacb2740 10 client.0 _ll_register_callbacks cb 0x7f1b98009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:01.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba2d9d640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x5565130788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 -- 172.21.15.96:0/3991909388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 msgr2=0x5565130788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x5565130788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 -- 172.21.15.96:0/3991909388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 msgr2=0x55651307db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba1d9b640 1 -- 172.21.15.96:0/3991909388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556513078ec0 con 0x556513069df0 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.614+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1ba1d9b640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x55651307e040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556513087e50 tx=0x7f1b98010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9aad682973b050 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1b89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98010e60 con 0x556513069df0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b940328f0 con 0x556513069df0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1b98004030 con 0x556513069df0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b98004230 con 0x556513069df0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.615+0000 7f1baacb2740 10 client.25055 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565130f2a60 con 0x556513069df0 2024-09-10T22:48:01.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b940328f0 con 0x556513069df0 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1b9802c050 con 0x556513069df0 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 10 client.25055 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 10 client.25055 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 20 client.25055 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5565130f3130 0x5565130f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 4 client.25055 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:01.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.616+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556512fe1df0 con 0x5565130f3130 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.617+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5565130f3130 0x5565130f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.617+0000 7f1b89ffb640 10 client.25055 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.617+0000 7f1ba259c640 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5565130f3130 0x5565130f5520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1dbb916f13a478e2 server_cookie=b3dcc832f846bd6d in_seq=0 out_seq=0 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1b89ffb640 1 -- 172.21.15.96:0/3991909388 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556512fe1df0 con 0x5565130f3130 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1b89ffb640 10 client.25055 handle_command_reply: tid=0 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 client.25055 shutdown 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 msgr2=0x55651307e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x55651307e040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556513087e50 tx=0x7f1b98010800 comp rx=0 tx=0).stop 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 shutdown_connections 2024-09-10T22:48:01.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5565130f3130 0x5565130f5520 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556513069df0 0x55651307e040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556513075d80 0x5565130788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 --2- 172.21.15.96:0/3991909388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565130694d0 0x55651307db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1b89ffb640 0 client.25055 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 >> 172.21.15.96:0/3991909388 conn(0x556513067300 msgr2=0x5565130749a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 shutdown_connections 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 1 -- 172.21.15.96:0/3991909388 wait complete. 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.618+0000 7f1baacb2740 20 client.25055 trim_cache size 0 max 16384 2024-09-10T22:48:01.630 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:01.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 msgr2=0x7f1b8c0b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 0x7f1b8c0b7a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c005fd0 tx=0x7f1b9c00fab0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 msgr2=0x7f1ba41a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b94035040 tx=0x7f1b94032d50 comp rx=0 tx=0).stop 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 shutdown_connections 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ba410fb90 0x7f1ba41a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1b8c0b5540 0x7f1b8c0b7a00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ba410f210 0x7f1ba41a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 --2- 172.21.15.96:0/2850701809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba410a3b0 0x7f1ba41a1cb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 >> 172.21.15.96:0/2850701809 conn(0x7f1ba406e2e0 msgr2=0x7f1ba4072620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 shutdown_connections 2024-09-10T22:48:01.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:01.619+0000 7f1ba88dd640 1 -- 172.21.15.96:0/2850701809 wait complete. 2024-09-10T22:48:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 -- 172.21.15.96:0/631735118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 msgr2=0x7fdb30107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 --2- 172.21.15.96:0/631735118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb30107f40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009900 tx=0x7fdb2402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 -- 172.21.15.96:0/631735118 shutdown_connections 2024-09-10T22:48:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 --2- 172.21.15.96:0/631735118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb30108680 0x7fdb3010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 --2- 172.21.15.96:0/631735118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb30107f40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 --2- 172.21.15.96:0/631735118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb3006b3d0 0x7fdb301055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 -- 172.21.15.96:0/631735118 >> 172.21.15.96:0/631735118 conn(0x7fdb300fd140 msgr2=0x7fdb300ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 -- 172.21.15.96:0/631735118 shutdown_connections 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.874+0000 7fdb36033640 1 -- 172.21.15.96:0/631735118 wait complete. 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 Processor -- start 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 -- start start 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 0x7fdb3019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 0x7fdb301a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30102ac0 con 0x7fdb30105b10 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30102940 con 0x7fdb3006b3d0 2024-09-10T22:48:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb36033640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30102c40 con 0x7fdb30108680 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb2ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 0x7fdb301a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41568/0 (socket says 172.21.15.96:41568) 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb2effd640 1 -- 172.21.15.96:0/3913260915 learned_addr learned my addr 172.21.15.96:0/3913260915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.875+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 0x7fdb3019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 -- 172.21.15.96:0/3913260915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 msgr2=0x7fdb3019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 0x7fdb3019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 -- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 msgr2=0x7fdb3019f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 -- 172.21.15.96:0/3913260915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301a0e90 con 0x7fdb30108680 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 0x7fdb3019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2effd640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2ffff640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 0x7fdb301a05d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800cab0 tx=0x7fdb2800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2cff9640 1 -- 172.21.15.96:0/3913260915 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2800ae20 con 0x7fdb30108680 2024-09-10T22:48:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2cff9640 1 -- 172.21.15.96:0/3913260915 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2800f900 con 0x7fdb30108680 2024-09-10T22:48:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb2cff9640 1 -- 172.21.15.96:0/3913260915 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb280189f0 con 0x7fdb30108680 2024-09-10T22:48:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb301a1180 con 0x7fdb30108680 2024-09-10T22:48:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.876+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301a7850 con 0x7fdb30108680 2024-09-10T22:48:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.880+0000 7fdb2cff9640 1 -- 172.21.15.96:0/3913260915 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fdb2800f450 con 0x7fdb30108680 2024-09-10T22:48:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 Processor -- start 2024-09-10T22:48:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 -- start start 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c589c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c58f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b25c4f96f0 con 0x55b25c58a190 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b25c4e8b20 con 0x55b25c587860 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb38e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b25c4ec930 con 0x55b25c58cbd0 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2cff9640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 0x7fdb180b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c589c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41570/0 (socket says 172.21.15.96:41570) 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2f7fe640 1 -- 172.21.15.96:0/1091139904 learned_addr learned my addr 172.21.15.96:0/1091139904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c58f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2cff9640 1 -- 172.21.15.96:0/3913260915 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb280e63d0 con 0x7fdb30108680 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.881+0000 7fdb2ffff640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 0x7fdb180b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 -- 172.21.15.96:0/1091139904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 msgr2=0x55b25c58f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c58f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 -- 172.21.15.96:0/1091139904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 msgr2=0x55b25c58c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 -- 172.21.15.96:0/1091139904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25c4ae530 con 0x55b25c587860 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c589c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb2402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e718822cd7b6258b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb2ffff640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 0x7fdb180b7970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800ca80 tx=0x7fdb280e6620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.882+0000 7fdb15ffb640 1 -- 172.21.15.96:0/1091139904 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24035030 con 0x55b25c587860 2024-09-10T22:48:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.883+0000 7fdb15ffb640 1 -- 172.21.15.96:0/1091139904 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2403d900 con 0x55b25c587860 2024-09-10T22:48:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.883+0000 7fdb15ffb640 1 -- 172.21.15.96:0/1091139904 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2403c730 con 0x55b25c587860 2024-09-10T22:48:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.883+0000 7fdb38e05740 1 -- 172.21.15.96:0/1091139904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 msgr2=0x55b25c589c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.883+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c589c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb2402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 -- 172.21.15.96:0/1091139904 shutdown_connections 2024-09-10T22:48:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c58f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c58c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1091139904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c589c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 -- 172.21.15.96:0/1091139904 >> 172.21.15.96:0/1091139904 conn(0x55b25c5829b0 msgr2=0x55b25c583740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 -- 172.21.15.96:0/1091139904 shutdown_connections 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 -- 172.21.15.96:0/1091139904 wait complete. 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 -- start start 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 Processor -- start 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.884+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c590e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c592100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b25c420ab0 con 0x55b25c58a190 2024-09-10T22:48:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b25c48e730 con 0x55b25c587860 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b25c58fb50 con 0x55b25c58cbd0 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 10 client.0 _ll_register_callbacks cb 0x55b25c4e26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c590e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41584/0 (socket says 172.21.15.96:41584) 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c590e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46510/0 (socket says 172.21.15.96:46510) 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 -- 172.21.15.96:0/4174173531 learned_addr learned my addr 172.21.15.96:0/4174173531 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2effd640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c592100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 -- 172.21.15.96:0/4174173531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 msgr2=0x55b25c592100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c592100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 -- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 msgr2=0x55b25c590e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c590e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 -- 172.21.15.96:0/4174173531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25c24a790 con 0x55b25c58a190 2024-09-10T22:48:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdb20012d10 tx=0x7fdb2000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26251f08059cbabd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb0effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200120d0 con 0x55b25c58a190 2024-09-10T22:48:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb20012e00 con 0x55b25c58a190 2024-09-10T22:48:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2403d900 con 0x55b25c58a190 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb20012270 con 0x55b25c58a190 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 10 client.15501 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.885+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b25c2c3620 con 0x55b25c58a190 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2403d900 con 0x55b25c58a190 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdb2001c750 con 0x55b25c58a190 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 20 client.15501 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c59ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 4 client.15501 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b25c4fb640 con 0x55b25c59c8e0 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.886+0000 7fdb2effd640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c59ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.887+0000 7fdb2effd640 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c59ed20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c2c66a5dd698d5 server_cookie=3d57d6c4ae4ba62b in_seq=0 out_seq=0 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.887+0000 7fdb0effd640 10 client.15501 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.887+0000 7fdb0effd640 1 -- 172.21.15.96:0/4174173531 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b25c4fb640 con 0x55b25c59c8e0 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.887+0000 7fdb0effd640 10 client.15501 handle_command_reply: tid=0 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 client.15501 shutdown 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 msgr2=0x55b25c591530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdb20012d10 tx=0x7fdb2000f790 comp rx=0 tx=0).stop 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 shutdown_connections 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c59ed20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb0effd640 0 client.15501 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58cbd0 0x55b25c592100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c591530 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 --2- 172.21.15.96:0/4174173531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c587860 0x55b25c590e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 >> 172.21.15.96:0/4174173531 conn(0x55b25c5829b0 msgr2=0x55b25c581bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 shutdown_connections 2024-09-10T22:48:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.888+0000 7fdb38e05740 1 -- 172.21.15.96:0/4174173531 wait complete. 2024-09-10T22:48:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.889+0000 7fdb38e05740 20 client.15501 trim_cache size 0 max 16384 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.908+0000 7fdb38e05740 1 Processor -- start 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.908+0000 7fdb38e05740 1 -- start start 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 0x55b25c586cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c587200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00e330 con 0x55b25c587860 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00ff30 con 0x55b25c58cbd0 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb38e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00ea00 con 0x55b25c58a190 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41598/0 (socket says 172.21.15.96:41598) 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 -- 172.21.15.96:0/977437250 learned_addr learned my addr 172.21.15.96:0/977437250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 0x55b25c586cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c587200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 -- 172.21.15.96:0/977437250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 msgr2=0x55b25c586cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 0x55b25c586cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 -- 172.21.15.96:0/977437250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 msgr2=0x55b25c587200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c587200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 -- 172.21.15.96:0/977437250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25c5926e0 con 0x55b25c587860 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 0x55b25c586cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb2effd640 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdb240058c0 tx=0x7fdb24005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1084ddfd44013660 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.909+0000 7fdb157fa640 1 -- 172.21.15.96:0/977437250 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24035040 con 0x55b25c587860 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.910+0000 7fdb157fa640 1 -- 172.21.15.96:0/977437250 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2403db40 con 0x55b25c587860 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.910+0000 7fdb157fa640 1 -- 172.21.15.96:0/977437250 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb240319d0 con 0x55b25c587860 2024-09-10T22:48:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.910+0000 7fdb38e05740 1 -- 172.21.15.96:0/977437250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 msgr2=0x55b25c59ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.910+0000 7fdb38e05740 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdb240058c0 tx=0x7fdb24005850 comp rx=0 tx=0).stop 2024-09-10T22:48:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.911+0000 7fdb38e05740 1 -- 172.21.15.96:0/977437250 shutdown_connections 2024-09-10T22:48:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.911+0000 7fdb38e05740 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c587200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.911+0000 7fdb38e05740 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c58a190 0x55b25c586cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.911+0000 7fdb38e05740 1 --2- 172.21.15.96:0/977437250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c587860 0x55b25c59ed00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.911+0000 7fdb38e05740 1 -- 172.21.15.96:0/977437250 >> 172.21.15.96:0/977437250 conn(0x55b25c5829b0 msgr2=0x55b25c57daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.912+0000 7fdb38e05740 1 -- 172.21.15.96:0/977437250 shutdown_connections 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.912+0000 7fdb38e05740 1 -- 172.21.15.96:0/977437250 wait complete. 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.912+0000 7fdb38e05740 1 -- start start 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.912+0000 7fdb38e05740 1 Processor -- start 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c598a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c607820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00f7a0 con 0x55b25c58a190 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00f0d0 con 0x55b25c58cbd0 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c00c310 con 0x55b25c587860 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb38e05740 10 client.0 _ll_register_callbacks cb 0x7fdb20009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb2effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40892/0 (socket says 172.21.15.96:40892) 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.913+0000 7fdb2effd640 1 -- 172.21.15.96:0/1407941875 learned_addr learned my addr 172.21.15.96:0/1407941875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c598a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c607820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 -- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 msgr2=0x55b25c607820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c607820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 -- 172.21.15.96:0/1407941875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 msgr2=0x55b25c598a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c598a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 -- 172.21.15.96:0/1407941875 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25c607fd0 con 0x55b25c587860 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb2effd640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdb24003670 tx=0x7fdb2402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c69ad71aa4f92e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb0effd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24035040 con 0x55b25c587860 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb2403dcf0 con 0x55b25c587860 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.914+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2403db40 con 0x55b25c587860 2024-09-10T22:48:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb2403d5b0 con 0x55b25c587860 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 10 client.25090 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b25c6092e0 con 0x55b25c587860 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2403db40 con 0x55b25c587860 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdb24068050 con 0x55b25c587860 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 10 client.25090 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 10 client.25090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 20 client.25090 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c60b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 4 client.25090 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b25c4f70f0 con 0x55b25c59c8e0 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.915+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c60b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.916+0000 7fdb2f7fe640 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c60b820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21de2fc27caafbc3 server_cookie=79a5fd521a9e3660 in_seq=0 out_seq=0 2024-09-10T22:48:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.916+0000 7fdb0effd640 10 client.25090 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb0effd640 1 -- 172.21.15.96:0/1407941875 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b25c4f70f0 con 0x55b25c59c8e0 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb0effd640 10 client.25090 handle_command_reply: tid=0 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 client.25090 shutdown 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 msgr2=0x55b25c5983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdb24003670 tx=0x7fdb2402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 shutdown_connections 2024-09-10T22:48:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b25c59c8e0 0x55b25c60b820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b25c58cbd0 0x55b25c607820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb0effd640 0 client.25090 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b25c58a190 0x55b25c598a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 --2- 172.21.15.96:0/1407941875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b25c587860 0x55b25c5983d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 >> 172.21.15.96:0/1407941875 conn(0x55b25c5829b0 msgr2=0x55b25c58ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 shutdown_connections 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.932+0000 7fdb38e05740 1 -- 172.21.15.96:0/1407941875 wait complete. 2024-09-10T22:48:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.933+0000 7fdb38e05740 20 client.25090 trim_cache size 0 max 16384 2024-09-10T22:48:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:48:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:48:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:48:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 msgr2=0x7fdb180b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 0x7fdb180b7970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800ca80 tx=0x7fdb280e6620 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 msgr2=0x7fdb301a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 0x7fdb301a05d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800cab0 tx=0x7fdb2800cf70 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 shutdown_connections 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb180b54b0 0x7fdb180b7970 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb30108680 0x7fdb301a05d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb30105b10 0x7fdb3019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 --2- 172.21.15.96:0/3913260915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3006b3d0 0x7fdb3019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 >> 172.21.15.96:0/3913260915 conn(0x7fdb300fd140 msgr2=0x7fdb30109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 shutdown_connections 2024-09-10T22:48:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:01.934+0000 7fdb36033640 1 -- 172.21.15.96:0/3913260915 wait complete. 2024-09-10T22:48:02.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:02 smithi096 ceph-mon[30869]: pgmap v369: 97 pgs: 97 active+clean; 953 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-09-10T22:48:02.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:48:02.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:02 smithi121 ceph-mon[35119]: pgmap v369: 97 pgs: 97 active+clean; 953 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-09-10T22:48:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:02 smithi117 ceph-mon[35081]: pgmap v369: 97 pgs: 97 active+clean; 953 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 MiB/s wr, 739 op/s 2024-09-10T22:48:02.819 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 -- 172.21.15.96:0/3375265029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd81093d0 msgr2=0x7f9bd81097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3375265029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd81093d0 0x7f9bd81097b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009900 tx=0x7f9bc802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 -- 172.21.15.96:0/3375265029 shutdown_connections 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3375265029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd810f760 0x7f9bd8115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3375265029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810ede0 0x7f9bd810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3375265029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd81093d0 0x7f9bd81097b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 -- 172.21.15.96:0/3375265029 >> 172.21.15.96:0/3375265029 conn(0x7f9bd8078a40 msgr2=0x7f9bd807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 -- 172.21.15.96:0/3375265029 shutdown_connections 2024-09-10T22:48:03.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.366+0000 7f9bdd026640 1 -- 172.21.15.96:0/3375265029 wait complete. 2024-09-10T22:48:03.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.368+0000 7f9bdd026640 1 Processor -- start 2024-09-10T22:48:03.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.368+0000 7f9bdd026640 1 -- start start 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd810ede0 0x7f9bd819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46548/0 (socket says 172.21.15.96:46548) 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 0x7f9bd81a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/3525068670 learned_addr learned my addr 172.21.15.96:0/3525068670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8119390 con 0x7f9bd810ede0 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bd659c640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd810ede0 0x7f9bd819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8119210 con 0x7f9bd81093d0 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 0x7f9bd81a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.369+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8119510 con 0x7f9bd810f760 2024-09-10T22:48:03.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 -- 172.21.15.96:0/3525068670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 msgr2=0x7f9bd819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 -- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd810ede0 msgr2=0x7f9bd819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd810ede0 0x7f9bd819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd81a5950 con 0x7f9bd810f760 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.370+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 0x7f9bd81a5180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e970 tx=0x7f9bc000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.371+0000 7f9bcffff640 1 -- 172.21.15.96:0/3525068670 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000ccc0 con 0x7f9bd810f760 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.371+0000 7f9bcffff640 1 -- 172.21.15.96:0/3525068670 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0005020 con 0x7f9bd810f760 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.371+0000 7f9bcffff640 1 -- 172.21.15.96:0/3525068670 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0004a00 con 0x7f9bd810f760 2024-09-10T22:48:03.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.371+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81a5c40 con 0x7f9bd810f760 2024-09-10T22:48:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.371+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81ac300 con 0x7f9bd810f760 2024-09-10T22:48:03.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.374+0000 7f9bcffff640 1 -- 172.21.15.96:0/3525068670 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f9bc0004ba0 con 0x7f9bd810f760 2024-09-10T22:48:03.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 Processor -- start 2024-09-10T22:48:03.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 -- start start 2024-09-10T22:48:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a7f384cea0 con 0x55a7f38d6a10 2024-09-10T22:48:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a7f383c2d0 con 0x55a7f38e29a0 2024-09-10T22:48:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.375+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a7f38400e0 con 0x55a7f38d60f0 2024-09-10T22:48:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41626/0 (socket says 172.21.15.96:41626) 2024-09-10T22:48:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/3755633492 learned_addr learned my addr 172.21.15.96:0/3755633492 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd659c640 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd659c640 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 0x7f9bc40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/3755633492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 msgr2=0x55a7f38d64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38d64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/3755633492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 msgr2=0x55a7f38e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.376+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/3755633492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f359a650 con 0x55a7f38d6a10 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bcffff640 1 -- 172.21.15.96:0/3525068670 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bc00e5b50 con 0x7f9bd810f760 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8000c00 tx=0x7f9bc8032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9585232459b6c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 0x7f9bc40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/3755633492 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8035040 con 0x55a7f38d6a10 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/3755633492 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc80317a0 con 0x55a7f38d6a10 2024-09-10T22:48:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.377+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/3755633492 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80047a0 con 0x55a7f38d6a10 2024-09-10T22:48:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/3755633492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 msgr2=0x55a7f38e2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bd759e640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 0x7f9bc40b78b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000cfd0 tx=0x7f9bc0009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8000c00 tx=0x7f9bc8032ed0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/3755633492 shutdown_connections 2024-09-10T22:48:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38e2460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/3755633492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.378+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/3755633492 >> 172.21.15.96:0/3755633492 conn(0x55a7f38d3f20 msgr2=0x55a7f38d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/3755633492 shutdown_connections 2024-09-10T22:48:03.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/3755633492 wait complete. 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- start start 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 Processor -- start 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38dfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a7f369d700 con 0x55a7f38d6a10 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a7f37d7160 con 0x55a7f38e29a0 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.380+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a7f38e55c0 con 0x55a7f38d60f0 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bdf3fb740 10 client.0 _ll_register_callbacks cb 0x55a7f37f09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41640/0 (socket says 172.21.15.96:41640) 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd6d9d640 1 -- 172.21.15.96:0/934236813 learned_addr learned my addr 172.21.15.96:0/934236813 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40918/0 (socket says 172.21.15.96:40918) 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38dfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 -- 172.21.15.96:0/934236813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 msgr2=0x55a7f38dfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38dfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 -- 172.21.15.96:0/934236813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 msgr2=0x55a7f38df760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 -- 172.21.15.96:0/934236813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f3804c90 con 0x55a7f38d60f0 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bd659c640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005fd0 tx=0x7f9bd0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53113804d2a61c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:03.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd00130d0 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bd0004b20 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.381+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80317a0 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9bd0013270 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 10 client.25102 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7f38f0f70 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc80317a0 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9bd0022040 con 0x55a7f38d60f0 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 10 client.25102 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 10 client.25102 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 20 client.25102 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f38f4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 4 client.25102 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f384edf0 con 0x55a7f38f2360 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.382+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f38f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.383+0000 7f9bbe7fc640 10 client.25102 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.383+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f38f4780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86ec980c12346d69 server_cookie=ac82ed0a5d27a2ae in_seq=0 out_seq=0 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.383+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/934236813 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7f384edf0 con 0x55a7f38f2360 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.383+0000 7f9bbe7fc640 10 client.25102 handle_command_reply: tid=0 2024-09-10T22:48:03.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.383+0000 7f9bdf3fb740 1 client.25102 shutdown 2024-09-10T22:48:03.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 msgr2=0x55a7f38e2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005fd0 tx=0x7f9bd0009ed0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 shutdown_connections 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f38f4780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bbe7fc640 0 client.25102 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38dfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/934236813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38e2310 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.384+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 >> 172.21.15.96:0/934236813 conn(0x55a7f38d3f20 msgr2=0x55a7f38d9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.385+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 shutdown_connections 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.385+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/934236813 wait complete. 2024-09-10T22:48:03.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.385+0000 7f9bdf3fb740 20 client.25102 trim_cache size 0 max 16384 2024-09-10T22:48:03.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.403+0000 7f9bdf3fb740 1 Processor -- start 2024-09-10T22:48:03.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.403+0000 7f9bdf3fb740 1 -- start start 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b4850 con 0x55a7f38d6a10 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b6450 con 0x55a7f38e29a0 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b4f20 con 0x55a7f38d60f0 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40922/0 (socket says 172.21.15.96:40922) 2024-09-10T22:48:03.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bd659c640 1 -- 172.21.15.96:0/580801600 learned_addr learned my addr 172.21.15.96:0/580801600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:03.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.404+0000 7f9bd659c640 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 -- 172.21.15.96:0/580801600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 msgr2=0x55a7f38e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 -- 172.21.15.96:0/580801600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 msgr2=0x55a7f38d3760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:03.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38d3760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 -- 172.21.15.96:0/580801600 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f38d6840 con 0x55a7f38d60f0 2024-09-10T22:48:03.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.405+0000 7f9bd659c640 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009a70 tx=0x7f9bd0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c1d87767cf158e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.406+0000 7f9bccff9640 1 -- 172.21.15.96:0/580801600 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0013410 con 0x55a7f38d60f0 2024-09-10T22:48:03.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.406+0000 7f9bccff9640 1 -- 172.21.15.96:0/580801600 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd00092a0 con 0x55a7f38d60f0 2024-09-10T22:48:03.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.407+0000 7f9bccff9640 1 -- 172.21.15.96:0/580801600 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000f3c0 con 0x55a7f38d60f0 2024-09-10T22:48:03.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.407+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/580801600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 msgr2=0x55a7f38ee7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009a70 tx=0x7f9bd0005fd0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/580801600 shutdown_connections 2024-09-10T22:48:03.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38e29a0 0x55a7f38e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38d6a10 0x55a7f38d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/580801600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38ee7f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/580801600 >> 172.21.15.96:0/580801600 conn(0x55a7f38d3f20 msgr2=0x55a7f38d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/580801600 shutdown_connections 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/580801600 wait complete. 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 -- start start 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.408+0000 7f9bdf3fb740 1 Processor -- start 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38d6a10 0x55a7f38f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38e29a0 0x55a7f38fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b5cc0 con 0x55a7f38e29a0 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b55f0 con 0x55a7f38d6a10 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81b2870 con 0x55a7f38d60f0 2024-09-10T22:48:03.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bdf3fb740 10 client.0 _ll_register_callbacks cb 0x7f9bd000e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:03.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40936/0 (socket says 172.21.15.96:40936) 2024-09-10T22:48:03.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.409+0000 7f9bd659c640 1 -- 172.21.15.96:0/1546111603 learned_addr learned my addr 172.21.15.96:0/1546111603 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:03.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38e29a0 0x55a7f38fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38d6a10 0x55a7f38f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 -- 172.21.15.96:0/1546111603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38d6a10 msgr2=0x55a7f38f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38d6a10 0x55a7f38f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 -- 172.21.15.96:0/1546111603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38e29a0 msgr2=0x55a7f38fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38e29a0 0x55a7f38fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 -- 172.21.15.96:0/1546111603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f38faa30 con 0x55a7f38d60f0 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bd659c640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0002230 tx=0x7f9bd0013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197caad52974fe03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.410+0000 7f9bbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:03.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0038030 con 0x55a7f38d60f0 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd00092a0 con 0x55a7f38d60f0 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bd0022b20 con 0x55a7f38d60f0 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9bd0020e90 con 0x55a7f38d60f0 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bdf3fb740 10 client.25108 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.411+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7f395f310 con 0x55a7f38d60f0 2024-09-10T22:48:03.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd00092a0 con 0x55a7f38d60f0 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9bd0022040 con 0x55a7f38d60f0 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 10 client.25108 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 10 client.25108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 20 client.25108 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f3961980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 4 client.25108 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7f384a8a0 con 0x55a7f38f2360 2024-09-10T22:48:03.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.412+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f3961980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.413+0000 7f9bd6d9d640 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f3961980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=823bfa10564ec9c1 server_cookie=87a0ce39fe05a16e in_seq=0 out_seq=0 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.413+0000 7f9bbe7fc640 10 client.25108 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.413+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1546111603 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7f384a8a0 con 0x55a7f38f2360 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.413+0000 7f9bbe7fc640 10 client.25108 handle_command_reply: tid=0 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.413+0000 7f9bdf3fb740 1 client.25108 shutdown 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 msgr2=0x55a7f38eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0002230 tx=0x7f9bd0013b70 comp rx=0 tx=0).stop 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 shutdown_connections 2024-09-10T22:48:03.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55a7f38f2360 0x55a7f3961980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7f38d6a10 0x55a7f38f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7f38e29a0 0x55a7f38fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 --2- 172.21.15.96:0/1546111603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7f38d60f0 0x55a7f38eb060 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0002230 tx=0x7f9bd0013b70 comp rx=0 tx=0).stop 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 >> 172.21.15.96:0/1546111603 conn(0x55a7f38d3f20 msgr2=0x55a7f38e0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.414+0000 7f9bbe7fc640 0 client.25108 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:03.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.415+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 shutdown_connections 2024-09-10T22:48:03.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.415+0000 7f9bdf3fb740 1 -- 172.21.15.96:0/1546111603 wait complete. 2024-09-10T22:48:03.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.415+0000 7f9bdf3fb740 20 client.25108 trim_cache size 0 max 16384 2024-09-10T22:48:03.418 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 msgr2=0x7f9bc40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 0x7f9bc40b78b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000cfd0 tx=0x7f9bc0009620 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 msgr2=0x7f9bd81a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 0x7f9bd81a5180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e970 tx=0x7f9bc000ee30 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 shutdown_connections 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53f0 0x7f9bc40b78b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd810f760 0x7f9bd81a5180 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd810ede0 0x7f9bd819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 --2- 172.21.15.96:0/3525068670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd81093d0 0x7f9bd819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 >> 172.21.15.96:0/3525068670 conn(0x7f9bd8078a40 msgr2=0x7f9bd807ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:03.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 shutdown_connections 2024-09-10T22:48:03.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:03.416+0000 7f9bdd026640 1 -- 172.21.15.96:0/3525068670 wait complete. 2024-09-10T22:48:04.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:48:04.493 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:04.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:04 smithi096 ceph-mon[30869]: pgmap v370: 97 pgs: 97 active+clean; 966 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 787 op/s 2024-09-10T22:48:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:04 smithi121 ceph-mon[35119]: pgmap v370: 97 pgs: 97 active+clean; 966 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 787 op/s 2024-09-10T22:48:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:04 smithi117 ceph-mon[35081]: pgmap v370: 97 pgs: 97 active+clean; 966 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 787 op/s 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 -- 172.21.15.96:0/1144843226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076a0 msgr2=0x7f1fa8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/1144843226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076a0 0x7f1fa8107ae0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009900 tx=0x7f1f9802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 -- 172.21.15.96:0/1144843226 shutdown_connections 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/1144843226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108020 0x7f1fa810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/1144843226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076a0 0x7f1fa8107ae0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/1144843226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101c90 0x7f1fa8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 -- 172.21.15.96:0/1144843226 >> 172.21.15.96:0/1144843226 conn(0x7f1fa80fd700 msgr2=0x7f1fa80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 -- 172.21.15.96:0/1144843226 shutdown_connections 2024-09-10T22:48:05.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.012+0000 7f1fb07c1640 1 -- 172.21.15.96:0/1144843226 wait complete. 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 Processor -- start 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 -- start start 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 0x7f1fa819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 0x7f1fa819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 0x7f1fa81a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8119430 con 0x7f1fa8101c90 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81192b0 con 0x7f1fa81076a0 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fb07c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81195b0 con 0x7f1fa8108020 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.013+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 0x7f1fa819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 0x7f1fa819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 0x7f1fa81a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 0x7f1fa819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41672/0 (socket says 172.21.15.96:41672) 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fae55d640 1 -- 172.21.15.96:0/714838687 learned_addr learned my addr 172.21.15.96:0/714838687 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 -- 172.21.15.96:0/714838687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 msgr2=0x7f1fa819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 0x7f1fa819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 -- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 msgr2=0x7f1fa819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 0x7f1fa819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 -- 172.21.15.96:0/714838687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa81a5950 con 0x7f1fa8108020 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 0x7f1fa819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1faed5e640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 0x7f1fa81a5180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000da10 tx=0x7f1fa000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/714838687 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000bd70 con 0x7f1fa8108020 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/714838687 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa0003e20 con 0x7f1fa8108020 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/714838687 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00106c0 con 0x7f1fa8108020 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa81a5c40 con 0x7f1fa8108020 2024-09-10T22:48:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.014+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81ac300 con 0x7f1fa8108020 2024-09-10T22:48:05.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.018+0000 7f1fb2b96740 1 Processor -- start 2024-09-10T22:48:05.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.018+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/714838687 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f1fa0024060 con 0x7f1fa8108020 2024-09-10T22:48:05.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 -- start start 2024-09-10T22:48:05.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 0x55b182934840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182934d80 0x55b182937170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b18289eea0 con 0x55b182928df0 2024-09-10T22:48:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b18288e2d0 con 0x55b1829284d0 2024-09-10T22:48:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b1828920e0 con 0x55b182934d80 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46606/0 (socket says 172.21.15.96:46606) 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fadd5c640 1 -- 172.21.15.96:0/1606308315 learned_addr learned my addr 172.21.15.96:0/1606308315 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.019+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182934d80 0x55b182937170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.020+0000 7f1f9f7fe640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 0x7f1f940b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.020+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/714838687 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1fa00e5db0 con 0x7f1fa8108020 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fae55d640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 0x55b182934840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1faed5e640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 0x7f1f940b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fadd5c640 1 -- 172.21.15.96:0/1606308315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182934d80 msgr2=0x55b182937170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182934d80 0x55b182937170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fadd5c640 1 -- 172.21.15.96:0/1606308315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 msgr2=0x55b182934840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 0x55b182934840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fadd5c640 1 -- 172.21.15.96:0/1606308315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1825ec650 con 0x55b1829284d0 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1faed5e640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 0x7f1f940b7810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0005fd0 tx=0x7f1fa0004950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.021+0000 7f1fae55d640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 0x55b182934840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.022+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f98005e00 tx=0x7f1f98005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87187f789653e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.022+0000 7f1f9cff9640 1 -- 172.21.15.96:0/1606308315 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98035030 con 0x55b1829284d0 2024-09-10T22:48:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.022+0000 7f1f9cff9640 1 -- 172.21.15.96:0/1606308315 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f98003d40 con 0x55b1829284d0 2024-09-10T22:48:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.022+0000 7f1f9cff9640 1 -- 172.21.15.96:0/1606308315 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98004ca0 con 0x55b1829284d0 2024-09-10T22:48:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.023+0000 7f1fb2b96740 1 -- 172.21.15.96:0/1606308315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 msgr2=0x55b1829288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.023+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f98005e00 tx=0x7f1f98005d90 comp rx=0 tx=0).stop 2024-09-10T22:48:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.024+0000 7f1fb2b96740 1 -- 172.21.15.96:0/1606308315 shutdown_connections 2024-09-10T22:48:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.024+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182934d80 0x55b182937170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.024+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182928df0 0x55b182934840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.024+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/1606308315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b1829288b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.024+0000 7f1fb2b96740 1 -- 172.21.15.96:0/1606308315 >> 172.21.15.96:0/1606308315 conn(0x55b182926300 msgr2=0x55b1829274a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.025+0000 7f1fb2b96740 1 -- 172.21.15.96:0/1606308315 shutdown_connections 2024-09-10T22:48:05.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.025+0000 7f1fb2b96740 1 -- 172.21.15.96:0/1606308315 wait complete. 2024-09-10T22:48:05.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.026+0000 7f1fb2b96740 1 -- start start 2024-09-10T22:48:05.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.026+0000 7f1fb2b96740 1 Processor -- start 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.026+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b182934670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.026+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.026+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182934d80 0x55b1829320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b1826ef700 con 0x55b182934d80 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b182829160 con 0x55b1829284d0 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b1829379f0 con 0x55b182928df0 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b182934670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fb2b96740 10 client.0 _ll_register_callbacks cb 0x55b1828429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40960/0 (socket says 172.21.15.96:40960) 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b182934670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46622/0 (socket says 172.21.15.96:46622) 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 -- 172.21.15.96:0/835210685 learned_addr learned my addr 172.21.15.96:0/835210685 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182934d80 0x55b1829320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 -- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 msgr2=0x55b182934670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b182934670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 -- 172.21.15.96:0/835210685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182934d80 msgr2=0x55b1829320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182934d80 0x55b1829320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 -- 172.21.15.96:0/835210685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b182856c90 con 0x55b182928df0 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1fae55d640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40095f0 tx=0x7f1fa4009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0899a0091981ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.027+0000 7f1f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400c860 con 0x55b182928df0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f98003d40 con 0x55b182928df0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1fa400c660 con 0x55b182928df0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1fa400c860 con 0x55b182928df0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1fb2b96740 10 client.25120 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.028+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b182943340 con 0x55b182928df0 2024-09-10T22:48:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.029+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f98003d40 con 0x55b182928df0 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1fa4019b70 con 0x55b182928df0 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 10 client.25120 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 10 client.25120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 20 client.25120 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829447a0 0x55b182946bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 4 client.25120 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1828a0df0 con 0x55b1829447a0 2024-09-10T22:48:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.030+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829447a0 0x55b182946bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.031+0000 7f1f91ffb640 10 client.25120 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.031+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829447a0 0x55b182946bc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9824b33a42552604 server_cookie=20a6b1a43d16ea32 in_seq=0 out_seq=0 2024-09-10T22:48:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.031+0000 7f1f91ffb640 1 -- 172.21.15.96:0/835210685 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b1828a0df0 con 0x55b1829447a0 2024-09-10T22:48:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.031+0000 7f1f91ffb640 10 client.25120 handle_command_reply: tid=0 2024-09-10T22:48:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 client.25120 shutdown 2024-09-10T22:48:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 msgr2=0x55b182931ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40095f0 tx=0x7f1fa4009ab0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 shutdown_connections 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829447a0 0x55b182946bc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b182934d80 0x55b1829320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b182928df0 0x55b182931ac0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40095f0 tx=0x7f1fa4009ab0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/835210685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1829284d0 0x55b182934670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1f91ffb640 0 client.25120 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.032+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 >> 172.21.15.96:0/835210685 conn(0x55b182926300 msgr2=0x55b18292bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.033+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 shutdown_connections 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.033+0000 7f1fb2b96740 1 -- 172.21.15.96:0/835210685 wait complete. 2024-09-10T22:48:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.033+0000 7f1fb2b96740 20 client.25120 trim_cache size 0 max 16384 2024-09-10T22:48:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.056+0000 7f1fb2b96740 1 Processor -- start 2024-09-10T22:48:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.056+0000 7f1fb2b96740 1 -- start start 2024-09-10T22:48:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b182940c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b182927890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b4ad0 con 0x55b18292d440 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b66d0 con 0x55b182934d80 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b51a0 con 0x55b1829284d0 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46624/0 (socket says 172.21.15.96:46624) 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 -- 172.21.15.96:0/2379351877 learned_addr learned my addr 172.21.15.96:0/2379351877 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b182940c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b182927890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 -- 172.21.15.96:0/2379351877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 msgr2=0x55b182940c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b182940c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 -- 172.21.15.96:0/2379351877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 msgr2=0x55b182927890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b182927890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fae55d640 1 -- 172.21.15.96:0/2379351877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b182921990 con 0x55b182934d80 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.057+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b182927890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1fae55d640 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000c00 tx=0x7f1fa40186f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57754149c84f1db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1f93fff640 1 -- 172.21.15.96:0/2379351877 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400c860 con 0x55b182934d80 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1f93fff640 1 -- 172.21.15.96:0/2379351877 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa401e070 con 0x55b182934d80 2024-09-10T22:48:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1f93fff640 1 -- 172.21.15.96:0/2379351877 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4005330 con 0x55b182934d80 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1fb2b96740 1 -- 172.21.15.96:0/2379351877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 msgr2=0x55b182927350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.058+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000c00 tx=0x7f1fa40186f0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- 172.21.15.96:0/2379351877 shutdown_connections 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b182927350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b182927890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/2379351877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b182940c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- 172.21.15.96:0/2379351877 >> 172.21.15.96:0/2379351877 conn(0x55b182926300 msgr2=0x55b18292cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- 172.21.15.96:0/2379351877 shutdown_connections 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- 172.21.15.96:0/2379351877 wait complete. 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- start start 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 Processor -- start 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b18292f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b18293d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b5f40 con 0x55b18292d440 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b5870 con 0x55b182934d80 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81b2a50 con 0x55b1829284d0 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fb2b96740 10 client.0 _ll_register_callbacks cb 0x7f1fa4005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b18293d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fadd5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b18293d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46636/0 (socket says 172.21.15.96:46636) 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fae55d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41712/0 (socket says 172.21.15.96:41712) 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.059+0000 7f1fadd5c640 1 -- 172.21.15.96:0/267858705 learned_addr learned my addr 172.21.15.96:0/267858705 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b18292f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 -- 172.21.15.96:0/267858705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 msgr2=0x55b18292f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b18292f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 -- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 msgr2=0x55b18293d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b18293d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 -- 172.21.15.96:0/267858705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18293d900 con 0x55b18292d440 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fae55d640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40095c0 tx=0x7f1fa4005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582410a16ef0d431 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400c660 con 0x55b18292d440 2024-09-10T22:48:05.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1fa400c800 con 0x55b18292d440 2024-09-10T22:48:05.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.060+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa401e070 con 0x55b18292d440 2024-09-10T22:48:05.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.061+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1fa400ca00 con 0x55b18292d440 2024-09-10T22:48:05.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 10 client.15534 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:05.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1829b2680 con 0x55b18292d440 2024-09-10T22:48:05.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa401e070 con 0x55b18292d440 2024-09-10T22:48:05.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1fa4003ce0 con 0x55b18292d440 2024-09-10T22:48:05.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:05.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 10 client.15534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 20 client.15534 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829b2c80 0x55b1829b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 4 client.15534 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.062+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b18289c8a0 con 0x55b1829b2c80 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829b2c80 0x55b1829b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1fadd5c640 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829b2c80 0x55b1829b50a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6db7017e5cabe2da server_cookie=6246507f6385eec2 in_seq=0 out_seq=0 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1f91ffb640 10 client.15534 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1f91ffb640 1 -- 172.21.15.96:0/267858705 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b18289c8a0 con 0x55b1829b2c80 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1f91ffb640 10 client.15534 handle_command_reply: tid=0 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1fb2b96740 1 client.15534 shutdown 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 msgr2=0x55b18293cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.063+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40095c0 tx=0x7f1fa4005300 comp rx=0 tx=0).stop 2024-09-10T22:48:05.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 shutdown_connections 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b1829b2c80 0x55b1829b50a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b182934d80 0x55b18293d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b18292d440 0x55b18293cc80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1f91ffb640 0 client.15534 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 --2- 172.21.15.96:0/267858705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1829284d0 0x55b18292f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 >> 172.21.15.96:0/267858705 conn(0x55b182926300 msgr2=0x55b182927350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 shutdown_connections 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 1 -- 172.21.15.96:0/267858705 wait complete. 2024-09-10T22:48:05.069 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.064+0000 7f1fb2b96740 20 client.15534 trim_cache size 0 max 16384 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 msgr2=0x7f1f940b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 0x7f1f940b7810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0005fd0 tx=0x7f1fa0004950 comp rx=0 tx=0).stop 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 msgr2=0x7f1fa81a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 0x7f1fa81a5180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000da10 tx=0x7f1fa000ded0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 shutdown_connections 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076a0 0x7f1fa819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b5350 0x7f1f940b7810 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108020 0x7f1fa81a5180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 --2- 172.21.15.96:0/714838687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101c90 0x7f1fa819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 >> 172.21.15.96:0/714838687 conn(0x7f1fa80fd700 msgr2=0x7f1fa80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 shutdown_connections 2024-09-10T22:48:05.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:05.065+0000 7f1fb07c1640 1 -- 172.21.15.96:0/714838687 wait complete. 2024-09-10T22:48:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 30.0 s, 3876.8 tps, lat 25.817 ms stddev 26.527 2024-09-10T22:48:06.102 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:48:06.414 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:06 smithi096 ceph-mon[30869]: pgmap v371: 97 pgs: 97 active+clean; 972 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-09-10T22:48:06.471 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:06.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:06 smithi121 ceph-mon[35119]: pgmap v371: 97 pgs: 97 active+clean; 972 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-09-10T22:48:06.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:06 smithi117 ceph-mon[35081]: pgmap v371: 97 pgs: 97 active+clean; 972 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 821 op/s 2024-09-10T22:48:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 -- 172.21.15.96:0/969514442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f8109160 msgr2=0x7fa5f810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 --2- 172.21.15.96:0/969514442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f8109160 0x7fa5f810f960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009ca0 tx=0x7fa5f002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 -- 172.21.15.96:0/969514442 shutdown_connections 2024-09-10T22:48:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 --2- 172.21.15.96:0/969514442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f8109160 0x7fa5f810f960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 --2- 172.21.15.96:0/969514442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f81087e0 0x7fa5f8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 --2- 172.21.15.96:0/969514442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 0x7fa5f81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 -- 172.21.15.96:0/969514442 >> 172.21.15.96:0/969514442 conn(0x7fa5f80fe800 msgr2=0x7fa5f8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 -- 172.21.15.96:0/969514442 shutdown_connections 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.169+0000 7fa5fe381640 1 -- 172.21.15.96:0/969514442 wait complete. 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 Processor -- start 2024-09-10T22:48:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 -- start start 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 0x7fa5f819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 0x7fa5f819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 0x7fa5f81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8112e20 con 0x7fa5f8109160 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8112ca0 con 0x7fa5f81087e0 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fe381640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8112fa0 con 0x7fa5f8102dd0 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fc91e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 0x7fa5f81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 0x7fa5f819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fc91e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 0x7fa5f81a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43610/0 (socket says 172.21.15.96:43610) 2024-09-10T22:48:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.170+0000 7fa5fc91e640 1 -- 172.21.15.96:0/1116047168 learned_addr learned my addr 172.21.15.96:0/1116047168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 0x7fa5f819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 -- 172.21.15.96:0/1116047168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 msgr2=0x7fa5f819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 0x7fa5f819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 -- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 msgr2=0x7fa5f81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 0x7fa5f81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 -- 172.21.15.96:0/1116047168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f81a1460 con 0x7fa5f81087e0 2024-09-10T22:48:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 0x7fa5f819b8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00e9f0 tx=0x7fa5ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.171+0000 7fa5f57fa640 1 -- 172.21.15.96:0/1116047168 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00cd80 con 0x7fa5f81087e0 2024-09-10T22:48:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.172+0000 7fa5f57fa640 1 -- 172.21.15.96:0/1116047168 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec0035f0 con 0x7fa5f81087e0 2024-09-10T22:48:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.172+0000 7fa5f57fa640 1 -- 172.21.15.96:0/1116047168 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec010690 con 0x7fa5f81087e0 2024-09-10T22:48:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.172+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f81a1750 con 0x7fa5f81087e0 2024-09-10T22:48:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.172+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f806cb70 con 0x7fa5f81087e0 2024-09-10T22:48:07.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.176+0000 7fa600756740 1 Processor -- start 2024-09-10T22:48:07.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa5f57fa640 1 -- 172.21.15.96:0/1116047168 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fa5ec0108f0 con 0x7fa5f81087e0 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 -- start start 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ee60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ee0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ee1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f84e53ea0 con 0x555f84ee6600 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f84e432d0 con 0x555f84ee11b0 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa600756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555f84e470e0 con 0x555f84ee5ce0 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa5f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ee1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa5f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ee1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57160/0 (socket says 172.21.15.96:57160) 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.177+0000 7fa5f77fe640 1 -- 172.21.15.96:0/4051175481 learned_addr learned my addr 172.21.15.96:0/4051175481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ee0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ee60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 -- 172.21.15.96:0/4051175481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 msgr2=0x555f84ee1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ee1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 -- 172.21.15.96:0/4051175481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 msgr2=0x555f84ee0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ee0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.178+0000 7fa5fc91e640 1 -- 172.21.15.96:0/4051175481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f84ba1650 con 0x555f84ee5ce0 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5f57fa640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 0x7fa5e00b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ee60c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005cc0 tx=0x7fa5f0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc1106e933789c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5de7fc640 1 -- 172.21.15.96:0/4051175481 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0031cf0 con 0x555f84ee5ce0 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5de7fc640 1 -- 172.21.15.96:0/4051175481 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f0036040 con 0x555f84ee5ce0 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5de7fc640 1 -- 172.21.15.96:0/4051175481 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004080 con 0x555f84ee5ce0 2024-09-10T22:48:07.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.180+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 0x7fa5e00b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.181+0000 7fa5f57fa640 1 -- 172.21.15.96:0/1116047168 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa5ec0e6030 con 0x7fa5f81087e0 2024-09-10T22:48:07.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.181+0000 7fa600756740 1 -- 172.21.15.96:0/4051175481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 msgr2=0x555f84ee60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.181+0000 7fa600756740 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ee60c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005cc0 tx=0x7fa5f0005ef0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 0x7fa5e00b7930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x555f84ee57c0 tx=0x7fa5e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 -- 172.21.15.96:0/4051175481 shutdown_connections 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ee1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ee0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 --2- 172.21.15.96:0/4051175481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ee60c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 -- 172.21.15.96:0/4051175481 >> 172.21.15.96:0/4051175481 conn(0x555f84ee3b10 msgr2=0x555f84ee4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 -- 172.21.15.96:0/4051175481 shutdown_connections 2024-09-10T22:48:07.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 -- 172.21.15.96:0/4051175481 wait complete. 2024-09-10T22:48:07.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.182+0000 7fa600756740 1 -- start start 2024-09-10T22:48:07.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 Processor -- start 2024-09-10T22:48:07.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84eef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84eed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f84ca4700 con 0x555f84ee6600 2024-09-10T22:48:07.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f84dde160 con 0x555f84ee11b0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555f84ee1e00 con 0x555f84ee5ce0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa5f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa5f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57176/0 (socket says 172.21.15.96:57176) 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa5f7fff640 1 -- 172.21.15.96:0/1073609885 learned_addr learned my addr 172.21.15.96:0/1073609885 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa600756740 10 client.0 _ll_register_callbacks cb 0x555f84df79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.183+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84eef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84eed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 -- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 msgr2=0x555f84eef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84eef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 -- 172.21.15.96:0/1073609885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 msgr2=0x555f84eed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84eed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 -- 172.21.15.96:0/1073609885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f84e0bc90 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80045d0 tx=0x7fa5e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaae9a62af9213b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8009e60 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f0036040 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.184+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa5e801f640 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.185+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5e801f840 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.185+0000 7fa600756740 10 client.25118 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.185+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f84ef8320 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.185+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f0036040 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa5e8027070 con 0x555f84ee11b0 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 10 client.25118 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:07.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 10 client.25118 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:48:07.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 20 client.25118 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:07.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84ef9b40 0x555f84efbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 4 client.25118 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:07.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f84e55df0 con 0x555f84ef9b40 2024-09-10T22:48:07.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.186+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84ef9b40 0x555f84efbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.187+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84ef9b40 0x555f84efbfc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0ad0f5840e5578 server_cookie=31676b89b3ed2838 in_seq=0 out_seq=0 2024-09-10T22:48:07.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.187+0000 7fa5d77fe640 10 client.25118 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:07.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.187+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1073609885 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f84e55df0 con 0x555f84ef9b40 2024-09-10T22:48:07.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.187+0000 7fa5d77fe640 10 client.25118 handle_command_reply: tid=0 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.187+0000 7fa600756740 1 client.25118 shutdown 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 msgr2=0x555f84eef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80045d0 tx=0x7fa5e8004730 comp rx=0 tx=0).stop 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 shutdown_connections 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84eed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84eef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84ef9b40 0x555f84efbfc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 --2- 172.21.15.96:0/1073609885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84eef390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 >> 172.21.15.96:0/1073609885 conn(0x555f84ee3b10 msgr2=0x555f84ee9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa5d77fe640 0 client.25118 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 shutdown_connections 2024-09-10T22:48:07.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 1 -- 172.21.15.96:0/1073609885 wait complete. 2024-09-10T22:48:07.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.188+0000 7fa600756740 20 client.25118 trim_cache size 0 max 16384 2024-09-10T22:48:07.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.205+0000 7fa600756740 1 Processor -- start 2024-09-10T22:48:07.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 -- start start 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84edf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 0x555f84ee4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8070e70 con 0x555f84ee5ce0 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8072a70 con 0x555f84ee11b0 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa600756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8071540 con 0x555f84ee6600 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa5f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.206+0000 7fa5f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43628/0 (socket says 172.21.15.96:43628) 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 -- 172.21.15.96:0/3197008430 learned_addr learned my addr 172.21.15.96:0/3197008430 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 0x555f84ee4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 -- 172.21.15.96:0/3197008430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 msgr2=0x555f84ee4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 0x555f84ee4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 -- 172.21.15.96:0/3197008430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 msgr2=0x555f84edf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84edf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 -- 172.21.15.96:0/3197008430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f84ee42b0 con 0x555f84ee5ce0 2024-09-10T22:48:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 0x555f84ee4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.207+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0a8db0 tx=0x7fa5ec0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4ecac2a088d478 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.208+0000 7fa5ddffb640 1 -- 172.21.15.96:0/3197008430 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0a8680 con 0x555f84ee5ce0 2024-09-10T22:48:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.208+0000 7fa5ddffb640 1 -- 172.21.15.96:0/3197008430 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec0ac2f0 con 0x555f84ee5ce0 2024-09-10T22:48:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.208+0000 7fa5ddffb640 1 -- 172.21.15.96:0/3197008430 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0abf50 con 0x555f84ee5ce0 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 -- 172.21.15.96:0/3197008430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 msgr2=0x555f84edf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0a8db0 tx=0x7fa5ec0039f0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 -- 172.21.15.96:0/3197008430 shutdown_connections 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee6600 0x555f84ee4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee5ce0 0x555f84edf8c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 --2- 172.21.15.96:0/3197008430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84edf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 -- 172.21.15.96:0/3197008430 >> 172.21.15.96:0/3197008430 conn(0x555f84ee3b10 msgr2=0x555f84edae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.209+0000 7fa600756740 1 -- 172.21.15.96:0/3197008430 shutdown_connections 2024-09-10T22:48:07.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.210+0000 7fa600756740 1 -- 172.21.15.96:0/3197008430 wait complete. 2024-09-10T22:48:07.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.210+0000 7fa600756740 1 -- start start 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.210+0000 7fa600756740 1 Processor -- start 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.210+0000 7fa600756740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ef2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ef27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80722e0 con 0x555f84ee6600 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8071c10 con 0x555f84ee11b0 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f806ee50 con 0x555f84ee5ce0 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa600756740 10 client.0 _ll_register_callbacks cb 0x7fa5e8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43638/0 (socket says 172.21.15.96:43638) 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 -- 172.21.15.96:0/1692144202 learned_addr learned my addr 172.21.15.96:0/1692144202 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ef2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5f77fe640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ef27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 -- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 msgr2=0x555f84ef27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ef27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 -- 172.21.15.96:0/1692144202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 msgr2=0x555f84ef2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ef2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 -- 172.21.15.96:0/1692144202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f84f65ad0 con 0x555f84ee6600 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ef2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5fc91e640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005e30 tx=0x7fa5f002f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d501d2af83932b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.211+0000 7fa5d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.212+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0031450 con 0x555f84ee6600 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.212+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa5f00315f0 con 0x555f84ee6600 2024-09-10T22:48:07.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.212+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ec0ac2f0 con 0x555f84ee6600 2024-09-10T22:48:07.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.213+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5f00317f0 con 0x555f84ee6600 2024-09-10T22:48:07.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.213+0000 7fa600756740 10 client.15549 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:07.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.213+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f84f66dd0 con 0x555f84ee6600 2024-09-10T22:48:07.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.213+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f0036040 con 0x555f84ee6600 2024-09-10T22:48:07.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa5f0004530 con 0x555f84ee6600 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 10 client.15549 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 10 client.15549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 20 client.15549 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84f674e0 0x555f84f69900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 4 client.15549 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa5ec0ac2f0 con 0x555f84f674e0 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.214+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84f674e0 0x555f84f69900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa5f7fff640 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84f674e0 0x555f84f69900 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=541253842e0a0cfd server_cookie=12412ba0a02a1245 in_seq=0 out_seq=0 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa5d77fe640 10 client.15549 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa5d77fe640 1 -- 172.21.15.96:0/1692144202 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa5ec0ac2f0 con 0x555f84f674e0 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa5d77fe640 10 client.15549 handle_command_reply: tid=0 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 client.15549 shutdown 2024-09-10T22:48:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 msgr2=0x555f84ef2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005e30 tx=0x7fa5f002f330 comp rx=0 tx=0).stop 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 shutdown_connections 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f84ee6600 0x555f84ef2d00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f84ee5ce0 0x555f84ef27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x555f84f674e0 0x555f84f69900 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa5d77fe640 0 client.15549 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 --2- 172.21.15.96:0/1692144202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f84ee11b0 0x555f84ef2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.215+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 >> 172.21.15.96:0/1692144202 conn(0x555f84ee3b10 msgr2=0x555f84edf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.216+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 shutdown_connections 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.216+0000 7fa600756740 1 -- 172.21.15.96:0/1692144202 wait complete. 2024-09-10T22:48:07.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.216+0000 7fa600756740 20 client.15549 trim_cache size 0 max 16384 2024-09-10T22:48:07.220 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.217+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 msgr2=0x7fa5e00b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.217+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 0x7fa5e00b7930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x555f84ee57c0 tx=0x7fa5e8005f90 comp rx=0 tx=0).stop 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.217+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 msgr2=0x7fa5f819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.217+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 0x7fa5f819b8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00e9f0 tx=0x7fa5ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 shutdown_connections 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f81087e0 0x7fa5f819b8f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa5e00b5470 0x7fa5e00b7930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5f8109160 0x7fa5f81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 --2- 172.21.15.96:0/1116047168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5f8102dd0 0x7fa5f819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:07.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 >> 172.21.15.96:0/1116047168 conn(0x7fa5f80fe800 msgr2=0x7fa5f8104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:07.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 shutdown_connections 2024-09-10T22:48:07.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:07.218+0000 7fa5fe381640 1 -- 172.21.15.96:0/1116047168 wait complete. 2024-09-10T22:48:08.121 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:48:08.476 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:08.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:08 smithi096 ceph-mon[30869]: pgmap v372: 97 pgs: 97 active+clean; 973 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 709 op/s 2024-09-10T22:48:08.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:08 smithi121 ceph-mon[35119]: pgmap v372: 97 pgs: 97 active+clean; 973 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 709 op/s 2024-09-10T22:48:08.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:08 smithi117 ceph-mon[35081]: pgmap v372: 97 pgs: 97 active+clean; 973 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 709 op/s 2024-09-10T22:48:09.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.040+0000 7ff46b8de640 1 -- 172.21.15.96:0/4127410141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff46410a9d0 msgr2=0x7ff46410ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.040+0000 7ff46b8de640 1 --2- 172.21.15.96:0/4127410141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff46410a9d0 0x7ff46410ae10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff454009900 tx=0x7ff45402a410 comp rx=0 tx=0).stop 2024-09-10T22:48:09.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 -- 172.21.15.96:0/4127410141 shutdown_connections 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 --2- 172.21.15.96:0/4127410141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff46410a9d0 0x7ff46410ae10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 --2- 172.21.15.96:0/4127410141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 0x7ff464111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 --2- 172.21.15.96:0/4127410141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104fc0 0x7ff4641053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 -- 172.21.15.96:0/4127410141 >> 172.21.15.96:0/4127410141 conn(0x7ff4641009f0 msgr2=0x7ff464102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 -- 172.21.15.96:0/4127410141 shutdown_connections 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.042+0000 7ff46b8de640 1 -- 172.21.15.96:0/4127410141 wait complete. 2024-09-10T22:48:09.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.043+0000 7ff46b8de640 1 Processor -- start 2024-09-10T22:48:09.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 -- start start 2024-09-10T22:48:09.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff464104fc0 0x7ff46419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 0x7ff4641a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff464114fd0 con 0x7ff46410a9d0 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff464114e50 con 0x7ff464104fc0 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.045+0000 7ff46b8de640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff464115150 con 0x7ff46410b350 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.046+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.046+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43660/0 (socket says 172.21.15.96:43660) 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.046+0000 7ff468e79640 1 -- 172.21.15.96:0/283715384 learned_addr learned my addr 172.21.15.96:0/283715384 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.046+0000 7ff46967a640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff464104fc0 0x7ff46419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.046+0000 7ff469e7b640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 0x7ff4641a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 -- 172.21.15.96:0/283715384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 msgr2=0x7ff4641a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 0x7ff4641a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 -- 172.21.15.96:0/283715384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff464104fc0 msgr2=0x7ff46419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff464104fc0 0x7ff46419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 -- 172.21.15.96:0/283715384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641a1570 con 0x7ff46410a9d0 2024-09-10T22:48:09.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.047+0000 7ff468e79640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff454005940 tx=0x7ff4540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.049+0000 7ff45a7fc640 1 -- 172.21.15.96:0/283715384 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45403f030 con 0x7ff46410a9d0 2024-09-10T22:48:09.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.049+0000 7ff45a7fc640 1 -- 172.21.15.96:0/283715384 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff454003cb0 con 0x7ff46410a9d0 2024-09-10T22:48:09.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.050+0000 7ff45a7fc640 1 -- 172.21.15.96:0/283715384 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454035070 con 0x7ff46410a9d0 2024-09-10T22:48:09.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.055+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641a1800 con 0x7ff46410a9d0 2024-09-10T22:48:09.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.055+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a7f20 con 0x7ff46410a9d0 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 Processor -- start 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 -- start start 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b361b0 0x55e275b365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e275aa8ea0 con 0x55e275b3b600 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e275a982d0 con 0x55e275b3ace0 2024-09-10T22:48:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.060+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e275a9c0e0 con 0x55e275b361b0 2024-09-10T22:48:09.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.064+0000 7ff45a7fc640 1 -- 172.21.15.96:0/283715384 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7ff454003800 con 0x7ff46410a9d0 2024-09-10T22:48:09.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.065+0000 7ff469e7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.065+0000 7ff469e7b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57208/0 (socket says 172.21.15.96:57208) 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.065+0000 7ff469e7b640 1 -- 172.21.15.96:0/1650770378 learned_addr learned my addr 172.21.15.96:0/1650770378 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.065+0000 7ff46967a640 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.066+0000 7ff45a7fc640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 0x7ff4500b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.067+0000 7ff469e7b640 1 -- 172.21.15.96:0/1650770378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b361b0 msgr2=0x55e275b365f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.067+0000 7ff469e7b640 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b361b0 0x55e275b365f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.067+0000 7ff469e7b640 1 -- 172.21.15.96:0/1650770378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 msgr2=0x55e275b35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.067+0000 7ff469e7b640 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.067+0000 7ff469e7b640 1 -- 172.21.15.96:0/1650770378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2757f6650 con 0x55e275b3ace0 2024-09-10T22:48:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.068+0000 7ff469e7b640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 0x7ff4500b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.068+0000 7ff469e7b640 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff45c010370 tx=0x7ff45c010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d445347cc9455f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.068+0000 7ff45a7fc640 1 -- 172.21.15.96:0/283715384 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff454104100 con 0x7ff46410a9d0 2024-09-10T22:48:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.069+0000 7ff469e7b640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 0x7ff4500b7700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff45c01f000 tx=0x7ff45c022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.069+0000 7ff44f7fe640 1 -- 172.21.15.96:0/1650770378 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c022070 con 0x55e275b3ace0 2024-09-10T22:48:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.070+0000 7ff44f7fe640 1 -- 172.21.15.96:0/1650770378 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45c0113e0 con 0x55e275b3ace0 2024-09-10T22:48:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 -- 172.21.15.96:0/1650770378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 msgr2=0x55e275b3b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff45c010370 tx=0x7ff45c010830 comp rx=0 tx=0).stop 2024-09-10T22:48:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 -- 172.21.15.96:0/1650770378 shutdown_connections 2024-09-10T22:48:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b361b0 0x55e275b365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/1650770378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b3ace0 0x55e275b3b0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.071+0000 7ff46dcb3740 1 -- 172.21.15.96:0/1650770378 >> 172.21.15.96:0/1650770378 conn(0x55e275b38b10 msgr2=0x55e275b39cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.075+0000 7ff46dcb3740 1 -- 172.21.15.96:0/1650770378 shutdown_connections 2024-09-10T22:48:09.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.076+0000 7ff46dcb3740 1 -- 172.21.15.96:0/1650770378 wait complete. 2024-09-10T22:48:09.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.076+0000 7ff46dcb3740 1 -- start start 2024-09-10T22:48:09.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.077+0000 7ff46dcb3740 1 Processor -- start 2024-09-10T22:48:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.077+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e2758f9700 con 0x55e275b3ace0 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e275a33160 con 0x55e275b361b0 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e275b36e80 con 0x55e275b3b600 2024-09-10T22:48:09.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.078+0000 7ff46dcb3740 10 client.0 _ll_register_callbacks cb 0x55e275a4c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:09.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.079+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.079+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43680/0 (socket says 172.21.15.96:43680) 2024-09-10T22:48:09.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.079+0000 7ff469e7b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.079+0000 7ff469e7b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45090/0 (socket says 172.21.15.96:45090) 2024-09-10T22:48:09.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.079+0000 7ff469e7b640 1 -- 172.21.15.96:0/2546020003 learned_addr learned my addr 172.21.15.96:0/2546020003 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:09.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff468e79640 1 -- 172.21.15.96:0/2546020003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 msgr2=0x55e275b34140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff46967a640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff468e79640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff468e79640 1 -- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 msgr2=0x55e275b357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff468e79640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff468e79640 1 -- 172.21.15.96:0/2546020003 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e275a60c90 con 0x55e275b3ace0 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.080+0000 7ff46967a640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.081+0000 7ff468e79640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff454103eb0 tx=0x7ff4540c9270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3231ed22110ece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.081+0000 7ff44cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:09.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.087+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4540cc2d0 con 0x55e275b3ace0 2024-09-10T22:48:09.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.087+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff4540ccfd0 con 0x55e275b3ace0 2024-09-10T22:48:09.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.087+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c0113e0 con 0x55e275b3ace0 2024-09-10T22:48:09.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.089+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4540c82d0 con 0x55e275b3ace0 2024-09-10T22:48:09.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.091+0000 7ff46dcb3740 10 client.15558 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:09.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.091+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e275b5e3a0 con 0x55e275b3ace0 2024-09-10T22:48:09.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.092+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45c0113e0 con 0x55e275b3ace0 2024-09-10T22:48:09.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.092+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff4540b5070 con 0x55e275b3ace0 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 10 client.15558 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 20 client.15558 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275b5f860 0x55e275b61d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 4 client.15558 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:09.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.093+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e275aaadf0 con 0x55e275b5f860 2024-09-10T22:48:09.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.095+0000 7ff46967a640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275b5f860 0x55e275b61d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.101+0000 7ff46967a640 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275b5f860 0x55e275b61d00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428d0aba694392a0 server_cookie=3796e6a149a40b81 in_seq=0 out_seq=0 2024-09-10T22:48:09.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.101+0000 7ff44cff9640 10 client.15558 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:09.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.105+0000 7ff44cff9640 1 -- 172.21.15.96:0/2546020003 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e275aaadf0 con 0x55e275b5f860 2024-09-10T22:48:09.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.105+0000 7ff44cff9640 10 client.15558 handle_command_reply: tid=0 2024-09-10T22:48:09.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.105+0000 7ff46dcb3740 1 client.15558 shutdown 2024-09-10T22:48:09.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.105+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 msgr2=0x55e275b33c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.105+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff454103eb0 tx=0x7ff4540c9270 comp rx=0 tx=0).stop 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 shutdown_connections 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275b5f860 0x55e275b61d00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b33c00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2546020003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 >> 172.21.15.96:0/2546020003 conn(0x55e275b38b10 msgr2=0x55e275b3e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff44cff9640 0 client.15558 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 shutdown_connections 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2546020003 wait complete. 2024-09-10T22:48:09.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.106+0000 7ff46dcb3740 20 client.15558 trim_cache size 0 max 16384 2024-09-10T22:48:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.128+0000 7ff46dcb3740 1 Processor -- start 2024-09-10T22:48:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.128+0000 7ff46dcb3740 1 -- start start 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b67b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff468e79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43694/0 (socket says 172.21.15.96:43694) 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff468e79640 1 -- 172.21.15.96:0/2417553873 learned_addr learned my addr 172.21.15.96:0/2417553873 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641abfa0 con 0x55e275b3ace0 2024-09-10T22:48:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641adba0 con 0x55e275b361b0 2024-09-10T22:48:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46967a640 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.129+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641ac670 con 0x55e275b3b600 2024-09-10T22:48:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b67b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff468e79640 1 -- 172.21.15.96:0/2417553873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 msgr2=0x55e275b67b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff468e79640 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b67b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff468e79640 1 -- 172.21.15.96:0/2417553873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 msgr2=0x55e275b3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff468e79640 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.130+0000 7ff468e79640 1 -- 172.21.15.96:0/2417553873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e275b62380 con 0x55e275b3ace0 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.131+0000 7ff468e79640 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff4540c92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3325ff3f3ff64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.131+0000 7ff44effd640 1 -- 172.21.15.96:0/2417553873 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4540c82d0 con 0x55e275b3ace0 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.131+0000 7ff44effd640 1 -- 172.21.15.96:0/2417553873 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4540cd260 con 0x55e275b3ace0 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.131+0000 7ff44effd640 1 -- 172.21.15.96:0/2417553873 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454003e60 con 0x55e275b3ace0 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 msgr2=0x55e275b67610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff4540c92e0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 shutdown_connections 2024-09-10T22:48:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3b600 0x55e275b67b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3ace0 0x55e275b67610 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2417553873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.132+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 >> 172.21.15.96:0/2417553873 conn(0x55e275b38b10 msgr2=0x55e275b39c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.133+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 shutdown_connections 2024-09-10T22:48:09.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.133+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2417553873 wait complete. 2024-09-10T22:48:09.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.133+0000 7ff46dcb3740 1 -- start start 2024-09-10T22:48:09.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.133+0000 7ff46dcb3740 1 Processor -- start 2024-09-10T22:48:09.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 0x55e275b564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641ad410 con 0x55e275b3b600 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641acd40 con 0x55e275b361b0 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641a9f40 con 0x55e275b3ace0 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.134+0000 7ff46dcb3740 10 client.0 _ll_register_callbacks cb 0x7ff454005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.136+0000 7ff468e79640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 0x55e275b564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.136+0000 7ff468e79640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 0x55e275b564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45108/0 (socket says 172.21.15.96:45108) 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.136+0000 7ff468e79640 1 -- 172.21.15.96:0/2238387256 learned_addr learned my addr 172.21.15.96:0/2238387256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.136+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.138+0000 7ff469e7b640 1 -- 172.21.15.96:0/2238387256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 msgr2=0x55e275b564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.138+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 0x55e275b564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.138+0000 7ff469e7b640 1 -- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 msgr2=0x55e275b55fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.138+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b55fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.138+0000 7ff469e7b640 1 -- 172.21.15.96:0/2238387256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e275b36d10 con 0x55e275b3b600 2024-09-10T22:48:09.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.139+0000 7ff469e7b640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b56a30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f40 tx=0x7ff45c012900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe7f7d20cdfd3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:09.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.139+0000 7ff44cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:09.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.139+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c01c8e0 con 0x55e275b3b600 2024-09-10T22:48:09.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.139+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4540cd260 con 0x55e275b3b600 2024-09-10T22:48:09.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.139+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff45c01ca80 con 0x55e275b3b600 2024-09-10T22:48:09.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.140+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff45c01cc80 con 0x55e275b3b600 2024-09-10T22:48:09.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.142+0000 7ff46dcb3740 10 client.15564 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:09.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.142+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e275bcc830 con 0x55e275b3b600 2024-09-10T22:48:09.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.142+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4540cd260 con 0x55e275b3b600 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.142+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff45c00f040 con 0x55e275b3b600 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.142+0000 7ff46dcb3740 10 client.15564 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.143+0000 7ff46dcb3740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.143+0000 7ff46dcb3740 20 client.15564 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.143+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275bcd030 0x55e275bcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.143+0000 7ff46dcb3740 4 client.15564 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:09.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.143+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e275aa68a0 con 0x55e275bcd030 2024-09-10T22:48:09.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.144+0000 7ff46967a640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275bcd030 0x55e275bcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:09.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.146+0000 7ff46967a640 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275bcd030 0x55e275bcf450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cab1b44786aa06b server_cookie=3a091f85f1c56d90 in_seq=0 out_seq=0 2024-09-10T22:48:09.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.146+0000 7ff44cff9640 10 client.15564 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff44cff9640 1 -- 172.21.15.96:0/2238387256 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e275aa68a0 con 0x55e275bcd030 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff44cff9640 10 client.15564 handle_command_reply: tid=0 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 client.15564 shutdown 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 msgr2=0x55e275b56a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b56a30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009f40 tx=0x7ff45c012900 comp rx=0 tx=0).stop 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 shutdown_connections 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e275bcd030 0x55e275bcf450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e275b3b600 0x55e275b56a30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e275b3ace0 0x55e275b564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 --2- 172.21.15.96:0/2238387256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e275b361b0 0x55e275b55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.147+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 >> 172.21.15.96:0/2238387256 conn(0x55e275b38b10 msgr2=0x55e275b3de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.148+0000 7ff44cff9640 0 client.15564 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.148+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 shutdown_connections 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.148+0000 7ff46dcb3740 1 -- 172.21.15.96:0/2238387256 wait complete. 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.148+0000 7ff46dcb3740 20 client.15564 trim_cache size 0 max 16384 2024-09-10T22:48:09.150 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 msgr2=0x7ff4500b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 0x7ff4500b7700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff45c01f000 tx=0x7ff45c022000 comp rx=0 tx=0).stop 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 msgr2=0x7ff46419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff454005940 tx=0x7ff4540042f0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 shutdown_connections 2024-09-10T22:48:09.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff4500b5240 0x7ff4500b7700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff46410b350 0x7ff4641a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff46410a9d0 0x7ff46419ba00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 --2- 172.21.15.96:0/283715384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff464104fc0 0x7ff46419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 >> 172.21.15.96:0/283715384 conn(0x7ff4641009f0 msgr2=0x7ff464108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 shutdown_connections 2024-09-10T22:48:09.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:09.149+0000 7ff46b8de640 1 -- 172.21.15.96:0/283715384 wait complete. 2024-09-10T22:48:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:10 smithi096 ceph-mon[30869]: pgmap v373: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 854 op/s 2024-09-10T22:48:10.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:10 smithi121 ceph-mon[35119]: pgmap v373: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 854 op/s 2024-09-10T22:48:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:10 smithi117 ceph-mon[35081]: pgmap v373: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 854 op/s 2024-09-10T22:48:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 35.0 s, 3312.6 tps, lat 30.187 ms stddev 31.293 2024-09-10T22:48:11.005 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:48:11.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:48:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:12.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:12 smithi096 ceph-mon[30869]: pgmap v374: 97 pgs: 97 active+clean; 990 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T22:48:12.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:12 smithi121 ceph-mon[35119]: pgmap v374: 97 pgs: 97 active+clean; 990 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T22:48:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:12 smithi117 ceph-mon[35081]: pgmap v374: 97 pgs: 97 active+clean; 990 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T22:48:13.471 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:48:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:48:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "5c4cd589-68b1-4458-8ca3-09456aeee767", 2024-09-10T22:48:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:48:13.471 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:48:13.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.058+0000 7f35dfb78640 1 -- 172.21.15.96:0/1605685954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d8114810 msgr2=0x7f35d8116c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.058+0000 7f35dfb78640 1 --2- 172.21.15.96:0/1605685954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d8114810 0x7f35d8116c60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009ca0 tx=0x7f35d002a950 comp rx=0 tx=0).stop 2024-09-10T22:48:13.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.058+0000 7f35dfb78640 1 -- 172.21.15.96:0/1605685954 shutdown_connections 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 --2- 172.21.15.96:0/1605685954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d8114810 0x7f35d8116c60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 --2- 172.21.15.96:0/1605685954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d81142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 --2- 172.21.15.96:0/1605685954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d810b260 0x7f35d810b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 -- 172.21.15.96:0/1605685954 >> 172.21.15.96:0/1605685954 conn(0x7f35d806dfc0 msgr2=0x7f35d806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 -- 172.21.15.96:0/1605685954 shutdown_connections 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 -- 172.21.15.96:0/1605685954 wait complete. 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.059+0000 7f35dfb78640 1 Processor -- start 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 -- start start 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d810e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 0x7f35d8113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81184f0 con 0x7f35d810b260 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8118370 con 0x7f35d8114810 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dfb78640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8118670 con 0x7f35d810bb80 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dd914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dd914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43720/0 (socket says 172.21.15.96:43720) 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35dd914640 1 -- 172.21.15.96:0/289336418 learned_addr learned my addr 172.21.15.96:0/289336418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:13.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.060+0000 7f35de115640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 0x7f35d8113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35dd113640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d810e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35de115640 1 -- 172.21.15.96:0/289336418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 msgr2=0x7f35d810e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35de115640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d810e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35de115640 1 -- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 msgr2=0x7f35d810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35de115640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35de115640 1 -- 172.21.15.96:0/289336418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d8113f30 con 0x7f35d8114810 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35dd113640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.061+0000 7f35dd914640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.062+0000 7f35de115640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 0x7f35d8113760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35d002af40 tx=0x7f35d0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.062+0000 7f35ceffd640 1 -- 172.21.15.96:0/289336418 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0034040 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.063+0000 7f35ceffd640 1 -- 172.21.15.96:0/289336418 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d0030850 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.063+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d81141c0 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.063+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d81bd0a0 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.064+0000 7f35ceffd640 1 -- 172.21.15.96:0/289336418 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00046e0 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.067+0000 7f35ceffd640 1 -- 172.21.15.96:0/289336418 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f35d0039430 con 0x7f35d8114810 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 Processor -- start 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 -- start start 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bbc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bcab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568e3b32e60 con 0x5568e3bc8770 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568e3b22290 con 0x5568e3bbc7e0 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5568e3b260a0 con 0x5568e3bbbec0 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35dd113640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bbc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35dd113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bcab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35dd113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bcab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43724/0 (socket says 172.21.15.96:43724) 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35dd113640 1 -- 172.21.15.96:0/3632937019 learned_addr learned my addr 172.21.15.96:0/3632937019 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.068+0000 7f35dd914640 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 -- 172.21.15.96:0/3632937019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 msgr2=0x5568e3bc8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 -- 172.21.15.96:0/3632937019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 msgr2=0x5568e3bcab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bcab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 -- 172.21.15.96:0/3632937019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e3880930 con 0x5568e3bbbec0 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35dd113640 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bbc2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35c8010d60 tx=0x7f35c80096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50dbb8764ee94f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.069+0000 7f35c3fff640 1 -- 172.21.15.96:0/3632937019 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8016030 con 0x5568e3bbbec0 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.070+0000 7f35c3fff640 1 -- 172.21.15.96:0/3632937019 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c8010e50 con 0x5568e3bbbec0 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.070+0000 7f35c3fff640 1 -- 172.21.15.96:0/3632937019 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8019040 con 0x5568e3bbbec0 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3632937019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 msgr2=0x5568e3bbc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bbc2a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35c8010d60 tx=0x7f35c80096c0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3632937019 shutdown_connections 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35ceffd640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 0x7f35c40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bcab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3632937019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bbc2a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3632937019 >> 172.21.15.96:0/3632937019 conn(0x5568e3bb9cf0 msgr2=0x5568e3bbae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35dd113640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 0x7f35c40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.071+0000 7f35ceffd640 1 -- 172.21.15.96:0/289336418 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35d0107ad0 con 0x7f35d8114810 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.072+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3632937019 shutdown_connections 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.072+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3632937019 wait complete. 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.072+0000 7f35e1f4d740 1 -- start start 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.072+0000 7f35e1f4d740 1 Processor -- start 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.073+0000 7f35dd113640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 0x7f35c40b7a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009fd0 tx=0x7f35c8012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbbec0 0x5568e3bc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bc61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568e3983890 con 0x5568e3bc8770 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568e3abcd40 con 0x5568e3bbbec0 2024-09-10T22:48:13.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5568e3bcb420 con 0x5568e3bbc7e0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35e1f4d740 10 client.0 _ll_register_callbacks cb 0x5568e3b20c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35de115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bc61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35de115640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bc61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43732/0 (socket says 172.21.15.96:43732) 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35de115640 1 -- 172.21.15.96:0/1478531082 learned_addr learned my addr 172.21.15.96:0/1478531082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd914640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbbec0 0x5568e3bc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 -- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbbec0 msgr2=0x5568e3bc5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbbec0 0x5568e3bc5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 -- 172.21.15.96:0/1478531082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 msgr2=0x5568e3bc61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bc61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.074+0000 7f35dd113640 1 -- 172.21.15.96:0/1478531082 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e3bd4830 con 0x5568e3bbc7e0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35dd113640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc5c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004910 tx=0x7f35c80096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99461ffab0a51852 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8016070 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35c8026030 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c8010e50 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35c8004020 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35e1f4d740 10 client.25168 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.075+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568e3bd71e0 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.076+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c8010e50 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35c803d050 con 0x5568e3bbc7e0 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 10 client.25168 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 10 client.25168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 20 client.25168 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3bd8530 0x5568e3bda920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 4 client.25168 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.077+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e3b34db0 con 0x5568e3bd8530 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.079+0000 7f35dd914640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3bd8530 0x5568e3bda920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.079+0000 7f35dd914640 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3bd8530 0x5568e3bda920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba07e05d4e197f8a server_cookie=28458c7d3aff3fbd in_seq=0 out_seq=0 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.080+0000 7f35c17fa640 10 client.25168 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.080+0000 7f35c17fa640 1 -- 172.21.15.96:0/1478531082 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568e3b34db0 con 0x5568e3bd8530 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.080+0000 7f35c17fa640 10 client.25168 handle_command_reply: tid=0 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.080+0000 7f35e1f4d740 1 client.25168 shutdown 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 msgr2=0x5568e3bc5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc5c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004910 tx=0x7f35c80096c0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 shutdown_connections 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3bd8530 0x5568e3bda920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bc8770 0x5568e3bc61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc5c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1478531082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbbec0 0x5568e3bc5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 >> 172.21.15.96:0/1478531082 conn(0x5568e3bb9cf0 msgr2=0x5568e3bb8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35c17fa640 0 client.25168 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 shutdown_connections 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1478531082 wait complete. 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.081+0000 7f35e1f4d740 20 client.25168 trim_cache size 0 max 16384 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 Processor -- start 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 -- start start 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbbec0 0x5568e3bc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c57d0 con 0x5568e3bbbec0 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c73d0 con 0x5568e3bbc7e0 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c5ea0 con 0x5568e3bc8770 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35dd914640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbbec0 0x5568e3bc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35de115640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35de115640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45148/0 (socket says 172.21.15.96:45148) 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.106+0000 7f35de115640 1 -- 172.21.15.96:0/1411955616 learned_addr learned my addr 172.21.15.96:0/1411955616 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35dd113640 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 -- 172.21.15.96:0/1411955616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 msgr2=0x5568e3bc7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 -- 172.21.15.96:0/1411955616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbbec0 msgr2=0x5568e3bc31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbbec0 0x5568e3bc31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 -- 172.21.15.96:0/1411955616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e3bba770 con 0x5568e3bc8770 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35de115640 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35d00ce0c0 tx=0x7f35d002a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63af311176c1d20e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35c37fe640 1 -- 172.21.15.96:0/1411955616 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00cd8a0 con 0x5568e3bc8770 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35c37fe640 1 -- 172.21.15.96:0/1411955616 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d00cdc40 con 0x5568e3bc8770 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.107+0000 7f35c37fe640 1 -- 172.21.15.96:0/1411955616 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00d76c0 con 0x5568e3bc8770 2024-09-10T22:48:13.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1411955616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 msgr2=0x5568e3be0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35d00ce0c0 tx=0x7f35d002a950 comp rx=0 tx=0).stop 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1411955616 shutdown_connections 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bbc7e0 0x5568e3bc7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bc8770 0x5568e3be0040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/1411955616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbbec0 0x5568e3bc31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1411955616 >> 172.21.15.96:0/1411955616 conn(0x5568e3bb9cf0 msgr2=0x5568e3bc6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1411955616 shutdown_connections 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- 172.21.15.96:0/1411955616 wait complete. 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.108+0000 7f35e1f4d740 1 -- start start 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 Processor -- start 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 0x5568e3bccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bc8770 0x5568e3c44240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c6c40 con 0x5568e3bbc7e0 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.109+0000 7f35e1f4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c6570 con 0x5568e3bc8770 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45164/0 (socket says 172.21.15.96:45164) 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 -- 172.21.15.96:0/3380766551 learned_addr learned my addr 172.21.15.96:0/3380766551 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd113640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 0x5568e3bccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35d81c37b0 con 0x5568e3bbbec0 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35e1f4d740 10 client.0 _ll_register_callbacks cb 0x7f35c8009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 -- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bc8770 msgr2=0x5568e3c44240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bc8770 0x5568e3c44240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 -- 172.21.15.96:0/3380766551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 msgr2=0x5568e3bccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 0x5568e3bccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 -- 172.21.15.96:0/3380766551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e3c44780 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd113640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 0x5568e3bccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.110+0000 7f35dd914640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4014bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2cda682cbc02e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35c17fa640 10 client.25174 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d00cdc40 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35c17fa640 1 -- 172.21.15.96:0/3380766551 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4028060 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35c17fa640 1 -- 172.21.15.96:0/3380766551 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35d4011b80 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35c17fa640 1 -- 172.21.15.96:0/3380766551 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35d4011d80 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 10 client.25174 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5568e3c458c0 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35c17fa640 1 -- 172.21.15.96:0/3380766551 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35d402d070 con 0x5568e3bbbec0 2024-09-10T22:48:13.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 10 client.25174 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 10 client.25174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 20 client.25174 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3c45e80 0x5568e3c482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 4 client.25174 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.111+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f35d00cdc40 con 0x5568e3c45e80 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.112+0000 7f35dd113640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3c45e80 0x5568e3c482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.113+0000 7f35dd113640 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3c45e80 0x5568e3c482a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccae25efc7c3e82b server_cookie=29d8b1fbfbb81511 in_seq=0 out_seq=0 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.113+0000 7f35c17fa640 10 client.25174 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35c17fa640 1 -- 172.21.15.96:0/3380766551 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f35d00cdc40 con 0x5568e3c45e80 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35c17fa640 10 client.25174 handle_command_reply: tid=0 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 client.25174 shutdown 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 msgr2=0x5568e3bcc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4014bc0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 shutdown_connections 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5568e3c45e80 0x5568e3c482a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568e3bc8770 0x5568e3c44240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568e3bbc7e0 0x5568e3bccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 --2- 172.21.15.96:0/3380766551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568e3bbbec0 0x5568e3bcc9d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 >> 172.21.15.96:0/3380766551 conn(0x5568e3bb9cf0 msgr2=0x5568e3bd3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.114+0000 7f35c17fa640 0 client.25174 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 shutdown_connections 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35e1f4d740 1 -- 172.21.15.96:0/3380766551 wait complete. 2024-09-10T22:48:13.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35e1f4d740 20 client.25174 trim_cache size 0 max 16384 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 msgr2=0x7f35c40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 0x7f35c40b7a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009fd0 tx=0x7f35c8012040 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 msgr2=0x7f35d8113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 0x7f35d8113760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35d002af40 tx=0x7f35d0003f10 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 shutdown_connections 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d8114810 0x7f35d8113760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35c40b5580 0x7f35c40b7a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d810bb80 0x7f35d810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 --2- 172.21.15.96:0/289336418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35d810b260 0x7f35d810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.115+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 >> 172.21.15.96:0/289336418 conn(0x7f35d806dfc0 msgr2=0x7f35d80716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.116+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 shutdown_connections 2024-09-10T22:48:13.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:13.116+0000 7f35dfb78640 1 -- 172.21.15.96:0/289336418 wait complete. 2024-09-10T22:48:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:13 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:48:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:13 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:13 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:48:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:13 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:48:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:13 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:13 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:48:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:13 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:48:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:13 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:13 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:48:14.341 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:48:14.633 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:14.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:14 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:48:14.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:14 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:14.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:14 smithi096 ceph-mon[30869]: pgmap v375: 97 pgs: 97 active+clean; 995 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 851 op/s 2024-09-10T22:48:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:14 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:48:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:14 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:14 smithi121 ceph-mon[35119]: pgmap v375: 97 pgs: 97 active+clean; 995 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 851 op/s 2024-09-10T22:48:14.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:14 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:48:14.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:14 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:14.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:14 smithi117 ceph-mon[35081]: pgmap v375: 97 pgs: 97 active+clean; 995 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 851 op/s 2024-09-10T22:48:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 -- 172.21.15.96:0/1159342528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48103370 msgr2=0x7f1f48109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1159342528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48103370 0x7f1f48109cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009c10 tx=0x7f1f4002a6f0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 -- 172.21.15.96:0/1159342528 shutdown_connections 2024-09-10T22:48:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1159342528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48103370 0x7f1f48109cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1159342528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f481029b0 0x7f1f48102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1159342528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f48101860 0x7f1f48101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 -- 172.21.15.96:0/1159342528 >> 172.21.15.96:0/1159342528 conn(0x7f1f480fd140 msgr2=0x7f1f480ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 -- 172.21.15.96:0/1159342528 shutdown_connections 2024-09-10T22:48:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.052+0000 7f1f4d239640 1 -- 172.21.15.96:0/1159342528 wait complete. 2024-09-10T22:48:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.053+0000 7f1f4d239640 1 Processor -- start 2024-09-10T22:48:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.053+0000 7f1f4d239640 1 -- start start 2024-09-10T22:48:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48101860 0x7f1f48111e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f481029b0 0x7f1f48112450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810c3c0 con 0x7f1f48103370 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810c240 con 0x7f1f481029b0 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4d239640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810c540 con 0x7f1f48101860 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48101860 0x7f1f48111e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43768/0 (socket says 172.21.15.96:43768) 2024-09-10T22:48:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 -- 172.21.15.96:0/1772468193 learned_addr learned my addr 172.21.15.96:0/1772468193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f481029b0 0x7f1f48112450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 -- 172.21.15.96:0/1772468193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48101860 msgr2=0x7f1f48111e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48101860 0x7f1f48111e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 -- 172.21.15.96:0/1772468193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f481029b0 msgr2=0x7f1f48112450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f481029b0 0x7f1f48112450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.054+0000 7f1f4759e640 1 -- 172.21.15.96:0/1772468193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4811a9e0 con 0x7f1f48103370 2024-09-10T22:48:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f4759e640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f40004af0 tx=0x7f1f40030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f44d99640 1 -- 172.21.15.96:0/1772468193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40041020 con 0x7f1f48103370 2024-09-10T22:48:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4811ac70 con 0x7f1f48103370 2024-09-10T22:48:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481132b0 con 0x7f1f48103370 2024-09-10T22:48:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f44d99640 1 -- 172.21.15.96:0/1772468193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f400318f0 con 0x7f1f48103370 2024-09-10T22:48:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.055+0000 7f1f44d99640 1 -- 172.21.15.96:0/1772468193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40036040 con 0x7f1f48103370 2024-09-10T22:48:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.058+0000 7f1f5000c740 1 Processor -- start 2024-09-10T22:48:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.058+0000 7f1f44d99640 1 -- 172.21.15.96:0/1772468193 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f1f400045e0 con 0x7f1f48103370 2024-09-10T22:48:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 -- start start 2024-09-10T22:48:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b58c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640e7ac86f0 con 0x5640e7b5bbd0 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640e7ab7b20 con 0x5640e7b59190 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f5000c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5640e7abb930 con 0x5640e7b56860 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.059+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b5e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43780/0 (socket says 172.21.15.96:43780) 2024-09-10T22:48:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f3ffff640 1 -- 172.21.15.96:0/2477737663 learned_addr learned my addr 172.21.15.96:0/2477737663 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b58c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 -- 172.21.15.96:0/2477737663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 msgr2=0x5640e7b58c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b58c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 -- 172.21.15.96:0/2477737663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 msgr2=0x5640e7b5e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b5e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.060+0000 7f1f46d9d640 1 -- 172.21.15.96:0/2477737663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e7a7d530 con 0x5640e7b59190 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.061+0000 7f1f44d99640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 0x7f1f340b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.061+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b5b580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800bbf0 tx=0x7f1f380107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208f08f39aedd595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.061+0000 7f1f44d99640 1 -- 172.21.15.96:0/1772468193 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f40106c90 con 0x7f1f48103370 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.061+0000 7f1f3d7fa640 1 -- 172.21.15.96:0/2477737663 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38004030 con 0x5640e7b59190 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.061+0000 7f1f3d7fa640 1 -- 172.21.15.96:0/2477737663 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f380035f0 con 0x5640e7b59190 2024-09-10T22:48:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.062+0000 7f1f3d7fa640 1 -- 172.21.15.96:0/2477737663 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38011680 con 0x5640e7b59190 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.062+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 0x7f1f340b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.062+0000 7f1f5000c740 1 -- 172.21.15.96:0/2477737663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 msgr2=0x5640e7b5b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.062+0000 7f1f5000c740 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b5b580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800bbf0 tx=0x7f1f380107a0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 -- 172.21.15.96:0/2477737663 shutdown_connections 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b5b580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b5e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 --2- 172.21.15.96:0/2477737663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b58c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 -- 172.21.15.96:0/2477737663 >> 172.21.15.96:0/2477737663 conn(0x5640e7b519b0 msgr2=0x5640e7b52740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 -- 172.21.15.96:0/2477737663 shutdown_connections 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 -- 172.21.15.96:0/2477737663 wait complete. 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 -- start start 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.063+0000 7f1f5000c740 1 Processor -- start 2024-09-10T22:48:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 0x7f1f340b7930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005c80 tx=0x7f1f30009980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b60620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640e79efab0 con 0x5640e7b5bbd0 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640e7a5d730 con 0x5640e7b59190 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5640e7b5ec00 con 0x5640e7b56860 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f5000c740 10 client.0 _ll_register_callbacks cb 0x5640e7ab16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45208/0 (socket says 172.21.15.96:45208) 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 -- 172.21.15.96:0/4030067466 learned_addr learned my addr 172.21.15.96:0/4030067466 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f4759e640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 -- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 msgr2=0x5640e7b60620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b60620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b60620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 -- 172.21.15.96:0/4030067466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 msgr2=0x5640e7b611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f46d9d640 1 -- 172.21.15.96:0/4030067466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e7819790 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.064+0000 7f1f4759e640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f38011010 tx=0x7f1f380039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383b3c5310fea5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38003fe0 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f380035f0 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f38004180 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f38004380 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 10 client.25183 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5640e7892620 con 0x5640e7b56860 2024-09-10T22:48:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f380035f0 con 0x5640e7b56860 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1f38004aa0 con 0x5640e7b56860 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 10 client.25183 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 10 client.25183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 20 client.25183 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7b6b630 0x5640e7b6da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 4 client.25183 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.065+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640e7aca640 con 0x5640e7b6b630 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.066+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7b6b630 0x5640e7b6da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.067+0000 7f1f2e7fc640 10 client.25183 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.067+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7b6b630 0x5640e7b6da50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db417b78e3200d5e server_cookie=8bbe6d3ed6f52c19 in_seq=0 out_seq=0 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/4030067466 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5640e7aca640 con 0x5640e7b6b630 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f2e7fc640 10 client.25183 handle_command_reply: tid=0 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 client.25183 shutdown 2024-09-10T22:48:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 msgr2=0x5640e7b5ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f38011010 tx=0x7f1f380039d0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 shutdown_connections 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7b6b630 0x5640e7b6da50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f2e7fc640 0 client.25183 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b59190 0x5640e7b60620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 --2- 172.21.15.96:0/4030067466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b56860 0x5640e7b5ff80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 >> 172.21.15.96:0/4030067466 conn(0x5640e7b519b0 msgr2=0x5640e7b50bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 shutdown_connections 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.068+0000 7f1f5000c740 1 -- 172.21.15.96:0/4030067466 wait complete. 2024-09-10T22:48:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.069+0000 7f1f5000c740 20 client.25183 trim_cache size 0 max 16384 2024-09-10T22:48:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.091+0000 7f1f5000c740 1 Processor -- start 2024-09-10T22:48:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.091+0000 7f1f5000c740 1 -- start start 2024-09-10T22:48:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 0x5640e7b54d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57352/0 (socket says 172.21.15.96:57352) 2024-09-10T22:48:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f46d9d640 1 -- 172.21.15.96:0/198062051 learned_addr learned my addr 172.21.15.96:0/198062051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b72340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 0x5640e7b54d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f4759e640 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b72340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800e310 con 0x5640e7b5bbd0 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800ff10 con 0x5640e7b56860 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.092+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800e9e0 con 0x5640e7b59190 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 -- 172.21.15.96:0/198062051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 msgr2=0x5640e7b547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 -- 172.21.15.96:0/198062051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 msgr2=0x5640e7b72340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b72340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 -- 172.21.15.96:0/198062051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e7b5b8f0 con 0x5640e7b59190 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 0x5640e7b54d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5640e7b6e270 tx=0x7f1f30018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3447f0f96d1c0d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3cff9640 1 -- 172.21.15.96:0/198062051 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30036070 con 0x5640e7b59190 2024-09-10T22:48:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3cff9640 1 -- 172.21.15.96:0/198062051 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3002bc30 con 0x5640e7b59190 2024-09-10T22:48:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.093+0000 7f1f3cff9640 1 -- 172.21.15.96:0/198062051 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30031d00 con 0x5640e7b59190 2024-09-10T22:48:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 msgr2=0x5640e7b54d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 0x5640e7b54d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5640e7b6e270 tx=0x7f1f30018000 comp rx=0 tx=0).stop 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 shutdown_connections 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b5bbd0 0x5640e7b72340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b59190 0x5640e7b54d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 --2- 172.21.15.96:0/198062051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 >> 172.21.15.96:0/198062051 conn(0x5640e7b519b0 msgr2=0x5640e7b50bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 shutdown_connections 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.094+0000 7f1f5000c740 1 -- 172.21.15.96:0/198062051 wait complete. 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 -- start start 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 Processor -- start 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b5bbd0 0x5640e7bd05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800f780 con 0x5640e7b59190 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800f0b0 con 0x5640e7b56860 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800c2f0 con 0x5640e7b5bbd0 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.095+0000 7f1f5000c740 10 client.0 _ll_register_callbacks cb 0x7f1f30009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57364/0 (socket says 172.21.15.96:57364) 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43812/0 (socket says 172.21.15.96:43812) 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f46d9d640 1 -- 172.21.15.96:0/701866745 learned_addr learned my addr 172.21.15.96:0/701866745 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f4759e640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b5bbd0 0x5640e7bd05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 -- 172.21.15.96:0/701866745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b5bbd0 msgr2=0x5640e7bd05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b5bbd0 0x5640e7bd05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 -- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 msgr2=0x5640e7b66fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 -- 172.21.15.96:0/701866745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e7bd0e00 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f3ffff640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1f3001e560 tx=0x7f1f30031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c49b953094ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30036070 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f30031c80 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.096+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3002bc30 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f3002b580 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 10 client.15600 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5640e7bd8250 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3002bc30 con 0x5640e7b59190 2024-09-10T22:48:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1f300046c0 con 0x5640e7b59190 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 20 client.15600 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7bd8cf0 0x5640e7bdb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 4 client.15600 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640e7ac60f0 con 0x5640e7bd8cf0 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.097+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7bd8cf0 0x5640e7bdb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.098+0000 7f1f46d9d640 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7bd8cf0 0x5640e7bdb110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0d714fd0e38e8 server_cookie=19f424456f2268dd in_seq=0 out_seq=0 2024-09-10T22:48:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.098+0000 7f1f2e7fc640 10 client.15600 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.163+0000 7f1f2e7fc640 1 -- 172.21.15.96:0/701866745 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640e7ac60f0 con 0x5640e7bd8cf0 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f2e7fc640 10 client.15600 handle_command_reply: tid=0 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 client.15600 shutdown 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 msgr2=0x5640e7b67660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1f3001e560 tx=0x7f1f30031460 comp rx=0 tx=0).stop 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 shutdown_connections 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5640e7bd8cf0 0x5640e7bdb110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640e7b5bbd0 0x5640e7bd05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e7b59190 0x5640e7b67660 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 --2- 172.21.15.96:0/701866745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640e7b56860 0x5640e7b66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 >> 172.21.15.96:0/701866745 conn(0x5640e7b519b0 msgr2=0x5640e7b537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f2e7fc640 0 client.15600 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 shutdown_connections 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 1 -- 172.21.15.96:0/701866745 wait complete. 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.164+0000 7f1f5000c740 20 client.15600 trim_cache size 0 max 16384 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:48:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:48:15.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:48:15.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 msgr2=0x7f1f340b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 0x7f1f340b7930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005c80 tx=0x7f1f30009980 comp rx=0 tx=0).stop 2024-09-10T22:48:15.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 msgr2=0x7f1f4811a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f40004af0 tx=0x7f1f40030e00 comp rx=0 tx=0).stop 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 shutdown_connections 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f481029b0 0x7f1f48112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f340b5470 0x7f1f340b7930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f48103370 0x7f1f4811a120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 --2- 172.21.15.96:0/1772468193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f48101860 0x7f1f48111e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 >> 172.21.15.96:0/1772468193 conn(0x7f1f480fd140 msgr2=0x7f1f48105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 shutdown_connections 2024-09-10T22:48:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:15.165+0000 7f1f4d239640 1 -- 172.21.15.96:0/1772468193 wait complete. 2024-09-10T22:48:15.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 -- 172.21.15.96:0/110471400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc108020 msgr2=0x7f97bc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 --2- 172.21.15.96:0/110471400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc108020 0x7f97bc10e820 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009ca0 tx=0x7f97b402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 -- 172.21.15.96:0/110471400 shutdown_connections 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 --2- 172.21.15.96:0/110471400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc108020 0x7f97bc10e820 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 --2- 172.21.15.96:0/110471400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc1076a0 0x7f97bc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 --2- 172.21.15.96:0/110471400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97bc101c90 0x7f97bc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 -- 172.21.15.96:0/110471400 >> 172.21.15.96:0/110471400 conn(0x7f97bc0fd700 msgr2=0x7f97bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 -- 172.21.15.96:0/110471400 shutdown_connections 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.186+0000 7f97c344e640 1 -- 172.21.15.96:0/110471400 wait complete. 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 Processor -- start 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 -- start start 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1076a0 0x7f97bc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97bc108020 0x7f97bc1a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc111ca0 con 0x7f97bc1076a0 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc111b20 con 0x7f97bc108020 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c344e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc111e20 con 0x7f97bc101c90 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45234/0 (socket says 172.21.15.96:45234) 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c11ea640 1 -- 172.21.15.96:0/2077620016 learned_addr learned my addr 172.21.15.96:0/2077620016 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.187+0000 7f97c09e9640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1076a0 0x7f97bc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 -- 172.21.15.96:0/2077620016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97bc108020 msgr2=0x7f97bc1a50b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97bc108020 0x7f97bc1a50b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 -- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1076a0 msgr2=0x7f97bc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1076a0 0x7f97bc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 -- 172.21.15.96:0/2077620016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc197e20 con 0x7f97bc101c90 2024-09-10T22:48:15.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97b800e9f0 tx=0x7f97b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97b27fc640 1 -- 172.21.15.96:0/2077620016 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b800cd80 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc198110 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97b27fc640 1 -- 172.21.15.96:0/2077620016 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b80108e0 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97b27fc640 1 -- 172.21.15.96:0/2077620016 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8017920 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.188+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1986a0 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.191+0000 7f97b27fc640 1 -- 172.21.15.96:0/2077620016 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7f97b8010430 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 Processor -- start 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 -- start start 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97b27fc640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 0x7f97a80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97b27fc640 1 -- 172.21.15.96:0/2077620016 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f97b80e6300 con 0x7f97bc101c90 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba23a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556ba230ce50 con 0x556ba2396da0 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ba22fc280 con 0x556ba2396480 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.192+0000 7f97c5823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556ba2300090 con 0x556ba23a2d30 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57390/0 (socket says 172.21.15.96:57390) 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 -- 172.21.15.96:0/583677371 learned_addr learned my addr 172.21.15.96:0/583677371 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c19eb640 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 -- 172.21.15.96:0/583677371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 msgr2=0x556ba23a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 -- 172.21.15.96:0/583677371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 msgr2=0x556ba23a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba23a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c11ea640 1 -- 172.21.15.96:0/583677371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba205a650 con 0x556ba2396480 2024-09-10T22:48:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.193+0000 7f97c19eb640 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.194+0000 7f97c11ea640 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ac8f0 tx=0x7f97b80ac0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f0540e127e3ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.194+0000 7f97c19eb640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 0x7f97a80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.194+0000 7f97a77fe640 1 -- 172.21.15.96:0/583677371 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80b1210 con 0x556ba2396480 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.194+0000 7f97a77fe640 1 -- 172.21.15.96:0/583677371 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b80ac960 con 0x556ba2396480 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.194+0000 7f97a77fe640 1 -- 172.21.15.96:0/583677371 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80b5a60 con 0x556ba2396480 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- 172.21.15.96:0/583677371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 msgr2=0x556ba2396860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ac8f0 tx=0x7f97b80ac0d0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c19eb640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 0x7f97a80b78f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97b402aeb0 tx=0x7f97b40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- 172.21.15.96:0/583677371 shutdown_connections 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba23a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 --2- 172.21.15.96:0/583677371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396480 0x556ba2396860 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- 172.21.15.96:0/583677371 >> 172.21.15.96:0/583677371 conn(0x556ba23942b0 msgr2=0x556ba2395450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- 172.21.15.96:0/583677371 shutdown_connections 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- 172.21.15.96:0/583677371 wait complete. 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 -- start start 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.195+0000 7f97c5823740 1 Processor -- start 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396da0 0x556ba239fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba23a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556ba215d700 con 0x556ba2396480 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ba2297110 con 0x556ba23a2d30 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556ba23a5a00 con 0x556ba2396da0 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c5823740 10 client.0 _ll_register_callbacks cb 0x556ba22b0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c09e9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396da0 0x556ba239fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43850/0 (socket says 172.21.15.96:43850) 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 -- 172.21.15.96:0/3089741518 learned_addr learned my addr 172.21.15.96:0/3089741518 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c19eb640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba23a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 -- 172.21.15.96:0/3089741518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396da0 msgr2=0x556ba239fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396da0 0x556ba239fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 -- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 msgr2=0x556ba23a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba23a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 -- 172.21.15.96:0/3089741518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba22c4c40 con 0x556ba2396480 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97c11ea640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ac660 tx=0x7f97b800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e302199db1cebba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b80b1210 con 0x556ba2396480 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.196+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f97b80abbc0 con 0x556ba2396480 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.197+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b80ac960 con 0x556ba2396480 2024-09-10T22:48:15.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.197+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97b80cd050 con 0x556ba2396480 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.197+0000 7f97c5823740 10 client.15609 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.197+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ba23b1340 con 0x556ba2396480 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.198+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b80ac960 con 0x556ba2396480 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f97b80ab630 con 0x556ba2396480 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 20 client.15609 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba23b2690 0x556ba23b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 4 client.15609 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.199+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba230eda0 con 0x556ba23b2690 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.200+0000 7f97c09e9640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba23b2690 0x556ba23b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.200+0000 7f97c09e9640 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba23b2690 0x556ba23b4a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1661d60288cb7cd5 server_cookie=b223d7c3d5c86ea in_seq=0 out_seq=0 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.200+0000 7f97a4ff9640 10 client.15609 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97a4ff9640 1 -- 172.21.15.96:0/3089741518 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ba230eda0 con 0x556ba23b2690 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97a4ff9640 10 client.15609 handle_command_reply: tid=0 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 client.15609 shutdown 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 msgr2=0x556ba23a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ac660 tx=0x7f97b800cf20 comp rx=0 tx=0).stop 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 shutdown_connections 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba23b2690 0x556ba23b4a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97a4ff9640 0 client.15609 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba23a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396da0 0x556ba239fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 --2- 172.21.15.96:0/3089741518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23a26c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 >> 172.21.15.96:0/3089741518 conn(0x556ba23942b0 msgr2=0x556ba2399e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 shutdown_connections 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 1 -- 172.21.15.96:0/3089741518 wait complete. 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.201+0000 7f97c5823740 20 client.15609 trim_cache size 0 max 16384 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 Processor -- start 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 -- start start 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 0x556ba238f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba239e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b04e0 con 0x556ba2396da0 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b20e0 con 0x556ba23a2d30 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c5823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b0bb0 con 0x556ba2396480 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c09e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c09e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43860/0 (socket says 172.21.15.96:43860) 2024-09-10T22:48:15.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c19eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba239e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 0x556ba238f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.224+0000 7f97c09e9640 1 -- 172.21.15.96:0/1972514850 learned_addr learned my addr 172.21.15.96:0/1972514850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 -- 172.21.15.96:0/1972514850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 msgr2=0x556ba238f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 0x556ba238f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 -- 172.21.15.96:0/1972514850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 msgr2=0x556ba239e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba239e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 -- 172.21.15.96:0/1972514850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba23a0df0 con 0x556ba2396da0 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c19eb640 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba239e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c11ea640 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 0x556ba238f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97c09e9640 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x556ba2395190 tx=0x7f97ac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a659fb2a242ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97a6ffd640 1 -- 172.21.15.96:0/1972514850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac004250 con 0x556ba2396da0 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97a6ffd640 1 -- 172.21.15.96:0/1972514850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0043f0 con 0x556ba2396da0 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.225+0000 7f97a6ffd640 1 -- 172.21.15.96:0/1972514850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac01c7c0 con 0x556ba2396da0 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 -- 172.21.15.96:0/1972514850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 msgr2=0x556ba2393bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x556ba2395190 tx=0x7f97ac010800 comp rx=0 tx=0).stop 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 -- 172.21.15.96:0/1972514850 shutdown_connections 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba23a2d30 0x556ba239e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396da0 0x556ba2393bc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 --2- 172.21.15.96:0/1972514850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba2396480 0x556ba238f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 -- 172.21.15.96:0/1972514850 >> 172.21.15.96:0/1972514850 conn(0x556ba23942b0 msgr2=0x556ba239ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.226+0000 7f97c5823740 1 -- 172.21.15.96:0/1972514850 shutdown_connections 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.227+0000 7f97c5823740 1 -- 172.21.15.96:0/1972514850 wait complete. 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.227+0000 7f97c5823740 1 -- start start 2024-09-10T22:48:15.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.227+0000 7f97c5823740 1 Processor -- start 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396da0 0x556ba23ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b1950 con 0x556ba2396480 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1b1280 con 0x556ba2396da0 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1ae4e0 con 0x556ba23a2d30 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c5823740 10 client.0 _ll_register_callbacks cb 0x7f97b80d2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c09e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396da0 0x556ba23ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c11ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43876/0 (socket says 172.21.15.96:43876) 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c11ea640 1 -- 172.21.15.96:0/2279139396 learned_addr learned my addr 172.21.15.96:0/2279139396 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:15.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 -- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396da0 msgr2=0x556ba23ab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396da0 0x556ba23ab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 -- 172.21.15.96:0/2279139396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 msgr2=0x556ba23aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c19eb640 1 -- 172.21.15.96:0/2279139396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba23a61e0 con 0x556ba23a2d30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.228+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c19eb640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97b402f540 tx=0x7f97b402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc451a1b085ef6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4003e40 con 0x556ba23a2d30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac0043f0 con 0x556ba23a2d30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f97b4004050 con 0x556ba23a2d30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97b4004250 con 0x556ba23a2d30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c5823740 10 client.25204 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:15.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ba241fb40 con 0x556ba23a2d30 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac0043f0 con 0x556ba23a2d30 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f97b40305c0 con 0x556ba23a2d30 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c5823740 10 client.25204 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.229+0000 7f97c5823740 10 client.25204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c5823740 20 client.25204 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba2420400 0x556ba2422820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c5823740 4 client.25204 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ba230a850 con 0x556ba2420400 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba2420400 0x556ba2422820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97c11ea640 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba2420400 0x556ba2422820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae65988fe67289b4 server_cookie=ddbeec9bcc47ad0a in_seq=0 out_seq=0 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.230+0000 7f97a4ff9640 10 client.25204 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97a4ff9640 1 -- 172.21.15.96:0/2279139396 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x556ba230a850 con 0x556ba2420400 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97a4ff9640 10 client.25204 handle_command_reply: tid=0 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97c5823740 1 client.25204 shutdown 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 msgr2=0x556ba23a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97b402f540 tx=0x7f97b402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:48:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "5c4cd589-68b1-4458-8ca3-09456aeee767", 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:48:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:48:15.279 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 shutdown_connections 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.231+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556ba2420400 0x556ba2422820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba2396da0 0x556ba23ab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556ba23a2d30 0x556ba23a5c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97a4ff9640 0 client.25204 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 --2- 172.21.15.96:0/2279139396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ba2396480 0x556ba23aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 >> 172.21.15.96:0/2279139396 conn(0x556ba23942b0 msgr2=0x556ba239e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 shutdown_connections 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 1 -- 172.21.15.96:0/2279139396 wait complete. 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c5823740 20 client.25204 trim_cache size 0 max 16384 2024-09-10T22:48:15.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 msgr2=0x7f97a80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 0x7f97a80b78f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97b402aeb0 tx=0x7f97b40095c0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 msgr2=0x7f97bc19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97b800e9f0 tx=0x7f97b800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 shutdown_connections 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97bc108020 0x7f97bc1a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97a80b5430 0x7f97a80b78f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1076a0 0x7f97bc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 --2- 172.21.15.96:0/2077620016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97bc101c90 0x7f97bc19a030 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.232+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 >> 172.21.15.96:0/2077620016 conn(0x7f97bc0fd700 msgr2=0x7f97bc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.233+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 shutdown_connections 2024-09-10T22:48:15.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:15.233+0000 7f97c344e640 1 -- 172.21.15.96:0/2077620016 wait complete. 2024-09-10T22:48:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:15 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:15 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:15 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:48:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 40.0 s, 3049.8 tps, lat 32.841 ms stddev 56.533 2024-09-10T22:48:16.017 INFO:tasks.cephfs.filesystem:scrub status for tag:5c4cd589-68b1-4458-8ca3-09456aeee767 - {'path': '/', 'tag': '5c4cd589-68b1-4458-8ca3-09456aeee767', 'options': 'recursive,force'} 2024-09-10T22:48:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:16 smithi096 ceph-mon[30869]: pgmap v376: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 874 op/s 2024-09-10T22:48:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:16 smithi121 ceph-mon[35119]: pgmap v376: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 874 op/s 2024-09-10T22:48:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:16 smithi117 ceph-mon[35081]: pgmap v376: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 874 op/s 2024-09-10T22:48:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:18 smithi096 ceph-mon[30869]: pgmap v377: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 772 op/s 2024-09-10T22:48:18.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:18 smithi121 ceph-mon[35119]: pgmap v377: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 772 op/s 2024-09-10T22:48:18.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:18 smithi117 ceph-mon[35081]: pgmap v377: 97 pgs: 97 active+clean; 994 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 772 op/s 2024-09-10T22:48:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:19 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:48:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:19 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:48:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:19 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:48:20.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:20 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:48:20.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:20 smithi096 ceph-mon[30869]: pgmap v378: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-09-10T22:48:20.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:20 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:48:20.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:20 smithi121 ceph-mon[35119]: pgmap v378: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-09-10T22:48:20.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:20 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:48:20.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:20 smithi117 ceph-mon[35081]: pgmap v378: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-09-10T22:48:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 45.0 s, 3772.0 tps, lat 26.453 ms stddev 25.870 2024-09-10T22:48:22.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:22 smithi096 ceph-mon[30869]: pgmap v379: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-10T22:48:22.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:22 smithi121 ceph-mon[35119]: pgmap v379: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-10T22:48:22.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:22 smithi117 ceph-mon[35081]: pgmap v379: 97 pgs: 97 active+clean; 997 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-09-10T22:48:24.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:24 smithi096 ceph-mon[30869]: pgmap v380: 97 pgs: 97 active+clean; 1001 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-09-10T22:48:24.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:24 smithi121 ceph-mon[35119]: pgmap v380: 97 pgs: 97 active+clean; 1001 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-09-10T22:48:24.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:24 smithi117 ceph-mon[35081]: pgmap v380: 97 pgs: 97 active+clean; 1001 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 14 MiB/s wr, 933 op/s 2024-09-10T22:48:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 50.0 s, 3480.6 tps, lat 28.671 ms stddev 29.694 2024-09-10T22:48:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:26 smithi096 ceph-mon[30869]: pgmap v381: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-09-10T22:48:26.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:26 smithi121 ceph-mon[35119]: pgmap v381: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-09-10T22:48:26.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:26 smithi117 ceph-mon[35081]: pgmap v381: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/1817907592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4101650 msgr2=0x7fc0b4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/1817907592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4101650 0x7fc0b4101ad0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009900 tx=0x7fc0a402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/1817907592 shutdown_connections 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/1817907592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4102210 0x7fc0b4106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/1817907592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4101650 0x7fc0b4101ad0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/1817907592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4100c80 0x7fc0b4101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.304+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/1817907592 >> 172.21.15.96:0/1817907592 conn(0x7fc0b40fc880 msgr2=0x7fc0b40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.305+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/1817907592 shutdown_connections 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.305+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/1817907592 wait complete. 2024-09-10T22:48:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.305+0000 7fc0ba9d3640 1 Processor -- start 2024-09-10T22:48:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 -- start start 2024-09-10T22:48:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4100c80 0x7fc0b4111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4101650 0x7fc0b4111d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4108a00 con 0x7fc0b4100c80 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4108880 con 0x7fc0b4101650 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0ba9d3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4108b80 con 0x7fc0b4102210 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0b8f70640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4100c80 0x7fc0b4111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0b8f70640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:33666/0 (socket says 172.21.15.96:33666) 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0b8f70640 1 -- 172.21.15.96:0/3092112702 learned_addr learned my addr 172.21.15.96:0/3092112702 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.306+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4101650 0x7fc0b4111d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 -- 172.21.15.96:0/3092112702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4101650 msgr2=0x7fc0b4111d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4101650 0x7fc0b4111d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 -- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4100c80 msgr2=0x7fc0b4111700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4100c80 0x7fc0b4111700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 -- 172.21.15.96:0/3092112702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b4110880 con 0x7fc0b4102210 2024-09-10T22:48:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00eb10 tx=0x7fc0ac00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b17fa640 1 -- 172.21.15.96:0/3092112702 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac004b20 con 0x7fc0b4102210 2024-09-10T22:48:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b4110b70 con 0x7fc0b4102210 2024-09-10T22:48:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b17fa640 1 -- 172.21.15.96:0/3092112702 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ac004cc0 con 0x7fc0b4102210 2024-09-10T22:48:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.307+0000 7fc0b17fa640 1 -- 172.21.15.96:0/3092112702 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac010690 con 0x7fc0b4102210 2024-09-10T22:48:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.308+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b4114340 con 0x7fc0b4102210 2024-09-10T22:48:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.311+0000 7fc0bd881740 1 Processor -- start 2024-09-10T22:48:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.312+0000 7fc0b17fa640 1 -- 172.21.15.96:0/3092112702 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 153301+0+0 (secure 0 0 0) 0x7fc0ac005000 con 0x7fc0b4102210 2024-09-10T22:48:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.312+0000 7fc0bd881740 1 -- start start 2024-09-10T22:48:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.312+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e48763c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.312+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 0x556e48766580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.312+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e48769090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0bd881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e486d36f0 con 0x556e48764190 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0bd881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e486c2b20 con 0x556e48761860 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0bd881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556e486c6930 con 0x556e48766bd0 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e48763c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 0x556e48766580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e48763c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:53204/0 (socket says 172.21.15.96:53204) 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b37fe640 1 -- 172.21.15.96:0/3102669432 learned_addr learned my addr 172.21.15.96:0/3102669432 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e48769090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3102669432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 msgr2=0x556e48769090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e48769090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3102669432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 msgr2=0x556e48763c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e48763c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3102669432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e48688530 con 0x556e48764190 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.313+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 0x556e48766580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00bbe0 tx=0x7fc09c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1853c77367481c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.314+0000 7fc0b17fa640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 0x7fc0a00d9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.314+0000 7fc0aa7fc640 1 -- 172.21.15.96:0/3102669432 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c015070 con 0x556e48764190 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.314+0000 7fc0aa7fc640 1 -- 172.21.15.96:0/3102669432 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c0035f0 con 0x556e48764190 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.314+0000 7fc0b17fa640 1 -- 172.21.15.96:0/3092112702 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0ac0e6290 con 0x7fc0b4102210 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.314+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 0x7fc0a00d9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0aa7fc640 1 -- 172.21.15.96:0/3102669432 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0116a0 con 0x556e48764190 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 -- 172.21.15.96:0/3102669432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 msgr2=0x556e48766580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 0x556e48766580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00bbe0 tx=0x7fc09c00f730 comp rx=0 tx=0).stop 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 -- 172.21.15.96:0/3102669432 shutdown_connections 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e48769090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48764190 0x556e48766580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3102669432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e48763c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0bd881740 1 -- 172.21.15.96:0/3102669432 >> 172.21.15.96:0/3102669432 conn(0x556e4875c9b0 msgr2=0x556e4875d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.315+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 0x7fc0a00d9900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40096c0 tx=0x7fc0a4005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.316+0000 7fc0bd881740 1 -- 172.21.15.96:0/3102669432 shutdown_connections 2024-09-10T22:48:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.316+0000 7fc0bd881740 1 -- 172.21.15.96:0/3102669432 wait complete. 2024-09-10T22:48:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.316+0000 7fc0bd881740 1 -- start start 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 Processor -- start 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 0x556e4876aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48764190 0x556e4876b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e4876c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e485faab0 con 0x556e48761860 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e48668730 con 0x556e48764190 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556e48769c00 con 0x556e48766bd0 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0bd881740 10 client.0 _ll_register_callbacks cb 0x556e486bc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 0x556e4876aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.317+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 0x556e4876aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52522/0 (socket says 172.21.15.96:52522) 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b3fff640 1 -- 172.21.15.96:0/1163514117 learned_addr learned my addr 172.21.15.96:0/1163514117 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e4876c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 -- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48764190 msgr2=0x556e4876b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48764190 0x556e4876b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 -- 172.21.15.96:0/1163514117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 msgr2=0x556e4876aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 0x556e4876aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 -- 172.21.15.96:0/1163514117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e48424790 con 0x556e48766bd0 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e4876c150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0e3d40 tx=0x7fc0ac0ad010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50d9b946c8f510b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.318+0000 7fc0937fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a80d0 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09c0035f0 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0ac0aab40 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0ac0ac500 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0bd881740 10 client.25216 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.319+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e4849d620 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ac0ad0f0 con 0x556e48766bd0 2024-09-10T22:48:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc0ac0a82b0 con 0x556e48766bd0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 10 client.25216 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 10 client.25216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 20 client.25216 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e48776ce0 0x556e48779120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 4 client.25216 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.320+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc09c0035f0 con 0x556e48776ce0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.321+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e48776ce0 0x556e48779120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.321+0000 7fc0937fe640 10 client.25216 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.321+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e48776ce0 0x556e48779120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b583bc1ce164c40 server_cookie=539d345b9d3d5150 in_seq=0 out_seq=0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.322+0000 7fc0937fe640 1 -- 172.21.15.96:0/1163514117 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc09c0035f0 con 0x556e48776ce0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.322+0000 7fc0937fe640 10 client.25216 handle_command_reply: tid=0 2024-09-10T22:48:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.322+0000 7fc0bd881740 1 client.25216 shutdown 2024-09-10T22:48:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 msgr2=0x556e4876c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e4876c150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0e3d40 tx=0x7fc0ac0ad010 comp rx=0 tx=0).stop 2024-09-10T22:48:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 shutdown_connections 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e48776ce0 0x556e48779120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48764190 0x556e4876b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48766bd0 0x556e4876c150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1163514117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48761860 0x556e4876aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 >> 172.21.15.96:0/1163514117 conn(0x556e4875c9b0 msgr2=0x556e4875bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0937fe640 0 client.25216 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 shutdown_connections 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 1 -- 172.21.15.96:0/1163514117 wait complete. 2024-09-10T22:48:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.323+0000 7fc0bd881740 20 client.25216 trim_cache size 0 max 16384 2024-09-10T22:48:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.346+0000 7fc0bd881740 1 Processor -- start 2024-09-10T22:48:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.346+0000 7fc0bd881740 1 -- start start 2024-09-10T22:48:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e48766fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411ce90 con 0x556e48766bd0 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411ea90 con 0x556e48761860 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0bd881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411d560 con 0x556e48764190 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:53220/0 (socket says 172.21.15.96:53220) 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3743158378 learned_addr learned my addr 172.21.15.96:0/3743158378 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b8f70640 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e48766fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3743158378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 msgr2=0x556e48766fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e48766fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3743158378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 msgr2=0x556e487674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.347+0000 7fc0b3fff640 1 -- 172.21.15.96:0/3743158378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e487668f0 con 0x556e48761860 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.348+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00fc30 tx=0x7fc09c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476338f9e8d7b617 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.348+0000 7fc0a9ffb640 1 -- 172.21.15.96:0/3743158378 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c015040 con 0x556e48761860 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.348+0000 7fc0a9ffb640 1 -- 172.21.15.96:0/3743158378 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c0054d0 con 0x556e48761860 2024-09-10T22:48:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.348+0000 7fc0a9ffb640 1 -- 172.21.15.96:0/3743158378 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0044b0 con 0x556e48761860 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 -- 172.21.15.96:0/3743158378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 msgr2=0x556e4876c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00fc30 tx=0x7fc09c0056d0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 -- 172.21.15.96:0/3743158378 shutdown_connections 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e48766fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 --2- 172.21.15.96:0/3743158378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876c130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.349+0000 7fc0bd881740 1 -- 172.21.15.96:0/3743158378 >> 172.21.15.96:0/3743158378 conn(0x556e4875c9b0 msgr2=0x556e4875d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.350+0000 7fc0bd881740 1 -- 172.21.15.96:0/3743158378 shutdown_connections 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.350+0000 7fc0bd881740 1 -- 172.21.15.96:0/3743158378 wait complete. 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.350+0000 7fc0bd881740 1 -- start start 2024-09-10T22:48:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.350+0000 7fc0bd881740 1 Processor -- start 2024-09-10T22:48:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e4876fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411e300 con 0x556e48766bd0 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411dc30 con 0x556e48761860 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b411aec0 con 0x556e48764190 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0bd881740 10 client.0 _ll_register_callbacks cb 0x7fc0ac005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e4876fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e4876fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:33720/0 (socket says 172.21.15.96:33720) 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0b37fe640 1 -- 172.21.15.96:0/1936274653 learned_addr learned my addr 172.21.15.96:0/1936274653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.351+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 -- 172.21.15.96:0/1936274653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 msgr2=0x556e4876fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e4876fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 -- 172.21.15.96:0/1936274653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 msgr2=0x556e487709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 -- 172.21.15.96:0/1936274653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e487711d0 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0b3fff640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876f840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b970 tx=0x7fc09c005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f994a3fab4e4a9b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.352+0000 7fc0937fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c015040 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc09c004400 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09c0054d0 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc09c005020 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0bd881740 10 client.25193 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e48772540 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.353+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c005280 con 0x556e48761860 2024-09-10T22:48:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc09c004a90 con 0x556e48761860 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 10 client.25193 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 10 client.25193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 20 client.25193 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e487e3c30 0x556e487e6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 4 client.25193 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc09c0054d0 con 0x556e487e3c30 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.354+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e487e3c30 0x556e487e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.355+0000 7fc0b37fe640 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e487e3c30 0x556e487e6020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa06fb7eb8e2f66 server_cookie=deb4ebd3f4fd1c36 in_seq=0 out_seq=0 2024-09-10T22:48:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.355+0000 7fc0937fe640 10 client.25193 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.371+0000 7fc0937fe640 1 -- 172.21.15.96:0/1936274653 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc09c0054d0 con 0x556e487e3c30 2024-09-10T22:48:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.371+0000 7fc0937fe640 10 client.25193 handle_command_reply: tid=0 2024-09-10T22:48:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.371+0000 7fc0bd881740 1 client.25193 shutdown 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 msgr2=0x556e4876f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876f840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b970 tx=0x7fc09c005ee0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 shutdown_connections 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556e487e3c30 0x556e487e6020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0937fe640 0 client.25193 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e48766bd0 0x556e487709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556e48764190 0x556e4876fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 --2- 172.21.15.96:0/1936274653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e48761860 0x556e4876f840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 >> 172.21.15.96:0/1936274653 conn(0x556e4875c9b0 msgr2=0x556e4875d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 shutdown_connections 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 1 -- 172.21.15.96:0/1936274653 wait complete. 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.374+0000 7fc0bd881740 20 client.25193 trim_cache size 0 max 16384 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:48:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 msgr2=0x7fc0a00d9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 0x7fc0a00d9900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40096c0 tx=0x7fc0a4005a40 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 msgr2=0x7fc0b41101e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00eb10 tx=0x7fc0ac00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 shutdown_connections 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0b4101650 0x7fc0b4111d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0a00d7440 0x7fc0a00d9900 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0b4102210 0x7fc0b41101e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 --2- 172.21.15.96:0/3092112702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0b4100c80 0x7fc0b4111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 >> 172.21.15.96:0/3092112702 conn(0x7fc0b40fc880 msgr2=0x7fc0b4104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 shutdown_connections 2024-09-10T22:48:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:28.375+0000 7fc0ba9d3640 1 -- 172.21.15.96:0/3092112702 wait complete. 2024-09-10T22:48:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:28 smithi096 ceph-mon[30869]: pgmap v382: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-09-10T22:48:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:28 smithi121 ceph-mon[35119]: pgmap v382: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-09-10T22:48:29.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:28 smithi117 ceph-mon[35081]: pgmap v382: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: pgmap v383: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 905 op/s 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:30 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: pgmap v383: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 905 op/s 2024-09-10T22:48:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T22:48:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:30 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: pgmap v383: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 905 op/s 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:30 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 55.0 s, 3617.2 tps, lat 27.742 ms stddev 31.007 2024-09-10T22:48:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:48:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:48:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:48:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:32 smithi117 ceph-mon[35081]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 744 op/s 2024-09-10T22:48:33.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:32 smithi096 ceph-mon[30869]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 744 op/s 2024-09-10T22:48:33.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:32 smithi121 ceph-mon[35119]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 744 op/s 2024-09-10T22:48:35.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:34 smithi096 ceph-mon[30869]: pgmap v385: 97 pgs: 97 active+clean; 1019 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 743 op/s 2024-09-10T22:48:35.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:34 smithi121 ceph-mon[35119]: pgmap v385: 97 pgs: 97 active+clean; 1019 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 743 op/s 2024-09-10T22:48:35.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:34 smithi117 ceph-mon[35081]: pgmap v385: 97 pgs: 97 active+clean; 1019 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 743 op/s 2024-09-10T22:48:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 60.0 s, 3876.2 tps, lat 25.674 ms stddev 26.823 2024-09-10T22:48:37.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:36 smithi096 ceph-mon[30869]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-10T22:48:37.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:36 smithi121 ceph-mon[35119]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-10T22:48:37.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:36 smithi117 ceph-mon[35081]: pgmap v386: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.3 MiB/s wr, 768 op/s 2024-09-10T22:48:39.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:38 smithi096 ceph-mon[30869]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 716 op/s 2024-09-10T22:48:39.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:38 smithi121 ceph-mon[35119]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 716 op/s 2024-09-10T22:48:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:38 smithi117 ceph-mon[35081]: pgmap v387: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 716 op/s 2024-09-10T22:48:40.827 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:40 smithi117 ceph-mon[35081]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-10T22:48:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 65.0 s, 3774.8 tps, lat 26.611 ms stddev 27.104 2024-09-10T22:48:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:40 smithi096 ceph-mon[30869]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-10T22:48:41.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:40 smithi121 ceph-mon[35119]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-09-10T22:48:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 -- 172.21.15.96:0/1118666679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c06bc90 msgr2=0x7f515c105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 --2- 172.21.15.96:0/1118666679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c06bc90 0x7f515c105540 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5150009900 tx=0x7f515002a410 comp rx=0 tx=0).stop 2024-09-10T22:48:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 -- 172.21.15.96:0/1118666679 shutdown_connections 2024-09-10T22:48:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 --2- 172.21.15.96:0/1118666679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 --2- 172.21.15.96:0/1118666679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c108680 0x7f515c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 --2- 172.21.15.96:0/1118666679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c06bc90 0x7f515c105540 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 -- 172.21.15.96:0/1118666679 >> 172.21.15.96:0/1118666679 conn(0x7f515c0fd130 msgr2=0x7f515c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 -- 172.21.15.96:0/1118666679 shutdown_connections 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.502+0000 7f5163241640 1 -- 172.21.15.96:0/1118666679 wait complete. 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 Processor -- start 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 -- start start 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 0x7f515c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f515c101d70 con 0x7f515c108680 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f515c101bf0 con 0x7f515c105ad0 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5163241640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f515c101ef0 con 0x7f515c06bc90 2024-09-10T22:48:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f51617de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 0x7f515c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f515bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.503+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35614/0 (socket says 172.21.15.96:35614) 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 -- 172.21.15.96:0/3562755884 learned_addr learned my addr 172.21.15.96:0/3562755884 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f515bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49378/0 (socket says 172.21.15.96:49378) 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 -- 172.21.15.96:0/3562755884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 msgr2=0x7f515c19b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 -- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 msgr2=0x7f515c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 0x7f515c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 -- 172.21.15.96:0/3562755884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c19ca70 con 0x7f515c06bc90 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f515bfff640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f51617de640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 0x7f515c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5160fdd640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5150000c00 tx=0x7f5150032c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5159ffb640 1 -- 172.21.15.96:0/3562755884 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515003f030 con 0x7f515c06bc90 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c19cd00 con 0x7f515c06bc90 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5159ffb640 1 -- 172.21.15.96:0/3562755884 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5150032e40 con 0x7f515c06bc90 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5159ffb640 1 -- 172.21.15.96:0/3562755884 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150035070 con 0x7f515c06bc90 2024-09-10T22:48:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.504+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a3430 con 0x7f515c06bc90 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.507+0000 7f5159ffb640 1 -- 172.21.15.96:0/3562755884 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f515003bad0 con 0x7f515c06bc90 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 Processor -- start 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 -- start start 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11ae9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 0x564a11aec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564a11a596f0 con 0x564a11aea190 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a11a48b20 con 0x564a11ae7860 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f516600e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a11a4c930 con 0x564a11aecbd0 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f5159ffb640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 0x7f514c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 0x564a11aec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35616/0 (socket says 172.21.15.96:35616) 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 0x564a11aec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60876/0 (socket says 172.21.15.96:60876) 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 -- 172.21.15.96:0/12585778 learned_addr learned my addr 172.21.15.96:0/12585778 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f51617de640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 0x7f514c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f51617de640 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11ae9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f5159ffb640 1 -- 172.21.15.96:0/3562755884 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5150106410 con 0x7f515c06bc90 2024-09-10T22:48:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 -- 172.21.15.96:0/12585778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 msgr2=0x564a11ae9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11ae9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 -- 172.21.15.96:0/12585778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 msgr2=0x564a11aec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 0x564a11aec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.508+0000 7f515bfff640 1 -- 172.21.15.96:0/12585778 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a11a0e530 con 0x564a11aecbd0 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.509+0000 7f515bfff640 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5148011540 tx=0x7f5148011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8487770c2115e0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.509+0000 7f51617de640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 0x7f514c0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5154009fd0 tx=0x7f5154005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.509+0000 7f5146ffd640 1 -- 172.21.15.96:0/12585778 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800f640 con 0x564a11aecbd0 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.509+0000 7f5146ffd640 1 -- 172.21.15.96:0/12585778 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514800fc90 con 0x564a11aecbd0 2024-09-10T22:48:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.509+0000 7f5146ffd640 1 -- 172.21.15.96:0/12585778 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148019cc0 con 0x564a11aecbd0 2024-09-10T22:48:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.510+0000 7f516600e740 1 -- 172.21.15.96:0/12585778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 msgr2=0x564a11aef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.510+0000 7f516600e740 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5148011540 tx=0x7f5148011a00 comp rx=0 tx=0).stop 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 -- 172.21.15.96:0/12585778 shutdown_connections 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aecbd0 0x564a11aef090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aea190 0x564a11aec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 --2- 172.21.15.96:0/12585778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11ae9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 -- 172.21.15.96:0/12585778 >> 172.21.15.96:0/12585778 conn(0x564a11ae29b0 msgr2=0x564a11ae3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 -- 172.21.15.96:0/12585778 shutdown_connections 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 -- 172.21.15.96:0/12585778 wait complete. 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 -- start start 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.511+0000 7f516600e740 1 Processor -- start 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11af0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564a11980ab0 con 0x564a11aecbd0 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a119ee730 con 0x564a11ae7860 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a11aefbb0 con 0x564a11aea190 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f516600e740 10 client.0 _ll_register_callbacks cb 0x564a11a426b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f51617de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35620/0 (socket says 172.21.15.96:35620) 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 -- 172.21.15.96:0/563264892 learned_addr learned my addr 172.21.15.96:0/563264892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f5160fdd640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11af0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 -- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 msgr2=0x564a11af0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11af0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 -- 172.21.15.96:0/563264892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 msgr2=0x564a11af2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 -- 172.21.15.96:0/563264892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a117aa790 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f51617de640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f515bfff640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51480097b0 tx=0x7f5148019bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e99f2035f59899e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.512+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800f640 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f514800f7e0 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514800fc90 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f514800f9e0 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f516600e740 10 client.25234 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.513+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a11823620 con 0x564a11aea190 2024-09-10T22:48:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514800fc90 con 0x564a11aea190 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5148004b00 con 0x564a11aea190 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 10 client.25234 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 10 client.25234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 20 client.25234 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11afeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 4 client.25234 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a11a5b640 con 0x564a11afc6c0 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.514+0000 7f5160fdd640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11afeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.515+0000 7f513bfff640 10 client.25234 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.515+0000 7f5160fdd640 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11afeb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b63afd36dcc7892 server_cookie=6ba78105d5731468 in_seq=0 out_seq=0 2024-09-10T22:48:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.516+0000 7f513bfff640 1 -- 172.21.15.96:0/563264892 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a11a5b640 con 0x564a11afc6c0 2024-09-10T22:48:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.516+0000 7f513bfff640 10 client.25234 handle_command_reply: tid=0 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.516+0000 7f516600e740 1 client.25234 shutdown 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.518+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 msgr2=0x564a11af1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.518+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51480097b0 tx=0x7f5148019bf0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.518+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 shutdown_connections 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11afeb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f513bfff640 0 client.25234 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af1650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 --2- 172.21.15.96:0/563264892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11af0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 >> 172.21.15.96:0/563264892 conn(0x564a11ae29b0 msgr2=0x564a11ae1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 shutdown_connections 2024-09-10T22:48:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 1 -- 172.21.15.96:0/563264892 wait complete. 2024-09-10T22:48:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.519+0000 7f516600e740 20 client.25234 trim_cache size 0 max 16384 2024-09-10T22:48:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.538+0000 7f516600e740 1 Processor -- start 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 -- start start 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11aed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ac010 con 0x564a11aecbd0 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1adc10 con 0x564a11ae7860 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f516600e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ac6e0 con 0x564a11aea190 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60898/0 (socket says 172.21.15.96:60898) 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 -- 172.21.15.96:0/1384605917 learned_addr learned my addr 172.21.15.96:0/1384605917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f515bfff640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f5160fdd640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11aed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 -- 172.21.15.96:0/1384605917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 msgr2=0x564a11af0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 -- 172.21.15.96:0/1384605917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 msgr2=0x564a11aed6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11aed6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 -- 172.21.15.96:0/1384605917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a11ae6a50 con 0x564a11aecbd0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f5160fdd640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11aed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.539+0000 7f51617de640 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5154012b50 tx=0x7f5154014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752ce56d58631a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.540+0000 7f51467fc640 1 -- 172.21.15.96:0/1384605917 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515402e070 con 0x564a11aecbd0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.540+0000 7f51467fc640 1 -- 172.21.15.96:0/1384605917 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154010db0 con 0x564a11aecbd0 2024-09-10T22:48:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.540+0000 7f51467fc640 1 -- 172.21.15.96:0/1384605917 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515401e5c0 con 0x564a11aecbd0 2024-09-10T22:48:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.541+0000 7f516600e740 1 -- 172.21.15.96:0/1384605917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 msgr2=0x564a11af14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.541+0000 7f516600e740 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5154012b50 tx=0x7f5154014040 comp rx=0 tx=0).stop 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.541+0000 7f516600e740 1 -- 172.21.15.96:0/1384605917 shutdown_connections 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af14f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11aea190 0x564a11af0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 --2- 172.21.15.96:0/1384605917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11ae7860 0x564a11aed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 -- 172.21.15.96:0/1384605917 >> 172.21.15.96:0/1384605917 conn(0x564a11ae29b0 msgr2=0x564a11aecfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 -- 172.21.15.96:0/1384605917 shutdown_connections 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 -- 172.21.15.96:0/1384605917 wait complete. 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 -- start start 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.542+0000 7f516600e740 1 Processor -- start 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 0x564a11af5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11aea190 0x564a11af65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ad480 con 0x564a11aecbd0 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 0x564a11af5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1acdb0 con 0x564a11aea190 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1a9ff0 con 0x564a11ae7860 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f5160fdd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 0x564a11af5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35644/0 (socket says 172.21.15.96:35644) 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f5160fdd640 1 -- 172.21.15.96:0/536672975 learned_addr learned my addr 172.21.15.96:0/536672975 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f516600e740 10 client.0 _ll_register_callbacks cb 0x7f5150031500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f515bfff640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11aea190 0x564a11af65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 -- 172.21.15.96:0/536672975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 msgr2=0x564a11af5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 0x564a11af5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 -- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11aea190 msgr2=0x564a11af65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11aea190 0x564a11af65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 -- 172.21.15.96:0/536672975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a11af79d0 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f51617de640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af71a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f515401fc50 tx=0x7f5154029640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb812c8beef41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.543+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515402e070 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5154005040 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5154010db0 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5154004e20 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f516600e740 10 client.15648 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a11af8cb0 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.544+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154010db0 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f515400f040 con 0x564a11aecbd0 2024-09-10T22:48:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 10 client.15648 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 20 client.15648 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11b6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 4 client.15648 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a11a570f0 con 0x564a11afc6c0 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f5160fdd640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11b6b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.545+0000 7f513bfff640 10 client.15648 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.546+0000 7f5160fdd640 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11b6b9d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae96b39595bb19 server_cookie=9647e6cf3be74c2d in_seq=0 out_seq=0 2024-09-10T22:48:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.566+0000 7f513bfff640 1 -- 172.21.15.96:0/536672975 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a11a570f0 con 0x564a11afc6c0 2024-09-10T22:48:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.566+0000 7f513bfff640 10 client.15648 handle_command_reply: tid=0 2024-09-10T22:48:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.566+0000 7f516600e740 1 client.15648 shutdown 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 msgr2=0x564a11af71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af71a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f515401fc50 tx=0x7f5154029640 comp rx=0 tx=0).stop 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 shutdown_connections 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564a11afc6c0 0x564a11b6b9d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a11aea190 0x564a11af65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a11aecbd0 0x564a11af71a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 --2- 172.21.15.96:0/536672975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a11ae7860 0x564a11af5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f513bfff640 0 client.15648 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 >> 172.21.15.96:0/536672975 conn(0x564a11ae29b0 msgr2=0x564a11af0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 shutdown_connections 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 1 -- 172.21.15.96:0/536672975 wait complete. 2024-09-10T22:48:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.567+0000 7f516600e740 20 client.15648 trim_cache size 0 max 16384 2024-09-10T22:48:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:48:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:48:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:48:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 msgr2=0x7f514c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 0x7f514c0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5154009fd0 tx=0x7f5154005bd0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 msgr2=0x7f515c19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5150000c00 tx=0x7f5150032c50 comp rx=0 tx=0).stop 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 shutdown_connections 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c105ad0 0x7f515c19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f514c0b52e0 0x7f514c0b77a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f515c108680 0x7f515c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 --2- 172.21.15.96:0/3562755884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c06bc90 0x7f515c19af30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 >> 172.21.15.96:0/3562755884 conn(0x7f515c0fd130 msgr2=0x7f515c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 shutdown_connections 2024-09-10T22:48:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:41.568+0000 7f5163241640 1 -- 172.21.15.96:0/3562755884 wait complete. 2024-09-10T22:48:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:42 smithi096 ceph-mon[30869]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-10T22:48:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:42 smithi121 ceph-mon[35119]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-10T22:48:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:42 smithi117 ceph-mon[35081]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-10T22:48:44.815 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:44 smithi121 ceph-mon[35119]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:48:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:44 smithi096 ceph-mon[30869]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:48:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:44 smithi117 ceph-mon[35081]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:48:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 70.0 s, 3663.2 tps, lat 27.283 ms stddev 27.568 2024-09-10T22:48:46.019 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:48:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:46.798 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:46 smithi096 ceph-mon[30869]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T22:48:46.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 -- 172.21.15.96:0/3927219476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 msgr2=0x7f4010102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 --2- 172.21.15.96:0/3927219476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f4010102070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4000009860 tx=0x7f400002a360 comp rx=0 tx=0).stop 2024-09-10T22:48:46.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 -- 172.21.15.96:0/3927219476 shutdown_connections 2024-09-10T22:48:46.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 --2- 172.21.15.96:0/3927219476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 --2- 172.21.15.96:0/3927219476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f4010107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 --2- 172.21.15.96:0/3927219476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f4010102070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 -- 172.21.15.96:0/3927219476 >> 172.21.15.96:0/3927219476 conn(0x7f40100fd700 msgr2=0x7f40100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 -- 172.21.15.96:0/3927219476 shutdown_connections 2024-09-10T22:48:46.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.916+0000 7f4014880640 1 -- 172.21.15.96:0/3927219476 wait complete. 2024-09-10T22:48:46.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.917+0000 7f4014880640 1 Processor -- start 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.917+0000 7f4014880640 1 -- start start 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f401019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4010111ca0 con 0x7f4010108020 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4010111b20 con 0x7f4010101c90 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4014880640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4010111e20 con 0x7f40101076a0 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f400e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f401019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48984/0 (socket says 172.21.15.96:48984) 2024-09-10T22:48:46.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 -- 172.21.15.96:0/2538177543 learned_addr learned my addr 172.21.15.96:0/2538177543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34944/0 (socket says 172.21.15.96:34944) 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 -- 172.21.15.96:0/2538177543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 msgr2=0x7f401019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f401019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 -- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 msgr2=0x7f401019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 -- 172.21.15.96:0/2538177543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40101a54b0 con 0x7f40101076a0 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f400ed9d640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f400e59c640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f401019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.918+0000 7f4007fff640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e9f0 tx=0x7f3ff800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.919+0000 7f40077fe640 1 -- 172.21.15.96:0/2538177543 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800cd80 con 0x7f40101076a0 2024-09-10T22:48:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.919+0000 7f40077fe640 1 -- 172.21.15.96:0/2538177543 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff80108e0 con 0x7f40101076a0 2024-09-10T22:48:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.919+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40101a57a0 con 0x7f40101076a0 2024-09-10T22:48:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.919+0000 7f40077fe640 1 -- 172.21.15.96:0/2538177543 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8018980 con 0x7f40101076a0 2024-09-10T22:48:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.919+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101981b0 con 0x7f40101076a0 2024-09-10T22:48:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.923+0000 7f40077fe640 1 -- 172.21.15.96:0/2538177543 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3ff8010430 con 0x7f40101076a0 2024-09-10T22:48:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.923+0000 7f4016c55740 1 Processor -- start 2024-09-10T22:48:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.923+0000 7f4016c55740 1 -- start start 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 0x55df7b8dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 0x55df7b8e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 0x55df7b8e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f40077fe640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 0x7f3ffc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 0x55df7b8e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 0x55df7b8e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34948/0 (socket says 172.21.15.96:34948) 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f40077fe640 1 -- 172.21.15.96:0/2538177543 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3ff80e5e80 con 0x7f40101076a0 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 0x55df7b8dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 0x55df7b8dac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48532/0 (socket says 172.21.15.96:48532) 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f400e59c640 1 -- 172.21.15.96:0/1679658570 learned_addr learned my addr 172.21.15.96:0/1679658570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df7b848e50 con 0x55df7b8db1d0 2024-09-10T22:48:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df7b838280 con 0x55df7b8da8b0 2024-09-10T22:48:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4007fff640 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 0x55df7b8e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.924+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df7b83c090 con 0x55df7b8e7160 2024-09-10T22:48:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f400ed9d640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 0x7f3ffc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 -- 172.21.15.96:0/1679658570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 msgr2=0x55df7b8dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 0x55df7b8dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 -- 172.21.15.96:0/1679658570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 msgr2=0x55df7b8e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 0x55df7b8e6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 -- 172.21.15.96:0/1679658570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7b596650 con 0x55df7b8e7160 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.925+0000 7f4007fff640 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 0x55df7b8e9550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80e3a70 tx=0x7f3ff80abfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7440e77ad160dce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.926+0000 7f400ed9d640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 0x7f3ffc0b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f400800e7c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.926+0000 7f4004ff9640 1 -- 172.21.15.96:0/1679658570 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80b5c70 con 0x55df7b8e7160 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.926+0000 7f4004ff9640 1 -- 172.21.15.96:0/1679658570 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff80a8a00 con 0x55df7b8e7160 2024-09-10T22:48:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.926+0000 7f4004ff9640 1 -- 172.21.15.96:0/1679658570 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80a80e0 con 0x55df7b8e7160 2024-09-10T22:48:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 msgr2=0x55df7b8e9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 0x55df7b8e9550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80e3a70 tx=0x7f3ff80abfc0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 shutdown_connections 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8e7160 0x55df7b8e9550 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8db1d0 0x55df7b8e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 --2- 172.21.15.96:0/1679658570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8da8b0 0x55df7b8dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 >> 172.21.15.96:0/1679658570 conn(0x55df7b8d86e0 msgr2=0x55df7b8d9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 shutdown_connections 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- 172.21.15.96:0/1679658570 wait complete. 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 -- start start 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.927+0000 7f4016c55740 1 Processor -- start 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8da8b0 0x55df7b8e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8db1d0 0x55df7b8e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df7b699700 con 0x55df7b8e7160 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df7b7d3110 con 0x55df7b8db1d0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df7b8e9d80 con 0x55df7b8da8b0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34952/0 (socket says 172.21.15.96:34952) 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 -- 172.21.15.96:0/1349519409 learned_addr learned my addr 172.21.15.96:0/1349519409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400e59c640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8da8b0 0x55df7b8e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4007fff640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8db1d0 0x55df7b8e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f4016c55740 10 client.0 _ll_register_callbacks cb 0x55df7b7ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 -- 172.21.15.96:0/1349519409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8da8b0 msgr2=0x55df7b8e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8da8b0 0x55df7b8e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 -- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8db1d0 msgr2=0x55df7b8e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8db1d0 0x55df7b8e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.928+0000 7f400ed9d640 1 -- 172.21.15.96:0/1349519409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7b800c40 con 0x55df7b8e7160 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.929+0000 7f400ed9d640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4008004280 tx=0x7f400801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbb8572237f3e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.929+0000 7f3ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.929+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008029060 con 0x55df7b8e7160 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.929+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff80a8a00 con 0x55df7b8e7160 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.930+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4008004800 con 0x55df7b8e7160 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.930+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f400801dbe0 con 0x55df7b8e7160 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.930+0000 7f4016c55740 10 client.15660 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.930+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df7b8f5780 con 0x55df7b8e7160 2024-09-10T22:48:46.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff80a8a00 con 0x55df7b8e7160 2024-09-10T22:48:46.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f400802e070 con 0x55df7b8e7160 2024-09-10T22:48:46.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 20 client.15660 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b8f7190 0x55df7b8f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 4 client.15660 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.931+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df7b84ada0 con 0x55df7b8f7190 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.932+0000 7f400e59c640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b8f7190 0x55df7b8f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.932+0000 7f400e59c640 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b8f7190 0x55df7b8f95f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd5625a65052d27 server_cookie=f79dbffd4d1c25 in_seq=0 out_seq=0 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.932+0000 7f3ff5ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:46.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/1349519409 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df7b84ada0 con 0x55df7b8f7190 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f3ff5ffb640 10 client.15660 handle_command_reply: tid=0 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 client.15660 shutdown 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 msgr2=0x55df7b8e4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4008004280 tx=0x7f400801d000 comp rx=0 tx=0).stop 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 shutdown_connections 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b8f7190 0x55df7b8f95f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8db1d0 0x55df7b8e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8e7160 0x55df7b8e4610 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 --2- 172.21.15.96:0/1349519409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8da8b0 0x55df7b8e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 >> 172.21.15.96:0/1349519409 conn(0x55df7b8d86e0 msgr2=0x55df7b8de280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.933+0000 7f3ff5ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.934+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 shutdown_connections 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.934+0000 7f4016c55740 1 -- 172.21.15.96:0/1349519409 wait complete. 2024-09-10T22:48:46.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.935+0000 7f4016c55740 20 client.15660 trim_cache size 0 max 16384 2024-09-10T22:48:46.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 Processor -- start 2024-09-10T22:48:46.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 -- start start 2024-09-10T22:48:46.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ac1d0 con 0x55df7b8da8b0 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40101addd0 con 0x55df7b8e7160 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f4016c55740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ac8a0 con 0x55df7b8db1d0 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34960/0 (socket says 172.21.15.96:34960) 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.961+0000 7f400e59c640 1 -- 172.21.15.96:0/3246600119 learned_addr learned my addr 172.21.15.96:0/3246600119 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f4007fff640 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400ed9d640 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 -- 172.21.15.96:0/3246600119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 msgr2=0x55df7b8f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 -- 172.21.15.96:0/3246600119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 msgr2=0x55df7b8e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 -- 172.21.15.96:0/3246600119 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7b8db000 con 0x55df7b8da8b0 2024-09-10T22:48:46.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.962+0000 7f400e59c640 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4000005cd0 tx=0x7f4000000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c445937058fb47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.963+0000 7f3ff7fff640 1 -- 172.21.15.96:0/3246600119 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400004e070 con 0x55df7b8da8b0 2024-09-10T22:48:46.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.963+0000 7f3ff7fff640 1 -- 172.21.15.96:0/3246600119 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40000307f0 con 0x55df7b8da8b0 2024-09-10T22:48:46.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.963+0000 7f3ff7fff640 1 -- 172.21.15.96:0/3246600119 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000037040 con 0x55df7b8da8b0 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 -- 172.21.15.96:0/3246600119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 msgr2=0x55df7b8d9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4000005cd0 tx=0x7f4000000bc0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 -- 172.21.15.96:0/3246600119 shutdown_connections 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 --2- 172.21.15.96:0/3246600119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8d9f20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.965+0000 7f4016c55740 1 -- 172.21.15.96:0/3246600119 >> 172.21.15.96:0/3246600119 conn(0x55df7b8d86e0 msgr2=0x55df7b8d9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- 172.21.15.96:0/3246600119 shutdown_connections 2024-09-10T22:48:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- 172.21.15.96:0/3246600119 wait complete. 2024-09-10T22:48:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- start start 2024-09-10T22:48:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 Processor -- start 2024-09-10T22:48:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ad640 con 0x55df7b8da8b0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40101acf70 con 0x55df7b8e7160 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40101aa1b0 con 0x55df7b8db1d0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4016c55740 10 client.0 _ll_register_callbacks cb 0x7f4008003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4007fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4007fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49022/0 (socket says 172.21.15.96:49022) 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f400e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34976/0 (socket says 172.21.15.96:34976) 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.966+0000 7f4007fff640 1 -- 172.21.15.96:0/3101433635 learned_addr learned my addr 172.21.15.96:0/3101433635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f400ed9d640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 -- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 msgr2=0x55df7b8fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 -- 172.21.15.96:0/3101433635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 msgr2=0x55df7b8faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 -- 172.21.15.96:0/3101433635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7b963230 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f400ed9d640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f400e59c640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4007fff640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80ac720 tx=0x7f3ff8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25ad0b58710ffb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f3ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80b11c0 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3ff80c7420 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40000307f0 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ff8018b20 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4016c55740 10 client.25258 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df7b964570 con 0x55df7b8db1d0 2024-09-10T22:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.967+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40000307f0 con 0x55df7b8db1d0 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3ff80b2350 con 0x55df7b8db1d0 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 10 client.25258 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 10 client.25258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 20 client.25258 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b964a10 0x55df7b966e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 4 client.25258 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.968+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df7b846850 con 0x55df7b964a10 2024-09-10T22:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.969+0000 7f400e59c640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b964a10 0x55df7b966e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.969+0000 7f400e59c640 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b964a10 0x55df7b966e00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60406c52ca0f3252 server_cookie=e538a7c7c03f536d in_seq=0 out_seq=0 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.969+0000 7f3ff5ffb640 10 client.25258 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f3ff5ffb640 1 -- 172.21.15.96:0/3101433635 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55df7b846850 con 0x55df7b964a10 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f3ff5ffb640 10 client.25258 handle_command_reply: tid=0 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 client.25258 shutdown 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 msgr2=0x55df7b8fb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80ac720 tx=0x7f3ff8009310 comp rx=0 tx=0).stop 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 shutdown_connections 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55df7b964a10 0x55df7b966e00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df7b8e7160 0x55df7b8fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df7b8db1d0 0x55df7b8fb460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 --2- 172.21.15.96:0/3101433635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df7b8da8b0 0x55df7b8faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f3ff5ffb640 0 client.25258 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 >> 172.21.15.96:0/3101433635 conn(0x55df7b8d86e0 msgr2=0x55df7b8e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 shutdown_connections 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 1 -- 172.21.15.96:0/3101433635 wait complete. 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.970+0000 7f4016c55740 20 client.25258 trim_cache size 0 max 16384 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:48:46.976 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:48:46.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:48:46.978 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 msgr2=0x7f3ffc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 0x7f3ffc0b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f400800e7c0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 msgr2=0x7f401019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e9f0 tx=0x7f3ff800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 shutdown_connections 2024-09-10T22:48:46.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3ffc0b52e0 0x7f3ffc0b77a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4010108020 0x7f401019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40101076a0 0x7f401019a5f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 --2- 172.21.15.96:0/2538177543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4010101c90 0x7f401019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 >> 172.21.15.96:0/2538177543 conn(0x7f40100fd700 msgr2=0x7f40100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 shutdown_connections 2024-09-10T22:48:46.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:46.971+0000 7f4014880640 1 -- 172.21.15.96:0/2538177543 wait complete. 2024-09-10T22:48:47.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:46 smithi121 ceph-mon[35119]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T22:48:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:46 smithi117 ceph-mon[35081]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T22:48:47.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:48:47.891 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:48:48.263 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:48.692 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:48 smithi096 ceph-mon[30869]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.800+0000 7f13bdd3a640 1 -- 172.21.15.96:0/755531058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 msgr2=0x7f13b81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 --2- 172.21.15.96:0/755531058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b81031a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009ca0 tx=0x7f13a802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 -- 172.21.15.96:0/755531058 shutdown_connections 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 --2- 172.21.15.96:0/755531058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8109150 0x7f13b810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 --2- 172.21.15.96:0/755531058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b81087d0 0x7f13b8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 --2- 172.21.15.96:0/755531058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b81031a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 -- 172.21.15.96:0/755531058 >> 172.21.15.96:0/755531058 conn(0x7f13b80fe870 msgr2=0x7f13b8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 -- 172.21.15.96:0/755531058 shutdown_connections 2024-09-10T22:48:48.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 -- 172.21.15.96:0/755531058 wait complete. 2024-09-10T22:48:48.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.801+0000 7f13bdd3a640 1 Processor -- start 2024-09-10T22:48:48.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 -- start start 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 0x7f13b81a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8115000 con 0x7f13b81087d0 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8114e80 con 0x7f13b8102dc0 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.802+0000 7f13bdd3a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8115180 con 0x7f13b8109150 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34992/0 (socket says 172.21.15.96:34992) 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 -- 172.21.15.96:0/4289026701 learned_addr learned my addr 172.21.15.96:0/4289026701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b77fe640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b7fff640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 0x7f13b81a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 -- 172.21.15.96:0/4289026701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 msgr2=0x7f13b81a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 0x7f13b81a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 -- 172.21.15.96:0/4289026701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 msgr2=0x7f13b819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.803+0000 7f13b6ffd640 1 -- 172.21.15.96:0/4289026701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b81a1590 con 0x7f13b81087d0 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.804+0000 7f13b77fe640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.804+0000 7f13b6ffd640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13a000e9f0 tx=0x7f13a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.804+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a000cd80 con 0x7f13b81087d0 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.804+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a00108e0 con 0x7f13b81087d0 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.804+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0018980 con 0x7f13b81087d0 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.805+0000 7f13b7fff640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 0x7f13b81a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.805+0000 7f13bdd3a640 1 -- 172.21.15.96:0/4289026701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b81a1880 con 0x7f13b81087d0 2024-09-10T22:48:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.806+0000 7f13bdd3a640 1 -- 172.21.15.96:0/4289026701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b81a7f40 con 0x7f13b81087d0 2024-09-10T22:48:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.807+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c005290 con 0x7f13b81087d0 2024-09-10T22:48:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.810+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f13a0010430 con 0x7f13b81087d0 2024-09-10T22:48:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.811+0000 7f13b4ff9640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 0x7f13a40b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:48.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.811+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13a00e60a0 con 0x7f13b81087d0 2024-09-10T22:48:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.812+0000 7f13b77fe640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 0x7f13a40b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:48.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.813+0000 7f13b77fe640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 0x7f13a40b7970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009430 tx=0x7f13a8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:48.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:48.823+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f13a00acf60 con 0x7f13b81087d0 2024-09-10T22:48:49.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:48 smithi121 ceph-mon[35119]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-09-10T22:48:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:48 smithi117 ceph-mon[35081]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 828 op/s 2024-09-10T22:48:49.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.143+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f139c005580 con 0x7f13b81087d0 2024-09-10T22:48:49.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.143+0000 7f13b4ff9640 1 -- 172.21.15.96:0/4289026701 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f13a00ac860 con 0x7f13b81087d0 2024-09-10T22:48:49.145 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:48:49.145 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":30,"btime":"2024-09-10T22:47:34:397648+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:47:34.250205+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:48:49.146 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 30 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.146+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 msgr2=0x7f13a40b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.146+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 0x7f13a40b7970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009430 tx=0x7f13a8009360 comp rx=0 tx=0).stop 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.146+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 msgr2=0x7f13b819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.146+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f13a000e9f0 tx=0x7f13a000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 shutdown_connections 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13a40b54b0 0x7f13a40b7970 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13b8109150 0x7f13b81a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b81087d0 0x7f13b819ba20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 --2- 172.21.15.96:0/4289026701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b8102dc0 0x7f13b819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:49.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 >> 172.21.15.96:0/4289026701 conn(0x7f13b80fe870 msgr2=0x7f13b80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:49.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 shutdown_connections 2024-09-10T22:48:49.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:49.147+0000 7f13ae7fc640 1 -- 172.21.15.96:0/4289026701 wait complete. 2024-09-10T22:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4289026701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:49.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:48:50.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:49 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4289026701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:49 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4289026701' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:48:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 75.0 s, 3749.6 tps, lat 26.521 ms stddev 28.043 2024-09-10T22:48:51.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:50 smithi096 ceph-mon[30869]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-10T22:48:51.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:50 smithi121 ceph-mon[35119]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-10T22:48:51.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:50 smithi117 ceph-mon[35081]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-09-10T22:48:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.467+0000 7fa704835640 1 -- 172.21.15.96:0/801095358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa700101a90 msgr2=0x7fa700101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.467+0000 7fa704835640 1 --2- 172.21.15.96:0/801095358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa700101a90 0x7fa700101e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009900 tx=0x7fa6f002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 -- 172.21.15.96:0/801095358 shutdown_connections 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 --2- 172.21.15.96:0/801095358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7001023b0 0x7fa700106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 --2- 172.21.15.96:0/801095358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700106ed0 0x7fa700107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 --2- 172.21.15.96:0/801095358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa700101a90 0x7fa700101e70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 -- 172.21.15.96:0/801095358 >> 172.21.15.96:0/801095358 conn(0x7fa7000fd700 msgr2=0x7fa7000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 -- 172.21.15.96:0/801095358 shutdown_connections 2024-09-10T22:48:51.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.468+0000 7fa704835640 1 -- 172.21.15.96:0/801095358 wait complete. 2024-09-10T22:48:51.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 Processor -- start 2024-09-10T22:48:51.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 -- start start 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 0x7fa70019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 0x7fa7001a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa700119410 con 0x7fa7001023b0 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa700119290 con 0x7fa700106ed0 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa704835640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa700119590 con 0x7fa700101a90 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 0x7fa70019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49050/0 (socket says 172.21.15.96:49050) 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.469+0000 7fa6fe59c640 1 -- 172.21.15.96:0/1244090644 learned_addr learned my addr 172.21.15.96:0/1244090644 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 0x7fa7001a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1244090644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 msgr2=0x7fa70019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1244090644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 msgr2=0x7fa7001a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 0x7fa7001a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1244090644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7001a59b0 con 0x7fa7001023b0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 0x7fa7001a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 0x7fa70019fe40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c940 tx=0x7fa6f400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6ef7fe640 1 -- 172.21.15.96:0/1244090644 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007bb0 con 0x7fa7001023b0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6ef7fe640 1 -- 172.21.15.96:0/1244090644 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f4007d50 con 0x7fa7001023b0 2024-09-10T22:48:51.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7001a5ca0 con 0x7fa7001023b0 2024-09-10T22:48:51.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa6ef7fe640 1 -- 172.21.15.96:0/1244090644 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4018660 con 0x7fa7001023b0 2024-09-10T22:48:51.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.470+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7001ac360 con 0x7fa7001023b0 2024-09-10T22:48:51.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 Processor -- start 2024-09-10T22:48:51.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 -- start start 2024-09-10T22:48:51.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a465ea9ea0 con 0x55a465f330f0 2024-09-10T22:48:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a465e992d0 con 0x55a465f3f9a0 2024-09-10T22:48:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.476+0000 7fa706c0a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a465e9d0e0 con 0x55a465f33a10 2024-09-10T22:48:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.477+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.477+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48586/0 (socket says 172.21.15.96:48586) 2024-09-10T22:48:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.477+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/2953167281 learned_addr learned my addr 172.21.15.96:0/2953167281 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/2953167281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 msgr2=0x55a465f3f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/2953167281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 msgr2=0x55a465f334d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f334d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/2953167281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a465bf7650 con 0x55a465f3f9a0 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f334d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.478+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40e9b90 tx=0x7fa6f40e9c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd2dc6a3bbf9c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.479+0000 7fa6ecff9640 1 -- 172.21.15.96:0/2953167281 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40ef040 con 0x55a465f3f9a0 2024-09-10T22:48:51.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.479+0000 7fa6ecff9640 1 -- 172.21.15.96:0/2953167281 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40eee00 con 0x55a465f3f9a0 2024-09-10T22:48:51.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.479+0000 7fa6ecff9640 1 -- 172.21.15.96:0/2953167281 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40f3e00 con 0x55a465f3f9a0 2024-09-10T22:48:51.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.479+0000 7fa6ef7fe640 1 -- 172.21.15.96:0/1244090644 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa6f400f450 con 0x7fa7001023b0 2024-09-10T22:48:51.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 -- 172.21.15.96:0/2953167281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 msgr2=0x55a465f41d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40e9b90 tx=0x7fa6f40e9c80 comp rx=0 tx=0).stop 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 -- 172.21.15.96:0/2953167281 shutdown_connections 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f41d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 --2- 172.21.15.96:0/2953167281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f334d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 -- 172.21.15.96:0/2953167281 >> 172.21.15.96:0/2953167281 conn(0x55a465f30f20 msgr2=0x55a465f320c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 -- 172.21.15.96:0/2953167281 shutdown_connections 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa706c0a740 1 -- 172.21.15.96:0/2953167281 wait complete. 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.480+0000 7fa6ef7fe640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 0x7fa6e80b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 -- start start 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6ef7fe640 1 -- 172.21.15.96:0/1244090644 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa6f40e5ab0 con 0x7fa7001023b0 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 Processor -- start 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f3f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f3ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a465cfa700 con 0x55a465f330f0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a465e34160 con 0x55a465f3f9a0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a465f53680 con 0x55a465f33a10 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa706c0a740 10 client.0 _ll_register_callbacks cb 0x55a465e4d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f3ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49060/0 (socket says 172.21.15.96:49060) 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1789742818 learned_addr learned my addr 172.21.15.96:0/1789742818 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 0x7fa6e80b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.481+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f3f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 msgr2=0x55a465f3ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f3ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1789742818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 msgr2=0x55a465f3f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f3f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/1789742818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a465e61c90 con 0x55a465f33a10 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f3ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ffd0 tx=0x7fa6f400fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a2f8d18872ec14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a88c0 con 0x55a465f33a10 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f40eee00 con 0x55a465f33a10 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa6f40b5570 con 0x55a465f33a10 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6f40ab9e0 con 0x55a465f33a10 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 0x7fa6e80b7740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a465f31e00 tx=0x7fa6f00039a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa706c0a740 10 client.25267 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.482+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a465f5efc0 con 0x55a465f33a10 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f40cf2e0 con 0x55a465f33a10 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa6f40f2070 con 0x55a465f33a10 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 10 client.25267 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 10 client.25267 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 20 client.25267 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465f60310 0x55a465f62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 4 client.25267 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa6f40eee00 con 0x55a465f60310 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.483+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465f60310 0x55a465f62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.484+0000 7fa6e5ffb640 10 client.25267 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.484+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465f60310 0x55a465f62700 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3bdef5b266586dd server_cookie=f79723a568f1ce71 in_seq=0 out_seq=0 2024-09-10T22:48:51.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.484+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/1789742818 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa6f40eee00 con 0x55a465f60310 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.484+0000 7fa6e5ffb640 10 client.25267 handle_command_reply: tid=0 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.484+0000 7fa706c0a740 1 client.25267 shutdown 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 msgr2=0x55a465f3c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ffd0 tx=0x7fa6f400fcd0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 shutdown_connections 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465f60310 0x55a465f62700 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f3f9a0 0x55a465f3ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f33a10 0x55a465f3c780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 --2- 172.21.15.96:0/1789742818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f330f0 0x55a465f3f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 >> 172.21.15.96:0/1789742818 conn(0x55a465f30f20 msgr2=0x55a465f36a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa6e5ffb640 0 client.25267 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 shutdown_connections 2024-09-10T22:48:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 1 -- 172.21.15.96:0/1789742818 wait complete. 2024-09-10T22:48:51.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.485+0000 7fa706c0a740 20 client.25267 trim_cache size 0 max 16384 2024-09-10T22:48:51.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.506+0000 7fa706c0a740 1 Processor -- start 2024-09-10T22:48:51.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.506+0000 7fa706c0a740 1 -- start start 2024-09-10T22:48:51.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.506+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 0x55a465f3c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 0x55a465f3bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 0x55a465f30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 0x55a465f3c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 0x55a465f3c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48600/0 (socket says 172.21.15.96:48600) 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 0x55a465f3bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 0x55a465f3bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35058/0 (socket says 172.21.15.96:35058) 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fe59c640 1 -- 172.21.15.96:0/134566066 learned_addr learned my addr 172.21.15.96:0/134566066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa700111300 con 0x55a465f33a10 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001add40 con 0x55a465f330f0 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.507+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 0x55a465f30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001ac810 con 0x55a465f3f9a0 2024-09-10T22:48:51.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 -- 172.21.15.96:0/134566066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 msgr2=0x55a465f3c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 0x55a465f3c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 -- 172.21.15.96:0/134566066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 msgr2=0x55a465f3bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 0x55a465f3bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 -- 172.21.15.96:0/134566066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a465f33840 con 0x55a465f3f9a0 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6fed9d640 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 0x55a465f30760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800aaa0 tx=0x7fa6f8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f933e66b996146 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6e7fff640 1 -- 172.21.15.96:0/134566066 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f801d030 con 0x55a465f3f9a0 2024-09-10T22:48:51.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6e7fff640 1 -- 172.21.15.96:0/134566066 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f8013c60 con 0x55a465f3f9a0 2024-09-10T22:48:51.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.508+0000 7fa6e7fff640 1 -- 172.21.15.96:0/134566066 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8017670 con 0x55a465f3f9a0 2024-09-10T22:48:51.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 msgr2=0x55a465f30760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 0x55a465f30760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800aaa0 tx=0x7fa6f8009650 comp rx=0 tx=0).stop 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 shutdown_connections 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f3f9a0 0x55a465f30760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f33a10 0x55a465f3bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 --2- 172.21.15.96:0/134566066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f330f0 0x55a465f3c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.510+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 >> 172.21.15.96:0/134566066 conn(0x55a465f30f20 msgr2=0x55a465f379f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.512+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 shutdown_connections 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.512+0000 7fa706c0a740 1 -- 172.21.15.96:0/134566066 wait complete. 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.513+0000 7fa706c0a740 1 -- start start 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.513+0000 7fa706c0a740 1 Processor -- start 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.513+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 0x55a465f39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.513+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 0x55a465f3a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.513+0000 7fa706c0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f3f9a0 0x55a465f58b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 0x55a465f3a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fdd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 0x55a465f3a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48606/0 (socket says 172.21.15.96:48606) 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fdd9b640 1 -- 172.21.15.96:0/4090035635 learned_addr learned my addr 172.21.15.96:0/4090035635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001ad5b0 con 0x55a465f3f9a0 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001acee0 con 0x55a465f33a10 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001068c0 con 0x55a465f330f0 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 0x55a465f39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa706c0a740 10 client.0 _ll_register_callbacks cb 0x7fa6f40e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 -- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 msgr2=0x55a465f3a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 0x55a465f3a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 -- 172.21.15.96:0/4090035635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f3f9a0 msgr2=0x55a465f58b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:51.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f3f9a0 0x55a465f58b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a465f59390 con 0x55a465f330f0 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6fe59c640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 0x55a465f39f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0003d20 tx=0x7fa6f0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4fef8f0eb1a6193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.514+0000 7fa6e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0035040 con 0x55a465f330f0 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa6f004b440 con 0x55a465f330f0 2024-09-10T22:48:51.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f8013c60 con 0x55a465f330f0 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6f0035040 con 0x55a465f330f0 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa706c0a740 10 client.25273 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.515+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a465fcd1b0 con 0x55a465f330f0 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f8013c60 con 0x55a465f330f0 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa6f0049040 con 0x55a465f330f0 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 10 client.25273 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 10 client.25273 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:48:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 20 client.25273 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465fcd740 0x55a465fcfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 4 client.25273 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a465eabdf0 con 0x55a465fcd740 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465fcd740 0x55a465fcfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa6fdd9b640 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465fcd740 0x55a465fcfba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b0aa02e010a9728 server_cookie=88e2cf96a1cc5cf5 in_seq=0 out_seq=0 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.516+0000 7fa6e5ffb640 10 client.25273 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa6e5ffb640 1 -- 172.21.15.96:0/4090035635 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a465eabdf0 con 0x55a465fcd740 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa6e5ffb640 10 client.25273 handle_command_reply: tid=0 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 client.25273 shutdown 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 msgr2=0x55a465f39f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 0x55a465f39f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0003d20 tx=0x7fa6f0005a20 comp rx=0 tx=0).stop 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 shutdown_connections 2024-09-10T22:48:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a465fcd740 0x55a465fcfba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a465f33a10 0x55a465f3a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a465f3f9a0 0x55a465f58b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 --2- 172.21.15.96:0/4090035635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a465f330f0 0x55a465f39f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 >> 172.21.15.96:0/4090035635 conn(0x55a465f30f20 msgr2=0x55a465f379f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa6e5ffb640 0 client.25273 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.517+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 shutdown_connections 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.518+0000 7fa706c0a740 1 -- 172.21.15.96:0/4090035635 wait complete. 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.518+0000 7fa706c0a740 20 client.25273 trim_cache size 0 max 16384 2024-09-10T22:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:51.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.519+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 msgr2=0x7fa6e80b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.519+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 0x7fa6e80b7740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55a465f31e00 tx=0x7fa6f00039a0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.519+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 msgr2=0x7fa70019fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:51.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.519+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 0x7fa70019fe40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c940 tx=0x7fa6f400ce00 comp rx=0 tx=0).stop 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 shutdown_connections 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa700106ed0 0x7fa7001a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6e80b5280 0x7fa6e80b7740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7001023b0 0x7fa70019fe40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 --2- 172.21.15.96:0/1244090644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa700101a90 0x7fa70019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 >> 172.21.15.96:0/1244090644 conn(0x7fa7000fd700 msgr2=0x7fa700104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 shutdown_connections 2024-09-10T22:48:51.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:51.520+0000 7fa704835640 1 -- 172.21.15.96:0/1244090644 wait complete. 2024-09-10T22:48:52.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:48:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:52.928 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:52 smithi096 ceph-mon[30869]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 724 op/s 2024-09-10T22:48:53.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:52 smithi121 ceph-mon[35119]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 724 op/s 2024-09-10T22:48:53.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:52 smithi117 ceph-mon[35081]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 724 op/s 2024-09-10T22:48:53.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.193+0000 7ff566f3e640 1 -- 172.21.15.96:0/1014953187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5601087c0 msgr2=0x7ff560108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.193+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1014953187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5601087c0 0x7ff560108c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 -- 172.21.15.96:0/1014953187 shutdown_connections 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1014953187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560109140 0x7ff56010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1014953187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5601087c0 0x7ff560108c00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1014953187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff560103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 -- 172.21.15.96:0/1014953187 >> 172.21.15.96:0/1014953187 conn(0x7ff5600fe860 msgr2=0x7ff560100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 -- 172.21.15.96:0/1014953187 shutdown_connections 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 -- 172.21.15.96:0/1014953187 wait complete. 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 Processor -- start 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.194+0000 7ff566f3e640 1 -- start start 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5601087c0 0x7ff56019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff560109140 0x7ff5601a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff560075fd0 con 0x7ff560109140 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff560075e50 con 0x7ff560102db0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff566f3e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff560076150 con 0x7ff5601087c0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48630/0 (socket says 172.21.15.96:48630) 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 -- 172.21.15.96:0/1163787917 learned_addr learned my addr 172.21.15.96:0/1163787917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff5654db640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff560109140 0x7ff5601a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff55ffff640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5601087c0 0x7ff56019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 -- 172.21.15.96:0/1163787917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5601087c0 msgr2=0x7ff56019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5601087c0 0x7ff56019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 -- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff560109140 msgr2=0x7ff5601a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff560109140 0x7ff5601a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 -- 172.21.15.96:0/1163787917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a14d0 con 0x7ff560102db0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff564cda640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff55400d950 tx=0x7ff55400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.195+0000 7ff55dffb640 1 -- 172.21.15.96:0/1163787917 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554017070 con 0x7ff560102db0 2024-09-10T22:48:53.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.196+0000 7ff55dffb640 1 -- 172.21.15.96:0/1163787917 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff554009c30 con 0x7ff560102db0 2024-09-10T22:48:53.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.196+0000 7ff55dffb640 1 -- 172.21.15.96:0/1163787917 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55400f620 con 0x7ff560102db0 2024-09-10T22:48:53.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.196+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a17c0 con 0x7ff560102db0 2024-09-10T22:48:53.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.196+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a7e30 con 0x7ff560102db0 2024-09-10T22:48:53.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 Processor -- start 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff55dffb640 1 -- 172.21.15.96:0/1163787917 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff554010460 con 0x7ff560102db0 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 -- start start 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e871fffea0 con 0x55e872092600 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e871fef2d0 con 0x55e87208d1b0 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.199+0000 7ff569313740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e871ff30e0 con 0x55e872091ce0 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35096/0 (socket says 172.21.15.96:35096) 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 -- 172.21.15.96:0/2656604543 learned_addr learned my addr 172.21.15.96:0/2656604543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff55ffff640 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff5654db640 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 -- 172.21.15.96:0/2656604543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 msgr2=0x55e8720920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 -- 172.21.15.96:0/2656604543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 msgr2=0x55e87208d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 -- 172.21.15.96:0/2656604543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e871d4d650 con 0x55e872092600 2024-09-10T22:48:53.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.200+0000 7ff564cda640 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5540ebc20 tx=0x7ff5540ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17577dd584aef9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff55dffb640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 0x7ff5500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff5654db640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 0x7ff5500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff54affd640 1 -- 172.21.15.96:0/2656604543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540f4070 con 0x55e872092600 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff55dffb640 1 -- 172.21.15.96:0/1163787917 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff5540e5820 con 0x7ff560102db0 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff54affd640 1 -- 172.21.15.96:0/2656604543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff554060e80 con 0x55e872092600 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.201+0000 7ff54affd640 1 -- 172.21.15.96:0/2656604543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540a8630 con 0x55e872092600 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- 172.21.15.96:0/2656604543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 msgr2=0x55e87208cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff5654db640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 0x7ff5500b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff558007f70 tx=0x7ff558007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff5540ebc20 tx=0x7ff5540ed670 comp rx=0 tx=0).stop 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- 172.21.15.96:0/2656604543 shutdown_connections 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872092600 0x55e87208cc70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 --2- 172.21.15.96:0/2656604543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- 172.21.15.96:0/2656604543 >> 172.21.15.96:0/2656604543 conn(0x55e87208fb10 msgr2=0x55e872090cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- 172.21.15.96:0/2656604543 shutdown_connections 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- 172.21.15.96:0/2656604543 wait complete. 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 -- start start 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.202+0000 7ff569313740 1 Processor -- start 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e871e50700 con 0x55e872091ce0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e871f8a160 con 0x55e87208d1b0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e87208de60 con 0x55e872092600 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff569313740 10 client.0 _ll_register_callbacks cb 0x55e871fa39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff55ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff55ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35104/0 (socket says 172.21.15.96:35104) 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff55ffff640 1 -- 172.21.15.96:0/4031946649 learned_addr learned my addr 172.21.15.96:0/4031946649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.203+0000 7ff564cda640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 -- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 msgr2=0x55e8720ac260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720ac260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 -- 172.21.15.96:0/4031946649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 msgr2=0x55e87208ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 -- 172.21.15.96:0/4031946649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e871fb7c90 con 0x55e872091ce0 2024-09-10T22:48:53.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff55ffff640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f507b0e3c73c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.204+0000 7ff53ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:53.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c040070 con 0x55e872091ce0 2024-09-10T22:48:53.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff54c004040 con 0x55e872091ce0 2024-09-10T22:48:53.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff554060e80 con 0x55e872091ce0 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff54c035040 con 0x55e872091ce0 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff569313740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.205+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8720b5340 con 0x55e872091ce0 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c0054e0 con 0x55e872091ce0 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff54c0049d0 con 0x55e872091ce0 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:53.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 10 client.15696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 20 client.15696 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e8720b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 4 client.15696 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.206+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff554060e80 con 0x55e8720b6800 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.207+0000 7ff564cda640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e8720b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.207+0000 7ff53ffff640 10 client.15696 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.207+0000 7ff564cda640 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e8720b8bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9da2eeb17140410c server_cookie=e651ddd5f832603f in_seq=0 out_seq=0 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff53ffff640 1 -- 172.21.15.96:0/4031946649 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff554060e80 con 0x55e8720b6800 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff53ffff640 10 client.15696 handle_command_reply: tid=0 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff569313740 1 client.15696 shutdown 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 msgr2=0x55e87208b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009900 tx=0x7ff54c032f60 comp rx=0 tx=0).stop 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 shutdown_connections 2024-09-10T22:48:53.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.208+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e8720b8bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff53ffff640 0 client.15696 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e87208b2a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 --2- 172.21.15.96:0/4031946649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 >> 172.21.15.96:0/4031946649 conn(0x55e87208fb10 msgr2=0x55e872095640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 shutdown_connections 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 1 -- 172.21.15.96:0/4031946649 wait complete. 2024-09-10T22:48:53.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.209+0000 7ff569313740 20 client.15696 trim_cache size 0 max 16384 2024-09-10T22:48:53.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 Processor -- start 2024-09-10T22:48:53.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 -- start start 2024-09-10T22:48:53.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e872087d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ac150 con 0x55e872091ce0 2024-09-10T22:48:53.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601add50 con 0x55e87208d1b0 2024-09-10T22:48:53.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.234+0000 7ff569313740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ac820 con 0x55e872092600 2024-09-10T22:48:53.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48664/0 (socket says 172.21.15.96:48664) 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 -- 172.21.15.96:0/3797627815 learned_addr learned my addr 172.21.15.96:0/3797627815 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 -- 172.21.15.96:0/3797627815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 msgr2=0x55e8720882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 -- 172.21.15.96:0/3797627815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 msgr2=0x55e872087d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e872087d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 -- 172.21.15.96:0/3797627815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87208b540 con 0x55e87208d1b0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.235+0000 7ff564cda640 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5540edb70 tx=0x7ff5540e5aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad52b4ad6c5fc1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.236+0000 7ff54a7fc640 1 -- 172.21.15.96:0/3797627815 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554020070 con 0x55e87208d1b0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.236+0000 7ff54a7fc640 1 -- 172.21.15.96:0/3797627815 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5540abaf0 con 0x55e87208d1b0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.236+0000 7ff54a7fc640 1 -- 172.21.15.96:0/3797627815 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540f4070 con 0x55e87208d1b0 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.236+0000 7ff569313740 1 -- 172.21.15.96:0/3797627815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 msgr2=0x55e87208f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.236+0000 7ff569313740 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5540edb70 tx=0x7ff5540e5aa0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 -- 172.21.15.96:0/3797627815 shutdown_connections 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872092600 0x55e8720882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e872091ce0 0x55e872087d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 --2- 172.21.15.96:0/3797627815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e87208d1b0 0x55e87208f0f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 -- 172.21.15.96:0/3797627815 >> 172.21.15.96:0/3797627815 conn(0x55e87208fb10 msgr2=0x55e8720893c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 -- 172.21.15.96:0/3797627815 shutdown_connections 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 -- 172.21.15.96:0/3797627815 wait complete. 2024-09-10T22:48:53.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 -- start start 2024-09-10T22:48:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.237+0000 7ff569313740 1 Processor -- start 2024-09-10T22:48:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e87208d1b0 0x55e872088820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 0x55e8720afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601ad5c0 con 0x55e87208d1b0 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601acef0 con 0x55e872092600 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601aa170 con 0x55e872091ce0 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff569313740 10 client.0 _ll_register_callbacks cb 0x7ff54c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff5654db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 0x55e8720afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff5654db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 0x55e8720afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48666/0 (socket says 172.21.15.96:48666) 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff5654db640 1 -- 172.21.15.96:0/1519288318 learned_addr learned my addr 172.21.15.96:0/1519288318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.238+0000 7ff55ffff640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 -- 172.21.15.96:0/1519288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 msgr2=0x55e8720afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 0x55e8720afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 -- 172.21.15.96:0/1519288318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e87208d1b0 msgr2=0x55e872088820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e87208d1b0 0x55e872088820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 -- 172.21.15.96:0/1519288318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8721226d0 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff55ffff640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720af300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff54c02a910 tx=0x7ff54c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5fc47e0b3b7b1e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff53ffff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c054070 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff54c004ad0 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.239+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5540abaf0 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.240+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff54c05e050 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 10 client.25279 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e872123a40 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5540abaf0 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff54c005130 con 0x55e872091ce0 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 10 client.25279 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 10 client.25279 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 20 client.25279 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:53.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e872126280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:53.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 4 client.25279 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:53.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.241+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e872001df0 con 0x55e8720b6800 2024-09-10T22:48:53.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff564cda640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e872126280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:53.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff53ffff640 10 client.25279 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:53.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff564cda640 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e872126280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba5baa4146a187c8 server_cookie=d264ef7e77be5e5e in_seq=0 out_seq=0 2024-09-10T22:48:53.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff53ffff640 1 -- 172.21.15.96:0/1519288318 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e872001df0 con 0x55e8720b6800 2024-09-10T22:48:53.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff53ffff640 10 client.25279 handle_command_reply: tid=0 2024-09-10T22:48:53.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.242+0000 7ff569313740 1 client.25279 shutdown 2024-09-10T22:48:53.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 msgr2=0x55e8720af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720af300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff54c02a910 tx=0x7ff54c005be0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 shutdown_connections 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8720b6800 0x55e872126280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e872092600 0x55e8720afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e872091ce0 0x55e8720af300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff53ffff640 0 client.25279 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 --2- 172.21.15.96:0/1519288318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e87208d1b0 0x55e872088820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.243+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 >> 172.21.15.96:0/1519288318 conn(0x55e87208fb10 msgr2=0x55e8720907f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.244+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 shutdown_connections 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.244+0000 7ff569313740 1 -- 172.21.15.96:0/1519288318 wait complete. 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.244+0000 7ff569313740 20 client.25279 trim_cache size 0 max 16384 2024-09-10T22:48:53.248 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:53.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 msgr2=0x7ff5500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 0x7ff5500b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff558007f70 tx=0x7ff558007520 comp rx=0 tx=0).stop 2024-09-10T22:48:53.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 msgr2=0x7ff56019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff55400d950 tx=0x7ff55400de10 comp rx=0 tx=0).stop 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 shutdown_connections 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5500b52e0 0x7ff5500b77a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff560109140 0x7ff5601a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5601087c0 0x7ff56019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 --2- 172.21.15.96:0/1163787917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560102db0 0x7ff56019b500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 >> 172.21.15.96:0/1163787917 conn(0x7ff5600fe860 msgr2=0x7ff5600fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 shutdown_connections 2024-09-10T22:48:53.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:53.245+0000 7ff566f3e640 1 -- 172.21.15.96:0/1163787917 wait complete. 2024-09-10T22:48:54.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:48:54.361 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/714486101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc105b10 msgr2=0x7fcfdc107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/714486101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc105b10 0x7fcfdc107f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009900 tx=0x7fcfd002a410 comp rx=0 tx=0).stop 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/714486101 shutdown_connections 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/714486101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc10ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/714486101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc105b10 0x7fcfdc107f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/714486101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.683+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/714486101 >> 172.21.15.96:0/714486101 conn(0x7fcfdc0fd150 msgr2=0x7fcfdc0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.684+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/714486101 shutdown_connections 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.684+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/714486101 wait complete. 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 Processor -- start 2024-09-10T22:48:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 -- start start 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc105b10 0x7fcfdc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10e640 con 0x7fcfdc105b10 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10e4c0 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.685+0000 7fcfe0f3d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10e7c0 con 0x7fcfdc1086c0 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfda59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfda59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48686/0 (socket says 172.21.15.96:48686) 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfda59c640 1 -- 172.21.15.96:0/4233771622 learned_addr learned my addr 172.21.15.96:0/4233771622 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc105b10 0x7fcfdc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfda59c640 1 -- 172.21.15.96:0/4233771622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 msgr2=0x7fcfdc19c280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:48:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.686+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfda59c640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfda59c640 1 -- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc105b10 msgr2=0x7fcfdc19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfda59c640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc105b10 0x7fcfdc19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfda59c640 1 -- 172.21.15.96:0/4233771622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc19cb40 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfda59c640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ea60 tx=0x7fcfcc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.687+0000 7fcfcb7fe640 1 -- 172.21.15.96:0/4233771622 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00cde0 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.688+0000 7fcfcb7fe640 1 -- 172.21.15.96:0/4233771622 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfcc0035f0 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.688+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc19ce30 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.688+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a3500 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.688+0000 7fcfcb7fe640 1 -- 172.21.15.96:0/4233771622 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc010690 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.692+0000 7fcfe3de9740 1 Processor -- start 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.693+0000 7fcfe3de9740 1 -- start start 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.693+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.693+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef189d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef0ee6f0 con 0x55a6ef1879a0 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef0ddb20 con 0x55a6ef185070 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef0e1930 con 0x55a6ef18a3e0 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfcb7fe640 1 -- 172.21.15.96:0/4233771622 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fcfcc010830 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48696/0 (socket says 172.21.15.96:48696) 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1165408812 learned_addr learned my addr 172.21.15.96:0/1165408812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfda59c640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef189d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1165408812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 msgr2=0x55a6ef18c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1165408812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 msgr2=0x55a6ef189d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef189d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1165408812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ef0a3530 con 0x55a6ef185070 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfda59c640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef189d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:48:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.694+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009f30 tx=0x7fcfd400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5028dbbd9442d619 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.695+0000 7fcfcb7fe640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 0x7fcfc40b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.695+0000 7fcfcb7fe640 1 -- 172.21.15.96:0/4233771622 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcfcc0e5db0 con 0x7fcfdc06bfb0 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.696+0000 7fcfc8ff9640 1 -- 172.21.15.96:0/1165408812 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4011640 con 0x55a6ef185070 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.696+0000 7fcfc8ff9640 1 -- 172.21.15.96:0/1165408812 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4011c90 con 0x55a6ef185070 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.696+0000 7fcfda59c640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 0x7fcfc40b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.696+0000 7fcfc8ff9640 1 -- 172.21.15.96:0/1165408812 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4010d10 con 0x55a6ef185070 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1165408812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 msgr2=0x55a6ef187460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009f30 tx=0x7fcfd400f7f0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfda59c640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 0x7fcfc40b7970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0ac270 tx=0x7fcfcc0a8460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1165408812 shutdown_connections 2024-09-10T22:48:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef189d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1165408812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef187460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1165408812 >> 172.21.15.96:0/1165408812 conn(0x55a6ef1779b0 msgr2=0x55a6ef178740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1165408812 shutdown_connections 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.697+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1165408812 wait complete. 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.698+0000 7fcfe3de9740 1 -- start start 2024-09-10T22:48:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.698+0000 7fcfe3de9740 1 Processor -- start 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef185070 0x55a6ef18e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef18edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef015ab0 con 0x55a6ef1879a0 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef083730 con 0x55a6ef18a3e0 2024-09-10T22:48:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6ef18d410 con 0x55a6ef185070 2024-09-10T22:48:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.702+0000 7fcfe3de9740 10 client.0 _ll_register_callbacks cb 0x55a6ef0d76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.703+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.703+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48704/0 (socket says 172.21.15.96:48704) 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.703+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1345906122 learned_addr learned my addr 172.21.15.96:0/1345906122 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.704+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1345906122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef185070 msgr2=0x55a6ef18e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.704+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef185070 0x55a6ef18e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.704+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1345906122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 msgr2=0x55a6ef18edf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:48:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.704+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef18edf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.704+0000 7fcfdad9d640 1 -- 172.21.15.96:0/1345906122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6eee3f790 con 0x55a6ef18a3e0 2024-09-10T22:48:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.705+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009c90 tx=0x7fcfd4009cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1516fb9a5b652172 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.705+0000 7fcfc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.706+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd402b060 con 0x55a6ef18a3e0 2024-09-10T22:48:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.706+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcfd4011440 con 0x55a6ef18a3e0 2024-09-10T22:48:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.707+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4011c90 con 0x55a6ef18a3e0 2024-09-10T22:48:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.707+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd4011640 con 0x55a6ef18a3e0 2024-09-10T22:48:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.707+0000 7fcfe3de9740 10 client.25274 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.707+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6eeeb8620 con 0x55a6ef18a3e0 2024-09-10T22:48:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4011c90 con 0x55a6ef18a3e0 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcfd401bcb0 con 0x55a6ef18a3e0 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 10 client.25274 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 10 client.25274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 20 client.25274 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef17fb20 0x55a6ef181fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 4 client.25274 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.708+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6ef0f0640 con 0x55a6ef17fb20 2024-09-10T22:48:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.714+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef17fb20 0x55a6ef181fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.715+0000 7fcfc1ffb640 10 client.25274 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.715+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef17fb20 0x55a6ef181fa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c2c2aaabebd56e server_cookie=5e8263258a6e8287 in_seq=0 out_seq=0 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/1345906122 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6ef0f0640 con 0x55a6ef17fb20 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfc1ffb640 10 client.25274 handle_command_reply: tid=0 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 client.25274 shutdown 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 msgr2=0x55a6ef18f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009c90 tx=0x7fcfd4009cc0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 shutdown_connections 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef17fb20 0x55a6ef181fa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef18f9c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef18edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/1345906122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef185070 0x55a6ef18e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfc1ffb640 0 client.25274 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 >> 172.21.15.96:0/1345906122 conn(0x55a6ef1779b0 msgr2=0x55a6ef176bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.716+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 shutdown_connections 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.717+0000 7fcfe3de9740 1 -- 172.21.15.96:0/1345906122 wait complete. 2024-09-10T22:48:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.717+0000 7fcfe3de9740 20 client.25274 trim_cache size 0 max 16384 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.740+0000 7fcfe3de9740 1 Processor -- start 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.740+0000 7fcfe3de9740 1 -- start start 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef1879a0 0x55a6ef1785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef17bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc00e400 con 0x55a6ef185070 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc010000 con 0x55a6ef18a3e0 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc00ead0 con 0x55a6ef1879a0 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35150/0 (socket says 172.21.15.96:35150) 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 -- 172.21.15.96:0/338507424 learned_addr learned my addr 172.21.15.96:0/338507424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef17bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef1879a0 0x55a6ef1785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 -- 172.21.15.96:0/338507424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef1879a0 msgr2=0x55a6ef1785f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef1879a0 0x55a6ef1785f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 -- 172.21.15.96:0/338507424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 msgr2=0x55a6ef17bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef17bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 -- 172.21.15.96:0/338507424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ef18eac0 con 0x55a6ef185070 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40256e0 tx=0x7fcfd4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a9d100c10db750 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfc3fff640 1 -- 172.21.15.96:0/338507424 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4035080 con 0x55a6ef185070 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.741+0000 7fcfc3fff640 1 -- 172.21.15.96:0/338507424 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4010750 con 0x55a6ef185070 2024-09-10T22:48:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfc3fff640 1 -- 172.21.15.96:0/338507424 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4011a40 con 0x55a6ef185070 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 -- 172.21.15.96:0/338507424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 msgr2=0x55a6ef190290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40256e0 tx=0x7fcfd4009500 comp rx=0 tx=0).stop 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 -- 172.21.15.96:0/338507424 shutdown_connections 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef18a3e0 0x55a6ef17bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef1879a0 0x55a6ef1785f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/338507424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef185070 0x55a6ef190290 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 -- 172.21.15.96:0/338507424 >> 172.21.15.96:0/338507424 conn(0x55a6ef1779b0 msgr2=0x55a6ef184890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 -- 172.21.15.96:0/338507424 shutdown_connections 2024-09-10T22:48:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.742+0000 7fcfe3de9740 1 -- 172.21.15.96:0/338507424 wait complete. 2024-09-10T22:48:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 -- start start 2024-09-10T22:48:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 Processor -- start 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef1930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef194260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc00f870 con 0x55a6ef1879a0 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc00f1a0 con 0x55a6ef185070 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc00c3e0 con 0x55a6ef18a3e0 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.743+0000 7fcfe3de9740 10 client.0 _ll_register_callbacks cb 0x7fcfd4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35154/0 (socket says 172.21.15.96:35154) 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 -- 172.21.15.96:0/808490710 learned_addr learned my addr 172.21.15.96:0/808490710 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef1930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 -- 172.21.15.96:0/808490710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 msgr2=0x55a6ef194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 -- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 msgr2=0x55a6ef1930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef1930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 -- 172.21.15.96:0/808490710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ef194a90 con 0x55a6ef1879a0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfd9d9b640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00096c0 tx=0x7fcfd0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93c56a8abccf72c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0035040 con 0x55a6ef1879a0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4010750 con 0x55a6ef1879a0 2024-09-10T22:48:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.744+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcfd0005510 con 0x55a6ef1879a0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.745+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd0004770 con 0x55a6ef1879a0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.745+0000 7fcfe3de9740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.745+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6ef195f50 con 0x55a6ef1879a0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.745+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4010750 con 0x55a6ef1879a0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcfd0031b00 con 0x55a6ef1879a0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 20 client.15711 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef220c80 0x55a6ef223070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 4 client.15711 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.746+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6ef0ec0f0 con 0x55a6ef220c80 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.747+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef220c80 0x55a6ef223070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.747+0000 7fcfdad9d640 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef220c80 0x55a6ef223070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb5a40d9f76b047 server_cookie=e0265548e8bc7883 in_seq=0 out_seq=0 2024-09-10T22:48:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.747+0000 7fcfc1ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:54.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:54 smithi096 ceph-mon[30869]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/808490710 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6ef0ec0f0 con 0x55a6ef220c80 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfc1ffb640 10 client.15711 handle_command_reply: tid=0 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 client.15711 shutdown 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 msgr2=0x55a6ef193790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00096c0 tx=0x7fcfd0004500 comp rx=0 tx=0).stop 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 shutdown_connections 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6ef220c80 0x55a6ef223070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfc1ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ef18a3e0 0x55a6ef194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6ef1879a0 0x55a6ef193790 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 --2- 172.21.15.96:0/808490710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ef185070 0x55a6ef1930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 >> 172.21.15.96:0/808490710 conn(0x55a6ef1779b0 msgr2=0x55a6ef178520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 shutdown_connections 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 1 -- 172.21.15.96:0/808490710 wait complete. 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.762+0000 7fcfe3de9740 20 client.15711 trim_cache size 0 max 16384 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:48:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:48:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:48:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 msgr2=0x7fcfc40b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 0x7fcfc40b7970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0ac270 tx=0x7fcfcc0a8460 comp rx=0 tx=0).stop 2024-09-10T22:48:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 msgr2=0x7fcfdc19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ea60 tx=0x7fcfcc00ef20 comp rx=0 tx=0).stop 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 shutdown_connections 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcfc40b54b0 0x7fcfc40b7970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfdc1086c0 0x7fcfdc19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc105b10 0x7fcfdc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 --2- 172.21.15.96:0/4233771622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc06bfb0 0x7fcfdc19b000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.763+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 >> 172.21.15.96:0/4233771622 conn(0x7fcfdc0fd150 msgr2=0x7fcfdc0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.764+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 shutdown_connections 2024-09-10T22:48:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:48:54.764+0000 7fcfe0f3d640 1 -- 172.21.15.96:0/4233771622 wait complete. 2024-09-10T22:48:54.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 -- 172.21.15.96:0/3043095395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b4108020 msgr2=0x7f67b406aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/3043095395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b4108020 0x7f67b406aab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ca0 tx=0x7f67ac02a8b0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 -- 172.21.15.96:0/3043095395 shutdown_connections 2024-09-10T22:48:54.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/3043095395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b41076a0 0x7f67b4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/3043095395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b4108020 0x7f67b406aab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/3043095395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 0x7f67b4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 -- 172.21.15.96:0/3043095395 >> 172.21.15.96:0/3043095395 conn(0x7f67b40fd700 msgr2=0x7f67b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.916+0000 7f67bc6ed640 1 -- 172.21.15.96:0/3043095395 shutdown_connections 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.917+0000 7f67bc6ed640 1 -- 172.21.15.96:0/3043095395 wait complete. 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 Processor -- start 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 -- start start 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 0x7f67b410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b41076a0 0x7f67b410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4112d70 con 0x7f67b4101c90 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4112bf0 con 0x7f67b4108020 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.918+0000 7f67bc6ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67b4112ef0 con 0x7f67b41076a0 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.919+0000 7f67bac8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.919+0000 7f67bac8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48734/0 (socket says 172.21.15.96:48734) 2024-09-10T22:48:54.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.919+0000 7f67bac8a640 1 -- 172.21.15.96:0/1243909402 learned_addr learned my addr 172.21.15.96:0/1243909402 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.923+0000 7f67ba489640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 0x7f67b410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.923+0000 7f67b9c88640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b41076a0 0x7f67b410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.923+0000 7f67bac8a640 1 -- 172.21.15.96:0/1243909402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b41076a0 msgr2=0x7f67b410c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.923+0000 7f67bac8a640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b41076a0 0x7f67b410c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.924+0000 7f67bac8a640 1 -- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 msgr2=0x7f67b410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.924+0000 7f67bac8a640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 0x7f67b410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.924+0000 7f67bac8a640 1 -- 172.21.15.96:0/1243909402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b41aba40 con 0x7f67b4108020 2024-09-10T22:48:54.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.924+0000 7f67bac8a640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f67ac003bd0 tx=0x7f67ac003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.926+0000 7f67ab7fe640 1 -- 172.21.15.96:0/1243909402 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac004240 con 0x7f67b4108020 2024-09-10T22:48:54.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.927+0000 7f67ab7fe640 1 -- 172.21.15.96:0/1243909402 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac040070 con 0x7f67b4108020 2024-09-10T22:48:54.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.927+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41abc10 con 0x7f67b4108020 2024-09-10T22:48:54.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.927+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41ac000 con 0x7f67b4108020 2024-09-10T22:48:54.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.927+0000 7f67ab7fe640 1 -- 172.21.15.96:0/1243909402 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac045d20 con 0x7f67b4108020 2024-09-10T22:48:54.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.932+0000 7f67ab7fe640 1 -- 172.21.15.96:0/1243909402 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f67ac031460 con 0x7f67b4108020 2024-09-10T22:48:54.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.932+0000 7f67beac2740 1 Processor -- start 2024-09-10T22:48:54.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.933+0000 7f67beac2740 1 -- start start 2024-09-10T22:48:54.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67ab7fe640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 0x7f67a00b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.933+0000 7f67beac2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67beac2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67beac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67ab7fe640 1 -- 172.21.15.96:0/1243909402 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f67ac106c70 con 0x7f67b4108020 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67beac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ebccdeea0 con 0x561ebcd74d80 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67beac2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ebccce2d0 con 0x561ebcd684d0 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.935+0000 7f67beac2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ebccd20e0 con 0x561ebcd68df0 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48740/0 (socket says 172.21.15.96:48740) 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 -- 172.21.15.96:0/2340849089 learned_addr learned my addr 172.21.15.96:0/2340849089 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67ba489640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 0x7f67a00b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67bac8a640 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 -- 172.21.15.96:0/2340849089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 msgr2=0x561ebcd74840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd74840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 -- 172.21.15.96:0/2340849089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 msgr2=0x561ebcd77170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd77170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.936+0000 7f67b9c88640 1 -- 172.21.15.96:0/2340849089 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ebca2c650 con 0x561ebcd684d0 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.937+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561ebcd671e0 tx=0x7f67a400b440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9905945a16895885 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.937+0000 7f67ba489640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 0x7f67a00b79b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005e10 tx=0x7f67b0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.937+0000 7f67a8ff9640 1 -- 172.21.15.96:0/2340849089 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400e920 con 0x561ebcd684d0 2024-09-10T22:48:54.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.937+0000 7f67a8ff9640 1 -- 172.21.15.96:0/2340849089 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a4011440 con 0x561ebcd684d0 2024-09-10T22:48:54.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.937+0000 7f67a8ff9640 1 -- 172.21.15.96:0/2340849089 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4010670 con 0x561ebcd684d0 2024-09-10T22:48:54.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 -- 172.21.15.96:0/2340849089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 msgr2=0x561ebcd688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561ebcd671e0 tx=0x7f67a400b440 comp rx=0 tx=0).stop 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 -- 172.21.15.96:0/2340849089 shutdown_connections 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd77170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd74840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 --2- 172.21.15.96:0/2340849089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd688b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.938+0000 7f67beac2740 1 -- 172.21.15.96:0/2340849089 >> 172.21.15.96:0/2340849089 conn(0x561ebcd66300 msgr2=0x561ebcd674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- 172.21.15.96:0/2340849089 shutdown_connections 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- 172.21.15.96:0/2340849089 wait complete. 2024-09-10T22:48:54.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- start start 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 Processor -- start 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 0x561ebcd71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ebcb2f700 con 0x561ebcd74d80 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ebcc69160 con 0x561ebcd68df0 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ebcd77a00 con 0x561ebcd684d0 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.939+0000 7f67beac2740 10 client.0 _ll_register_callbacks cb 0x561ebcc829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67b9c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 0x561ebcd71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49198/0 (socket says 172.21.15.96:49198) 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 -- 172.21.15.96:0/3420048890 learned_addr learned my addr 172.21.15.96:0/3420048890 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67bac8a640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67b9c88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 0x561ebcd71ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48752/0 (socket says 172.21.15.96:48752) 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 -- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 msgr2=0x561ebcd71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 0x561ebcd71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 -- 172.21.15.96:0/3420048890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 msgr2=0x561ebcd720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 -- 172.21.15.96:0/3420048890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ebcc96c90 con 0x561ebcd684d0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67ba489640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67b00109b0 tx=0x7f67b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83026725997c5344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f679dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0021070 con 0x561ebcd684d0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f67b000fcd0 con 0x561ebcd684d0 2024-09-10T22:48:54.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a4011440 con 0x561ebcd684d0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.940+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67b001d450 con 0x561ebcd684d0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 10 client.25300 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ebcd83360 con 0x561ebcd684d0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b002d070 con 0x561ebcd684d0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f67b00254d0 con 0x561ebcd684d0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 10 client.25300 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 10 client.25300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 20 client.25300 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcd84910 0x561ebcd86d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 4 client.25300 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ebcce0df0 con 0x561ebcd84910 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.941+0000 7f67b9c88640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcd84910 0x561ebcd86d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.942+0000 7f67b9c88640 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcd84910 0x561ebcd86d50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e458ddfb7b5b7fd server_cookie=b285815debcd437b in_seq=0 out_seq=0 2024-09-10T22:48:54.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.942+0000 7f679dffb640 10 client.25300 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:54.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.943+0000 7f679dffb640 1 -- 172.21.15.96:0/3420048890 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ebcce0df0 con 0x561ebcd84910 2024-09-10T22:48:54.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.943+0000 7f679dffb640 10 client.25300 handle_command_reply: tid=0 2024-09-10T22:48:54.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.943+0000 7f67beac2740 1 client.25300 shutdown 2024-09-10T22:48:54.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.943+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 msgr2=0x561ebcd74670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.943+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67b00109b0 tx=0x7f67b00046c0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 shutdown_connections 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcd84910 0x561ebcd86d50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd68df0 0x561ebcd71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 --2- 172.21.15.96:0/3420048890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd684d0 0x561ebcd74670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 >> 172.21.15.96:0/3420048890 conn(0x561ebcd66300 msgr2=0x561ebcd6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f679dffb640 0 client.25300 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 shutdown_connections 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 1 -- 172.21.15.96:0/3420048890 wait complete. 2024-09-10T22:48:54.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.944+0000 7f67beac2740 20 client.25300 trim_cache size 0 max 16384 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 Processor -- start 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 -- start start 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd80cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f679800e310 con 0x561ebcd74d80 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67bac8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67bac8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd6ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35202/0 (socket says 172.21.15.96:35202) 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67bac8a640 1 -- 172.21.15.96:0/872679761 learned_addr learned my addr 172.21.15.96:0/872679761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67ba489640 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd80cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f679800ff10 con 0x561ebcd684d0 2024-09-10T22:48:54.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f679800e9e0 con 0x561ebcd68df0 2024-09-10T22:48:54.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.962+0000 7f67b9c88640 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.963+0000 7f67ba489640 1 -- 172.21.15.96:0/872679761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 msgr2=0x561ebcd73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.963+0000 7f67ba489640 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd73800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.963+0000 7f67ba489640 1 -- 172.21.15.96:0/872679761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 msgr2=0x561ebcd6ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.963+0000 7f67ba489640 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd6ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.963+0000 7f67ba489640 1 -- 172.21.15.96:0/872679761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ebcd66d80 con 0x561ebcd684d0 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.964+0000 7f67ba489640 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd80cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e720 tx=0x7f67b0005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89fd57f2876bfed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.964+0000 7f679ffff640 1 -- 172.21.15.96:0/872679761 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0021040 con 0x561ebcd684d0 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.964+0000 7f679ffff640 1 -- 172.21.15.96:0/872679761 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b002d070 con 0x561ebcd684d0 2024-09-10T22:48:54.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.964+0000 7f679ffff640 1 -- 172.21.15.96:0/872679761 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0005280 con 0x561ebcd684d0 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 msgr2=0x561ebcd80cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd80cc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e720 tx=0x7f67b0005870 comp rx=0 tx=0).stop 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 shutdown_connections 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcd6ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 --2- 172.21.15.96:0/872679761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd80cc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 >> 172.21.15.96:0/872679761 conn(0x561ebcd66300 msgr2=0x561ebcd730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 shutdown_connections 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.965+0000 7f67beac2740 1 -- 172.21.15.96:0/872679761 wait complete. 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 -- start start 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 Processor -- start 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcdf0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67ba489640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67ba489640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:48772/0 (socket says 172.21.15.96:48772) 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67ba489640 1 -- 172.21.15.96:0/2050290201 learned_addr learned my addr 172.21.15.96:0/2050290201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f679800f780 con 0x561ebcd74d80 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f679800f0b0 con 0x561ebcd684d0 2024-09-10T22:48:54.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f679800c2f0 con 0x561ebcd68df0 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67bac8a640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcdf0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67beac2740 10 client.0 _ll_register_callbacks cb 0x7f67b0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.966+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 -- 172.21.15.96:0/2050290201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 msgr2=0x561ebcd795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 -- 172.21.15.96:0/2050290201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 msgr2=0x561ebcdf0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:54.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcdf0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ebcdf0d60 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67ba489640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67b00255e0 tx=0x7f67b0025610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e432a378e04da68a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f679dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0021070 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f67b0005030 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b002d070 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67b0042050 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67beac2740 10 client.25295 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ebcdf1f30 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.967+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b002d070 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f67b0004b10 con 0x561ebcd684d0 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 10 client.25295 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 10 client.25295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:48:54.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 20 client.25295 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcdf2510 0x561ebcdf4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 4 client.25295 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ebccdc8a0 con 0x561ebcdf2510 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcdf2510 0x561ebcdf4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f679dffb640 10 client.25295 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67b9c88640 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcdf2510 0x561ebcdf4930 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef954739a49ccb02 server_cookie=ca0ad106f01a60c2 in_seq=0 out_seq=0 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f679dffb640 1 -- 172.21.15.96:0/2050290201 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ebccdc8a0 con 0x561ebcdf2510 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f679dffb640 10 client.25295 handle_command_reply: tid=0 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.968+0000 7f67beac2740 1 client.25295 shutdown 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 msgr2=0x561ebcd79090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67b00255e0 tx=0x7f67b0025610 comp rx=0 tx=0).stop 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 shutdown_connections 2024-09-10T22:48:54.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x561ebcdf2510 0x561ebcdf4930 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ebcd74d80 0x561ebcdf0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebcd68df0 0x561ebcd795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 --2- 172.21.15.96:0/2050290201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ebcd684d0 0x561ebcd79090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f679dffb640 0 client.25295 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 >> 172.21.15.96:0/2050290201 conn(0x561ebcd66300 msgr2=0x561ebcd73910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 shutdown_connections 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 1 -- 172.21.15.96:0/2050290201 wait complete. 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.969+0000 7f67beac2740 20 client.25295 trim_cache size 0 max 16384 2024-09-10T22:48:54.976 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:54.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 msgr2=0x7f67a00b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 0x7f67a00b79b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005e10 tx=0x7f67b0005da0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 msgr2=0x7f67b410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f67ac003bd0 tx=0x7f67ac003c00 comp rx=0 tx=0).stop 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 shutdown_connections 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b4108020 0x7f67b410cf60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f67a00b54f0 0x7f67a00b79b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b41076a0 0x7f67b410c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 --2- 172.21.15.96:0/1243909402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67b4101c90 0x7f67b410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 >> 172.21.15.96:0/1243909402 conn(0x7f67b40fd700 msgr2=0x7f67b41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 shutdown_connections 2024-09-10T22:48:54.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:54.970+0000 7f67bc6ed640 1 -- 172.21.15.96:0/1243909402 wait complete. 2024-09-10T22:48:55.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:54 smithi121 ceph-mon[35119]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T22:48:55.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:54 smithi117 ceph-mon[35081]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T22:48:55.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:48:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 80.0 s, 3422.2 tps, lat 29.288 ms stddev 30.810 2024-09-10T22:48:56.141 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 -- 172.21.15.96:0/1622855945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 msgr2=0x7f02e810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 --2- 172.21.15.96:0/1622855945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e810e820 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009ca0 tx=0x7f02e002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 -- 172.21.15.96:0/1622855945 shutdown_connections 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 --2- 172.21.15.96:0/1622855945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e810e820 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 --2- 172.21.15.96:0/1622855945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e81076a0 0x7f02e8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 --2- 172.21.15.96:0/1622855945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e8101c90 0x7f02e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 -- 172.21.15.96:0/1622855945 >> 172.21.15.96:0/1622855945 conn(0x7f02e80fd700 msgr2=0x7f02e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 -- 172.21.15.96:0/1622855945 shutdown_connections 2024-09-10T22:48:56.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.745+0000 7f02ef017640 1 -- 172.21.15.96:0/1622855945 wait complete. 2024-09-10T22:48:56.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 Processor -- start 2024-09-10T22:48:56.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 -- start start 2024-09-10T22:48:56.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 0x7f02e819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81076a0 0x7f02e819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111ca0 con 0x7f02e8108020 2024-09-10T22:48:56.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111b20 con 0x7f02e81076a0 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ed5b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ecdb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 0x7f02e819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ed5b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51044/0 (socket says 172.21.15.96:51044) 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1854932581 learned_addr learned my addr 172.21.15.96:0/1854932581 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02ef017640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8111e20 con 0x7f02e8101c90 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.746+0000 7f02e7fff640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81076a0 0x7f02e819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1854932581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 msgr2=0x7f02e819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 0x7f02e819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1854932581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81076a0 msgr2=0x7f02e819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81076a0 0x7f02e819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1854932581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e81a5470 con 0x7f02e8108020 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 0x7f02e819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098d0 tx=0x7f02e00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02e5ffb640 1 -- 172.21.15.96:0/1854932581 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0003b70 con 0x7f02e8108020 2024-09-10T22:48:56.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02e5ffb640 1 -- 172.21.15.96:0/1854932581 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e0003d10 con 0x7f02e8108020 2024-09-10T22:48:56.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02e5ffb640 1 -- 172.21.15.96:0/1854932581 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e002f990 con 0x7f02e8108020 2024-09-10T22:48:56.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.747+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e8197e60 con 0x7f02e8108020 2024-09-10T22:48:56.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.749+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e8198290 con 0x7f02e8108020 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 Processor -- start 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02e5ffb640 1 -- 172.21.15.96:0/1854932581 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f02e002fb30 con 0x7f02e8108020 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 -- start start 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf63840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf66170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55755becdea0 con 0x55755bf63d80 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55755bebd2d0 con 0x55755bf57df0 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.752+0000 7f02f13ec740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55755bec10e0 con 0x55755bf574d0 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35916/0 (socket says 172.21.15.96:35916) 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 -- 172.21.15.96:0/2870637582 learned_addr learned my addr 172.21.15.96:0/2870637582 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf66170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf63840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 -- 172.21.15.96:0/2870637582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 msgr2=0x55755bf63840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf63840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 -- 172.21.15.96:0/2870637582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 msgr2=0x55755bf66170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf66170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 -- 172.21.15.96:0/2870637582 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755bc1b650 con 0x55755bf574d0 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e5ffb640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 0x7f02d80bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e7fff640 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02d4012790 tx=0x7f02d4009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c7aba3588d544a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 0x7f02d80bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02d2ffd640 1 -- 172.21.15.96:0/2870637582 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4012d10 con 0x55755bf574d0 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02e5ffb640 1 -- 172.21.15.96:0/1854932581 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f02e0106e30 con 0x7f02e8108020 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02d2ffd640 1 -- 172.21.15.96:0/2870637582 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d40107d0 con 0x55755bf574d0 2024-09-10T22:48:56.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.753+0000 7f02d2ffd640 1 -- 172.21.15.96:0/2870637582 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d40044e0 con 0x55755bf574d0 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 -- 172.21.15.96:0/2870637582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 msgr2=0x55755bf578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 0x7f02d80bbdc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55755bf561e0 tx=0x7f02dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02d4012790 tx=0x7f02d4009700 comp rx=0 tx=0).stop 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 -- 172.21.15.96:0/2870637582 shutdown_connections 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf63840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf66170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 --2- 172.21.15.96:0/2870637582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf578b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.754+0000 7f02f13ec740 1 -- 172.21.15.96:0/2870637582 >> 172.21.15.96:0/2870637582 conn(0x55755bf55300 msgr2=0x55755bf564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 -- 172.21.15.96:0/2870637582 shutdown_connections 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 -- 172.21.15.96:0/2870637582 wait complete. 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 -- start start 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 Processor -- start 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf63650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf60aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.755+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02f13ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55755bd1e700 con 0x55755bf63d80 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02f13ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55755be58160 con 0x55755bf57df0 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02f13ec740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55755bf66a00 con 0x55755bf574d0 2024-09-10T22:48:56.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02f13ec740 10 client.0 _ll_register_callbacks cb 0x55755be719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51068/0 (socket says 172.21.15.96:51068) 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1161939995 learned_addr learned my addr 172.21.15.96:0/1161939995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 msgr2=0x55755bf63650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf63650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1161939995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 msgr2=0x55755bf60aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf60aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.756+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1161939995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755be85c90 con 0x55755bf63d80 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.757+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cd580 tx=0x7f02e00cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721888b5890e223e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.757+0000 7f02c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.757+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00cd5f0 con 0x55755bf63d80 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.757+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f02e00d66f0 con 0x55755bf63d80 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.757+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d40107d0 con 0x55755bf63d80 2024-09-10T22:48:56.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02e00e4020 con 0x55755bf63d80 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 10 client.15738 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55755bf7ab50 con 0x55755bf63d80 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d40107d0 con 0x55755bf63d80 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f02e00cc850 con 0x55755bf63d80 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 10 client.15738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 20 client.15738 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bf7bf90 0x55755bf7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 4 client.15738 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755becfdf0 con 0x55755bf7bf90 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.758+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bf7bf90 0x55755bf7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.759+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bf7bf90 0x55755bf7e410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd7e1113c930fb58 server_cookie=5228517f615d283a in_seq=0 out_seq=0 2024-09-10T22:48:56.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.759+0000 7f02c7fff640 10 client.15738 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.759+0000 7f02c7fff640 1 -- 172.21.15.96:0/1161939995 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55755becfdf0 con 0x55755bf7bf90 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.759+0000 7f02c7fff640 10 client.15738 handle_command_reply: tid=0 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 client.15738 shutdown 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 msgr2=0x55755bf61080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cd580 tx=0x7f02e00cd000 comp rx=0 tx=0).stop 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 shutdown_connections 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bf7bf90 0x55755bf7e410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf60aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bf61080 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1161939995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf63650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 >> 172.21.15.96:0/1161939995 conn(0x55755bf55300 msgr2=0x55755bf5ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02c7fff640 0 client.15738 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 shutdown_connections 2024-09-10T22:48:56.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 1 -- 172.21.15.96:0/1161939995 wait complete. 2024-09-10T22:48:56.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.760+0000 7f02f13ec740 20 client.15738 trim_cache size 0 max 16384 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 Processor -- start 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 -- start start 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf5dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 0x55755bf5e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81ac0d0 con 0x55755bf574d0 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81adcd0 con 0x55755bf57df0 2024-09-10T22:48:56.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02f13ec740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81ac7a0 con 0x55755bf63d80 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02ecdb3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02ecdb3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51074/0 (socket says 172.21.15.96:51074) 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02ecdb3640 1 -- 172.21.15.96:0/1631310971 learned_addr learned my addr 172.21.15.96:0/1631310971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 0x55755bf5e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.778+0000 7f02e7fff640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf5dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1631310971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 msgr2=0x55755bf5dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf5dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1631310971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 msgr2=0x55755bf54c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 -- 172.21.15.96:0/1631310971 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755bf639c0 con 0x55755bf63d80 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 0x55755bf5e1f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02e0045000 tx=0x7f02e00cd070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e2f9d16f73249b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02d27fc640 1 -- 172.21.15.96:0/1631310971 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00ec070 con 0x55755bf63d80 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02d27fc640 1 -- 172.21.15.96:0/1631310971 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e0036040 con 0x55755bf63d80 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.779+0000 7f02d27fc640 1 -- 172.21.15.96:0/1631310971 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00c8f40 con 0x55755bf63d80 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- 172.21.15.96:0/1631310971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 msgr2=0x55755bf5e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 0x55755bf5e1f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f02e0045000 tx=0x7f02e00cd070 comp rx=0 tx=0).stop 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- 172.21.15.96:0/1631310971 shutdown_connections 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf5dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf63d80 0x55755bf5e1f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 --2- 172.21.15.96:0/1631310971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf574d0 0x55755bf54c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- 172.21.15.96:0/1631310971 >> 172.21.15.96:0/1631310971 conn(0x55755bf55300 msgr2=0x55755bf5f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- 172.21.15.96:0/1631310971 shutdown_connections 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- 172.21.15.96:0/1631310971 wait complete. 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.780+0000 7f02f13ec740 1 -- start start 2024-09-10T22:48:56.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.781+0000 7f02f13ec740 1 Processor -- start 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.781+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.781+0000 7f02f13ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02ecdb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02ecdb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf680d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35938/0 (socket says 172.21.15.96:35938) 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02ecdb3640 1 -- 172.21.15.96:0/3336137869 learned_addr learned my addr 172.21.15.96:0/3336137869 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bfe7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02ed5b4640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bfe7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81ad540 con 0x55755bf63d80 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81ace70 con 0x55755bf57df0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81aa0b0 con 0x55755bf574d0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02f13ec740 10 client.0 _ll_register_callbacks cb 0x7f02e0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 -- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 msgr2=0x55755bf680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 -- 172.21.15.96:0/3336137869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 msgr2=0x55755bfe7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bfe7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.782+0000 7f02e7fff640 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755bfe8420 con 0x55755bf57df0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02e7fff640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf68610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55755bf04ef0 tx=0x7f02d4005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91d1eafc92167e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4009530 con 0x55755bf57df0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f02d4012240 con 0x55755bf57df0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e0036040 con 0x55755bf57df0 2024-09-10T22:48:56.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02d4012440 con 0x55755bf57df0 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02f13ec740 10 client.25310 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55755bfe9690 con 0x55755bf57df0 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.783+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d4010c50 con 0x55755bf57df0 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f02d4003a00 con 0x55755bf57df0 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 10 client.25310 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 10 client.25310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 20 client.25310 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bfe9b80 0x55755bfebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 4 client.25310 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f02e0036040 con 0x55755bfe9b80 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bfe9b80 0x55755bfebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02c7fff640 10 client.25310 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:56.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02ecdb3640 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bfe9b80 0x55755bfebfa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a2981068d0afc1 server_cookie=d39f8f51d279cd44 in_seq=0 out_seq=0 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02c7fff640 1 -- 172.21.15.96:0/3336137869 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f02e0036040 con 0x55755bfe9b80 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02c7fff640 10 client.25310 handle_command_reply: tid=0 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.784+0000 7f02f13ec740 1 client.25310 shutdown 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 msgr2=0x55755bf68610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf68610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55755bf04ef0 tx=0x7f02d4005680 comp rx=0 tx=0).stop 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 shutdown_connections 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55755bfe9b80 0x55755bfebfa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55755bf57df0 0x55755bf68610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55755bf63d80 0x55755bfe7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 --2- 172.21.15.96:0/3336137869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55755bf574d0 0x55755bf680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 >> 172.21.15.96:0/3336137869 conn(0x55755bf55300 msgr2=0x55755bf628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02c7fff640 0 client.25310 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:48:56.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 shutdown_connections 2024-09-10T22:48:56.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 1 -- 172.21.15.96:0/3336137869 wait complete. 2024-09-10T22:48:56.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.785+0000 7f02f13ec740 20 client.25310 trim_cache size 0 max 16384 2024-09-10T22:48:56.792 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.786+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 msgr2=0x7f02d80bbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.786+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 0x7f02d80bbdc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55755bf561e0 tx=0x7f02dc00b390 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.786+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 msgr2=0x7f02e819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.786+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098d0 tx=0x7f02e00319f0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 shutdown_connections 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02e81076a0 0x7f02e819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f02d80b9900 0x7f02d80bbdc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02e8108020 0x7f02e819ab30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 --2- 172.21.15.96:0/1854932581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e8101c90 0x7f02e819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 >> 172.21.15.96:0/1854932581 conn(0x7f02e80fd700 msgr2=0x7f02e80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 shutdown_connections 2024-09-10T22:48:56.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:56.787+0000 7f02ef017640 1 -- 172.21.15.96:0/1854932581 wait complete. 2024-09-10T22:48:56.844 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:56 smithi096 ceph-mon[30869]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 866 op/s 2024-09-10T22:48:57.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:56 smithi121 ceph-mon[35119]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 866 op/s 2024-09-10T22:48:57.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:56 smithi117 ceph-mon[35081]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 866 op/s 2024-09-10T22:48:57.580 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:48:57.947 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:48:58.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- 172.21.15.96:0/2418850704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 msgr2=0x7f753c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 --2- 172.21.15.96:0/2418850704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1040b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7530009900 tx=0x7f753002a400 comp rx=0 tx=0).stop 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- 172.21.15.96:0/2418850704 shutdown_connections 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 --2- 172.21.15.96:0/2418850704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c1045f0 0x7f753c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 --2- 172.21.15.96:0/2418850704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c10d130 0x7f753c10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 --2- 172.21.15.96:0/2418850704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1040b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- 172.21.15.96:0/2418850704 >> 172.21.15.96:0/2418850704 conn(0x7f753c0fd770 msgr2=0x7f753c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- 172.21.15.96:0/2418850704 shutdown_connections 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- 172.21.15.96:0/2418850704 wait complete. 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 Processor -- start 2024-09-10T22:48:58.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 -- start start 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.501+0000 7f75430bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f75430bf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1045f0 0x7f753c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f75430bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c10d130 0x7f753c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f75430bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112df0 con 0x7f753c103cd0 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f75430bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112c70 con 0x7f753c10d130 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f75430bf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f753c112f70 con 0x7f753c1045f0 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51098/0 (socket says 172.21.15.96:51098) 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 -- 172.21.15.96:0/115518497 learned_addr learned my addr 172.21.15.96:0/115518497 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f753bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1045f0 0x7f753c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f754165c640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c10d130 0x7f753c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 -- 172.21.15.96:0/115518497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1045f0 msgr2=0x7f753c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1045f0 0x7f753c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 -- 172.21.15.96:0/115518497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c10d130 msgr2=0x7f753c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c10d130 0x7f753c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 -- 172.21.15.96:0/115518497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19f370 con 0x7f753c103cd0 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.502+0000 7f7540e5b640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7530005ab0 tx=0x7f7530003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.503+0000 7f7539ffb640 1 -- 172.21.15.96:0/115518497 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753003f070 con 0x7f753c103cd0 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.503+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19f600 con 0x7f753c103cd0 2024-09-10T22:48:58.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.503+0000 7f7539ffb640 1 -- 172.21.15.96:0/115518497 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530031d60 con 0x7f753c103cd0 2024-09-10T22:48:58.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.503+0000 7f7539ffb640 1 -- 172.21.15.96:0/115518497 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530035030 con 0x7f753c103cd0 2024-09-10T22:48:58.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.503+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a5d20 con 0x7f753c103cd0 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 Processor -- start 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 -- start start 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cbd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563953c27ea0 con 0x563953cbd9a0 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563953c172d0 con 0x563953cb1a10 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f7545494740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563953c1b0e0 con 0x563953cb10f0 2024-09-10T22:48:58.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f754165c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f753bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f754165c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35960/0 (socket says 172.21.15.96:35960) 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.507+0000 7f754165c640 1 -- 172.21.15.96:0/1348887549 learned_addr learned my addr 172.21.15.96:0/1348887549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f7539ffb640 1 -- 172.21.15.96:0/115518497 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f753003a3c0 con 0x7f753c103cd0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 -- 172.21.15.96:0/1348887549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 msgr2=0x563953cbd460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cbd460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 -- 172.21.15.96:0/1348887549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 msgr2=0x563953cbfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 -- 172.21.15.96:0/1348887549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563953975650 con 0x563953cb10f0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f754165c640 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7534010370 tx=0x7f7534010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed69e04fcea13fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f7526ffd640 1 -- 172.21.15.96:0/1348887549 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534016070 con 0x563953cb10f0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f7526ffd640 1 -- 172.21.15.96:0/1348887549 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7534004d20 con 0x563953cb10f0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f7526ffd640 1 -- 172.21.15.96:0/1348887549 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534005350 con 0x563953cb10f0 2024-09-10T22:48:58.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.508+0000 7f7539ffb640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 0x7f752c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.509+0000 7f7539ffb640 1 -- 172.21.15.96:0/115518497 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7530106620 con 0x7f753c103cd0 2024-09-10T22:48:58.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.509+0000 7f7540e5b640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 0x7f752c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- 172.21.15.96:0/1348887549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 msgr2=0x563953cb14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7534010370 tx=0x7f7534010830 comp rx=0 tx=0).stop 2024-09-10T22:48:58.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- 172.21.15.96:0/1348887549 shutdown_connections 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cbd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 --2- 172.21.15.96:0/1348887549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cb14d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- 172.21.15.96:0/1348887549 >> 172.21.15.96:0/1348887549 conn(0x563953caef20 msgr2=0x563953cb00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- 172.21.15.96:0/1348887549 shutdown_connections 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- 172.21.15.96:0/1348887549 wait complete. 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7540e5b640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 0x7f752c0b7820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75300ccec0 tx=0x7f753003b920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.510+0000 7f7545494740 1 -- start start 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 Processor -- start 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f754165c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f754165c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35972/0 (socket says 172.21.15.96:35972) 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f754165c640 1 -- 172.21.15.96:0/3341061561 learned_addr learned my addr 172.21.15.96:0/3341061561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563953a78700 con 0x563953cbd9a0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563953bb2160 con 0x563953cb1a10 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f753bfff640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563953cc0620 con 0x563953cb10f0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.511+0000 7f7545494740 10 client.0 _ll_register_callbacks cb 0x563953bcb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 -- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 msgr2=0x563953cba730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cba730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 -- 172.21.15.96:0/3341061561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 msgr2=0x563953cbadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563953bdfc90 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f754165c640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7534009bf0 tx=0x7f7534009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4c0164b2b19279 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534016070 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7534004d20 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7534022040 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75340129e0 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f7545494740 10 client.25333 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563953cd47c0 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7534004d20 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.512+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f753400f040 con 0x563953cb10f0 2024-09-10T22:48:58.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 10 client.25333 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 20 client.25333 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953cd7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 4 client.25333 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563953c29df0 con 0x563953cd5b10 2024-09-10T22:48:58.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.513+0000 7f753bfff640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953cd7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.514+0000 7f753bfff640 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953cd7f00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71dd638133eb8354 server_cookie=25ef1c9652bfaf91 in_seq=0 out_seq=0 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.514+0000 7f751bfff640 10 client.25333 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f751bfff640 1 -- 172.21.15.96:0/3341061561 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563953c29df0 con 0x563953cd5b10 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f751bfff640 10 client.25333 handle_command_reply: tid=0 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 client.25333 shutdown 2024-09-10T22:48:58.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 msgr2=0x563953cbd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7534009bf0 tx=0x7f7534009920 comp rx=0 tx=0).stop 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 shutdown_connections 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953cd7f00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f751bfff640 0 client.25333 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:58.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cbd9a0 0x563953cbadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 --2- 172.21.15.96:0/3341061561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cb10f0 0x563953cbd2e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 >> 172.21.15.96:0/3341061561 conn(0x563953caef20 msgr2=0x563953cb4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 shutdown_connections 2024-09-10T22:48:58.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.515+0000 7f7545494740 1 -- 172.21.15.96:0/3341061561 wait complete. 2024-09-10T22:48:58.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.516+0000 7f7545494740 20 client.25333 trim_cache size 0 max 16384 2024-09-10T22:48:58.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.533+0000 7f7545494740 1 Processor -- start 2024-09-10T22:48:58.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.533+0000 7f7545494740 1 -- start start 2024-09-10T22:48:58.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 0x563953cd2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 0x563953cb9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cbed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1ac0f0 con 0x563953cb1a10 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1adcf0 con 0x563953cb10f0 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f7545494740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1ac7c0 con 0x563953cbd9a0 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f754165c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 0x563953cd2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f754165c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 0x563953cd2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60674/0 (socket says 172.21.15.96:60674) 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f754165c640 1 -- 172.21.15.96:0/1587449393 learned_addr learned my addr 172.21.15.96:0/1587449393 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.534+0000 7f753bfff640 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 0x563953cb9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f754165c640 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cbed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f753bfff640 1 -- 172.21.15.96:0/1587449393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 msgr2=0x563953cbed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f753bfff640 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cbed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f753bfff640 1 -- 172.21.15.96:0/1587449393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 msgr2=0x563953cd2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f753bfff640 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 0x563953cd2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.535+0000 7f753bfff640 1 -- 172.21.15.96:0/1587449393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563953caf9a0 con 0x563953cb1a10 2024-09-10T22:48:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.536+0000 7f753bfff640 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 0x563953cb9c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563953cd85d0 tx=0x7f7528012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7408028fa72ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.536+0000 7f75267fc640 1 -- 172.21.15.96:0/1587449393 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528012d10 con 0x563953cb1a10 2024-09-10T22:48:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.536+0000 7f75267fc640 1 -- 172.21.15.96:0/1587449393 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75280035f0 con 0x563953cb1a10 2024-09-10T22:48:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.536+0000 7f75267fc640 1 -- 172.21.15.96:0/1587449393 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752801d680 con 0x563953cb1a10 2024-09-10T22:48:58.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- 172.21.15.96:0/1587449393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 msgr2=0x563953cb9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 0x563953cb9c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563953cd85d0 tx=0x7f7528012be0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- 172.21.15.96:0/1587449393 shutdown_connections 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cbed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb1a10 0x563953cb9c20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 --2- 172.21.15.96:0/1587449393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb10f0 0x563953cd2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- 172.21.15.96:0/1587449393 >> 172.21.15.96:0/1587449393 conn(0x563953caef20 msgr2=0x563953cb8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- 172.21.15.96:0/1587449393 shutdown_connections 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- 172.21.15.96:0/1587449393 wait complete. 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 -- start start 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.537+0000 7f7545494740 1 Processor -- start 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cc9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f754165c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1ad560 con 0x563953cb10f0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1ace90 con 0x563953cb1a10 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f754165c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51146/0 (socket says 172.21.15.96:51146) 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1aa130 con 0x563953cbd9a0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f754165c640 1 -- 172.21.15.96:0/1191561978 learned_addr learned my addr 172.21.15.96:0/1191561978 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f7545494740 10 client.0 _ll_register_callbacks cb 0x7f7528012760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f753bfff640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f754165c640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cc9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.538+0000 7f754165c640 1 -- 172.21.15.96:0/1191561978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 msgr2=0x563953cc9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f754165c640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cc9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f754165c640 1 -- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 msgr2=0x563953cce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f754165c640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f754165c640 1 -- 172.21.15.96:0/1191561978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563953cc97a0 con 0x563953cb10f0 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f753bfff640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f754165c640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7534005ad0 tx=0x7f7534010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8565093f452e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f751bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:48:58.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753403a020 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f753401f040 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.539+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75280035f0 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7534016040 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 10 client.15762 fetch_fsmap learned FSMap version 30 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563953d431d0 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75280035f0 con 0x563953cb10f0 2024-09-10T22:48:58.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7534005020 con 0x563953cb10f0 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 10 client.15762 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 20 client.15762 populate_metadata read hostname 'smithi096' 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953d45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 4 client.15762 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.540+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563953c258a0 con 0x563953cd5b10 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f753bfff640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953d45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:48:58.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f751bfff640 10 client.15762 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f753bfff640 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953d45880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ab8816fe3c78f3 server_cookie=d9f938c30a19a171 in_seq=0 out_seq=0 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f751bfff640 1 -- 172.21.15.96:0/1191561978 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563953c258a0 con 0x563953cd5b10 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f751bfff640 10 client.15762 handle_command_reply: tid=0 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f7545494740 1 client.15762 shutdown 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 msgr2=0x563953cce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.541+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7534005ad0 tx=0x7f7534010fa0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 shutdown_connections 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563953cd5b10 0x563953d45880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f751bfff640 0 client.15762 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563953cb1a10 0x563953cce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563953cbd9a0 0x563953cc9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 --2- 172.21.15.96:0/1191561978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563953cb10f0 0x563953cce360 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 >> 172.21.15.96:0/1191561978 conn(0x563953caef20 msgr2=0x563953cadf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 shutdown_connections 2024-09-10T22:48:58.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 1 -- 172.21.15.96:0/1191561978 wait complete. 2024-09-10T22:48:58.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f7545494740 20 client.15762 trim_cache size 0 max 16384 2024-09-10T22:48:58.552 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 msgr2=0x7f752c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 0x7f752c0b7820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75300ccec0 tx=0x7f753003b920 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 msgr2=0x7f753c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.542+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7530005ab0 tx=0x7f7530003cd0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 shutdown_connections 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c10d130 0x7f753c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f752c0b5360 0x7f752c0b7820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1045f0 0x7f753c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 --2- 172.21.15.96:0/115518497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f753c103cd0 0x7f753c1992c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 >> 172.21.15.96:0/115518497 conn(0x7f753c0fd770 msgr2=0x7f753c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 shutdown_connections 2024-09-10T22:48:58.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:48:58.543+0000 7f75430bf640 1 -- 172.21.15.96:0/115518497 wait complete. 2024-09-10T22:48:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:48:58 smithi096 ceph-mon[30869]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-10T22:48:59.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:48:58 smithi121 ceph-mon[35119]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-10T22:48:59.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:48:58 smithi117 ceph-mon[35081]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 675 op/s 2024-09-10T22:49:00.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:49:00.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:49:00.649 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:00.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:00 smithi096 ceph-mon[30869]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 816 op/s 2024-09-10T22:49:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 85.0 s, 3746.4 tps, lat 26.730 ms stddev 29.654 2024-09-10T22:49:01.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:00 smithi121 ceph-mon[35119]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 816 op/s 2024-09-10T22:49:01.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:00 smithi117 ceph-mon[35081]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 816 op/s 2024-09-10T22:49:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 -- 172.21.15.96:0/4039496228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 msgr2=0x7febcc111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 --2- 172.21.15.96:0/4039496228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc111b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febc4009ca0 tx=0x7febc402a8f0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 -- 172.21.15.96:0/4039496228 shutdown_connections 2024-09-10T22:49:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 --2- 172.21.15.96:0/4039496228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc111b50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 --2- 172.21.15.96:0/4039496228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc10a9d0 0x7febcc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 --2- 172.21.15.96:0/4039496228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc104fc0 0x7febcc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 -- 172.21.15.96:0/4039496228 >> 172.21.15.96:0/4039496228 conn(0x7febcc1009f0 msgr2=0x7febcc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 -- 172.21.15.96:0/4039496228 shutdown_connections 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.251+0000 7febd17be640 1 -- 172.21.15.96:0/4039496228 wait complete. 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.252+0000 7febd17be640 1 Processor -- start 2024-09-10T22:49:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 -- start start 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 0x7febcc111080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc10a9d0 0x7febcc1115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febcc079180 con 0x7febcc104fc0 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7febcc079000 con 0x7febcc10a9d0 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febd17be640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7febcc079300 con 0x7febcc10b350 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febcb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febcb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35992/0 (socket says 172.21.15.96:35992) 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febcb7fe640 1 -- 172.21.15.96:0/1185202376 learned_addr learned my addr 172.21.15.96:0/1185202376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.253+0000 7febca7fc640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc10a9d0 0x7febcc1115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcaffd640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 0x7febcc111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 -- 172.21.15.96:0/1185202376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc10a9d0 msgr2=0x7febcc1115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc10a9d0 0x7febcc1115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 -- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 msgr2=0x7febcc111080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 0x7febcc111080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 -- 172.21.15.96:0/1185202376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc1a5a80 con 0x7febcc10b350 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcaffd640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 0x7febcc111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.254+0000 7febcb7fe640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febc4031a50 tx=0x7febc402f4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.255+0000 7febb3fff640 1 -- 172.21.15.96:0/1185202376 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4003db0 con 0x7febcc10b350 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.255+0000 7febb3fff640 1 -- 172.21.15.96:0/1185202376 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc402d070 con 0x7febcc10b350 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.255+0000 7febb3fff640 1 -- 172.21.15.96:0/1185202376 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc402fc70 con 0x7febcc10b350 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.255+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1a5d10 con 0x7febcc10b350 2024-09-10T22:49:01.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.255+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1ac450 con 0x7febcc10b350 2024-09-10T22:49:01.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.259+0000 7febd3b93740 1 Processor -- start 2024-09-10T22:49:01.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febb3fff640 1 -- 172.21.15.96:0/1185202376 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7febc4004040 con 0x7febcc10b350 2024-09-10T22:49:01.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 -- start start 2024-09-10T22:49:01.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7628250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a7628790 0x5609a762ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609a7592e60 con 0x5609a761bee0 2024-09-10T22:49:01.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609a7582290 con 0x5609a7628790 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febd3b93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609a75860a0 con 0x5609a761c800 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febcaffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7628250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51178/0 (socket says 172.21.15.96:51178) 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febcaffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7628250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36002/0 (socket says 172.21.15.96:36002) 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febca7fc640 1 -- 172.21.15.96:0/1429555938 learned_addr learned my addr 172.21.15.96:0/1429555938 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.260+0000 7febca7fc640 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a7628790 0x5609a762ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 -- 172.21.15.96:0/1429555938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 msgr2=0x5609a7628250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7628250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 -- 172.21.15.96:0/1429555938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a7628790 msgr2=0x5609a762ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a7628790 0x5609a762ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 -- 172.21.15.96:0/1429555938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a72e0930 con 0x5609a761bee0 2024-09-10T22:49:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febb3fff640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 0x7febb40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febca7fc640 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7febc0028760 tx=0x7febc00094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd4eedf15ccdf24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.261+0000 7febb3fff640 1 -- 172.21.15.96:0/1185202376 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7febc4106f90 con 0x7febcc10b350 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.262+0000 7febcaffd640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 0x7febb40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.262+0000 7febb17fa640 1 -- 172.21.15.96:0/1429555938 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0026770 con 0x5609a761bee0 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.262+0000 7febb17fa640 1 -- 172.21.15.96:0/1429555938 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc0026dc0 con 0x5609a761bee0 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.262+0000 7febb17fa640 1 -- 172.21.15.96:0/1429555938 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0024ca0 con 0x5609a761bee0 2024-09-10T22:49:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.263+0000 7febcaffd640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 0x7febb40b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5609a761f060 tx=0x7febbc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 -- 172.21.15.96:0/1429555938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 msgr2=0x5609a761c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7febc0028760 tx=0x7febc00094f0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 -- 172.21.15.96:0/1429555938 shutdown_connections 2024-09-10T22:49:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a7628790 0x5609a762ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7628250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 --2- 172.21.15.96:0/1429555938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a761c2c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 -- 172.21.15.96:0/1429555938 >> 172.21.15.96:0/1429555938 conn(0x5609a7619d10 msgr2=0x5609a761aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 -- 172.21.15.96:0/1429555938 shutdown_connections 2024-09-10T22:49:01.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.264+0000 7febd3b93740 1 -- 172.21.15.96:0/1429555938 wait complete. 2024-09-10T22:49:01.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 -- start start 2024-09-10T22:49:01.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 Processor -- start 2024-09-10T22:49:01.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7625ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a76261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609a73e3890 con 0x5609a7628790 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609a751cd40 con 0x5609a761bee0 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609a762b3e0 con 0x5609a761c800 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.265+0000 7febd3b93740 10 client.0 _ll_register_callbacks cb 0x5609a7580c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60708/0 (socket says 172.21.15.96:60708) 2024-09-10T22:49:01.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 -- 172.21.15.96:0/2611599010 learned_addr learned my addr 172.21.15.96:0/2611599010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:01.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febca7fc640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7625ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcb7fe640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a76261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 -- 172.21.15.96:0/2611599010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 msgr2=0x5609a7625ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7625ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 -- 172.21.15.96:0/2611599010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 msgr2=0x5609a76261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a76261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 -- 172.21.15.96:0/2611599010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a76347f0 con 0x5609a761bee0 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febcaffd640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febbc0157c0 tx=0x7febbc004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b228919a4256402 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.266+0000 7febaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc023070 con 0x5609a761bee0 2024-09-10T22:49:01.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7febbc013b20 con 0x5609a761bee0 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc0026dc0 con 0x5609a761bee0 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febbc02f050 con 0x5609a761bee0 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 10 client.25334 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5609a7637200 con 0x5609a761bee0 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc0026dc0 con 0x5609a761bee0 2024-09-10T22:49:01.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7febbc01e070 con 0x5609a761bee0 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 10 client.25334 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 20 client.25334 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a763aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 4 client.25334 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:01.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a7594db0 con 0x5609a7638660 2024-09-10T22:49:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.267+0000 7febca7fc640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a763aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.268+0000 7febaeff5640 10 client.25334 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.268+0000 7febca7fc640 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a763aad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a98ffaffb2bbfa server_cookie=a8edda228b6434f in_seq=0 out_seq=0 2024-09-10T22:49:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febaeff5640 1 -- 172.21.15.96:0/2611599010 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609a7594db0 con 0x5609a7638660 2024-09-10T22:49:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febaeff5640 10 client.25334 handle_command_reply: tid=0 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 client.25334 shutdown 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 msgr2=0x5609a7625760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febbc0157c0 tx=0x7febbc004700 comp rx=0 tx=0).stop 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 shutdown_connections 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a763aad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a76261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761c800 0x5609a7625ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febaeff5640 0 client.25334 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 --2- 172.21.15.96:0/2611599010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761bee0 0x5609a7625760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 >> 172.21.15.96:0/2611599010 conn(0x5609a7619d10 msgr2=0x5609a7618f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 shutdown_connections 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 1 -- 172.21.15.96:0/2611599010 wait complete. 2024-09-10T22:49:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.269+0000 7febd3b93740 20 client.25334 trim_cache size 0 max 16384 2024-09-10T22:49:01.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 Processor -- start 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 -- start start 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a7624ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7625de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a7628790 0x5609a7622990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febcc074ea0 con 0x5609a761bee0 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1adda0 con 0x5609a761c800 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febd3b93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1ac870 con 0x5609a7628790 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febcaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a7624ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febcaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a7624ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51190/0 (socket says 172.21.15.96:51190) 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.293+0000 7febcaffd640 1 -- 172.21.15.96:0/1706277846 learned_addr learned my addr 172.21.15.96:0/1706277846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:01.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7625de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febcb7fe640 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a7628790 0x5609a7622990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 -- 172.21.15.96:0/1706277846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a7628790 msgr2=0x5609a7622990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a7628790 0x5609a7622990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 -- 172.21.15.96:0/1706277846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 msgr2=0x5609a7624ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a7624ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 -- 172.21.15.96:0/1706277846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a761c630 con 0x5609a761c800 2024-09-10T22:49:01.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.294+0000 7febca7fc640 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7625de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7febc00248c0 tx=0x7febc0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e069af64f350d8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febb0ff9640 1 -- 172.21.15.96:0/1706277846 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0024660 con 0x5609a761c800 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febb0ff9640 1 -- 172.21.15.96:0/1706277846 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc0024da0 con 0x5609a761c800 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 -- 172.21.15.96:0/1706277846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 msgr2=0x5609a7625de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7625de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7febc00248c0 tx=0x7febc0008040 comp rx=0 tx=0).stop 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 -- 172.21.15.96:0/1706277846 shutdown_connections 2024-09-10T22:49:01.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7625de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a7628790 0x5609a7622990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 --2- 172.21.15.96:0/1706277846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a761bee0 0x5609a7624ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.295+0000 7febd3b93740 1 -- 172.21.15.96:0/1706277846 >> 172.21.15.96:0/1706277846 conn(0x5609a7619d10 msgr2=0x5609a76256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- 172.21.15.96:0/1706277846 shutdown_connections 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- 172.21.15.96:0/1706277846 wait complete. 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- start start 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 Processor -- start 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761bee0 0x5609a7630880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a762d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1ad610 con 0x5609a7628790 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1acf40 con 0x5609a761c800 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.296+0000 7febd3b93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7febcc072ea0 con 0x5609a761bee0 2024-09-10T22:49:01.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febd3b93740 10 client.0 _ll_register_callbacks cb 0x7febbc00a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60722/0 (socket says 172.21.15.96:60722) 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 -- 172.21.15.96:0/674871048 learned_addr learned my addr 172.21.15.96:0/674871048 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febcaffd640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761bee0 0x5609a7630880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 -- 172.21.15.96:0/674871048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761bee0 msgr2=0x5609a7630880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761bee0 0x5609a7630880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 -- 172.21.15.96:0/674871048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 msgr2=0x5609a762d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a762d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.297+0000 7febca7fc640 1 -- 172.21.15.96:0/674871048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a762dea0 con 0x5609a761c800 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febca7fc640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0028450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe315599391421bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:01.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:01.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0024660 con 0x5609a761c800 2024-09-10T22:49:01.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7febc0004040 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc0024da0 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febc00248f0 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febd3b93740 10 client.25340 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.298+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5609a76a60d0 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc0024da0 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7febc0042070 con 0x5609a761c800 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 10 client.25340 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 10 client.25340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 20 client.25340 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a76a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 4 client.25340 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609a7590860 con 0x5609a7638660 2024-09-10T22:49:01.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.299+0000 7febcaffd640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a76a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:01.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.300+0000 7febcaffd640 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a76a8840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b682ac359ff16d6 server_cookie=8b26cbd81388ceb7 in_seq=0 out_seq=0 2024-09-10T22:49:01.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.300+0000 7febaeff5640 10 client.25340 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:01.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.301+0000 7febaeff5640 1 -- 172.21.15.96:0/674871048 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609a7590860 con 0x5609a7638660 2024-09-10T22:49:01.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.301+0000 7febaeff5640 10 client.25340 handle_command_reply: tid=0 2024-09-10T22:49:01.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.301+0000 7febd3b93740 1 client.25340 shutdown 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 msgr2=0x5609a7630e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0028450 comp rx=0 tx=0).stop 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 shutdown_connections 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5609a7638660 0x5609a76a8840 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609a761c800 0x5609a7630e00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a7628790 0x5609a762d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 --2- 172.21.15.96:0/674871048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609a761bee0 0x5609a7630880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 >> 172.21.15.96:0/674871048 conn(0x5609a7619d10 msgr2=0x5609a7625ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febaeff5640 0 client.25340 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 shutdown_connections 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 1 -- 172.21.15.96:0/674871048 wait complete. 2024-09-10T22:49:01.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.303+0000 7febd3b93740 20 client.25340 trim_cache size 0 max 16384 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "125bbf28-a536-4687-81aa-19c7bbe31e42", 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:49:01.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 msgr2=0x7febb40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 0x7febb40b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5609a761f060 tx=0x7febbc008040 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 msgr2=0x7febcc1a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febc4031a50 tx=0x7febc402f4e0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 shutdown_connections 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febcc10a9d0 0x7febcc1115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7febb40b5360 0x7febb40b7820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febcc10b350 0x7febcc1a52b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 --2- 172.21.15.96:0/1185202376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febcc104fc0 0x7febcc111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 >> 172.21.15.96:0/1185202376 conn(0x7febcc1009f0 msgr2=0x7febcc108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 shutdown_connections 2024-09-10T22:49:01.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:01.304+0000 7febd17be640 1 -- 172.21.15.96:0/1185202376 wait complete. 2024-09-10T22:49:02.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:49:02.928 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 769 op/s 2024-09-10T22:49:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:02 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 769 op/s 2024-09-10T22:49:03.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:02 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:49:03.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:49:03.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:49:03.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 MiB/s wr, 769 op/s 2024-09-10T22:49:03.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:02 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 -- 172.21.15.96:0/2849081843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 msgr2=0x7f125c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 --2- 172.21.15.96:0/2849081843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c101ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f124c0098c0 tx=0x7f124c02c570 comp rx=0 tx=0).stop 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 -- 172.21.15.96:0/2849081843 shutdown_connections 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 --2- 172.21.15.96:0/2849081843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107e90 0x7f125c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 --2- 172.21.15.96:0/2849081843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107510 0x7f125c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 --2- 172.21.15.96:0/2849081843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c101ee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 -- 172.21.15.96:0/2849081843 >> 172.21.15.96:0/2849081843 conn(0x7f125c0fd5b0 msgr2=0x7f125c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 -- 172.21.15.96:0/2849081843 shutdown_connections 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.492+0000 7f125b59e640 1 -- 172.21.15.96:0/2849081843 wait complete. 2024-09-10T22:49:03.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 Processor -- start 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 -- start start 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107510 0x7f125c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110a90 con 0x7f125c107e90 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110910 con 0x7f125c107510 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125b59e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110c10 con 0x7f125c101b00 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51216/0 (socket says 172.21.15.96:51216) 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125ad9d640 1 -- 172.21.15.96:0/3496909826 learned_addr learned my addr 172.21.15.96:0/3496909826 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f125a59c640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.493+0000 7f1253fff640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107510 0x7f125c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.494+0000 7f125ad9d640 1 -- 172.21.15.96:0/3496909826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 msgr2=0x7f125c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.494+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.494+0000 7f125ad9d640 1 -- 172.21.15.96:0/3496909826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107510 msgr2=0x7f125c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.494+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107510 0x7f125c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.494+0000 7f125ad9d640 1 -- 172.21.15.96:0/3496909826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a1260 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.495+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f125400eaa0 tx=0x7f125400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.495+0000 7f12537fe640 1 -- 172.21.15.96:0/3496909826 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254004d20 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.495+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a1550 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.496+0000 7f12537fe640 1 -- 172.21.15.96:0/3496909826 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1254005020 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.496+0000 7f12537fe640 1 -- 172.21.15.96:0/3496909826 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125401f060 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.496+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a7c10 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.499+0000 7f126312b740 1 Processor -- start 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f12537fe640 1 -- 172.21.15.96:0/3496909826 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f125400f040 con 0x7f125c107e90 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 -- start start 2024-09-10T22:49:03.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d23902da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d2390399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d238fa3e50 con 0x55d23902df70 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d238f93280 con 0x55d23902d650 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.500+0000 7f126312b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d238f97090 con 0x55d239039f00 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f125a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d2390399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36070/0 (socket says 172.21.15.96:36070) 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 -- 172.21.15.96:0/2400095591 learned_addr learned my addr 172.21.15.96:0/2400095591 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d23902da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 -- 172.21.15.96:0/2400095591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 msgr2=0x55d23902da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d23902da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 -- 172.21.15.96:0/2400095591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 msgr2=0x55d2390399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d2390399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 -- 172.21.15.96:0/2400095591 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d238cf1650 con 0x55d239039f00 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f125a59c640 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d2390399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f12537fe640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 0x7f12480b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1253fff640 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1244009f90 tx=0x7f1244005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12cc38c9f0535a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1250ff9640 1 -- 172.21.15.96:0/2400095591 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124401e420 con 0x55d239039f00 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1250ff9640 1 -- 172.21.15.96:0/2400095591 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124401ea70 con 0x55d239039f00 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f1250ff9640 1 -- 172.21.15.96:0/2400095591 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124400fa00 con 0x55d239039f00 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.501+0000 7f12537fe640 1 -- 172.21.15.96:0/3496909826 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f12540e58c0 con 0x7f125c107e90 2024-09-10T22:49:03.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f125a59c640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 0x7f12480b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 -- 172.21.15.96:0/2400095591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 msgr2=0x55d23903c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1244009f90 tx=0x7f1244005850 comp rx=0 tx=0).stop 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f125a59c640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 0x7f12480b7850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e4b0 tx=0x7f124c00e410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 -- 172.21.15.96:0/2400095591 shutdown_connections 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23903c2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d2390399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 --2- 172.21.15.96:0/2400095591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d23902da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 -- 172.21.15.96:0/2400095591 >> 172.21.15.96:0/2400095591 conn(0x55d23902b480 msgr2=0x55d23902c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 -- 172.21.15.96:0/2400095591 shutdown_connections 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.502+0000 7f126312b740 1 -- 172.21.15.96:0/2400095591 wait complete. 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 -- start start 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 Processor -- start 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239036d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d2390373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d238df4700 con 0x55d239039f00 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d238f2e110 con 0x55d23902d650 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f126312b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d23903cb80 con 0x55d23902df70 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f125a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f125a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60756/0 (socket says 172.21.15.96:60756) 2024-09-10T22:49:03.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.503+0000 7f125a59c640 1 -- 172.21.15.96:0/5054885 learned_addr learned my addr 172.21.15.96:0/5054885 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f126312b740 10 client.0 _ll_register_callbacks cb 0x55d238f47990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 -- 172.21.15.96:0/5054885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 msgr2=0x55d239036d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239036d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 -- 172.21.15.96:0/5054885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 msgr2=0x55d2390373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d2390373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 -- 172.21.15.96:0/5054885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d238f5bc40 con 0x55d23902d650 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f125a59c640 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e480 tx=0x7f124c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84fa60028f8462d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.504+0000 7f1241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.505+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c032d80 con 0x55d23902d650 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f124c0048d0 con 0x55d23902d650 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124401ea70 con 0x55d23902d650 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f124c004ad0 con 0x55d23902d650 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 10 client.25352 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:03.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2390485e0 con 0x55d23902d650 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124401ea70 con 0x55d23902d650 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f124c046030 con 0x55d23902d650 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 10 client.25352 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 10 client.25352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 20 client.25352 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d239049af0 0x55d23904bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 4 client.25352 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.506+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d238fa5da0 con 0x55d239049af0 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.507+0000 7f1253fff640 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d239049af0 0x55d23904bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.507+0000 7f1253fff640 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d239049af0 0x55d23904bf40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f185174eee992d server_cookie=293969cc4bbe308 in_seq=0 out_seq=0 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.507+0000 7f1241ffb640 10 client.25352 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f1241ffb640 1 -- 172.21.15.96:0/5054885 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d238fa5da0 con 0x55d239049af0 2024-09-10T22:49:03.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f1241ffb640 10 client.25352 handle_command_reply: tid=0 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 client.25352 shutdown 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 msgr2=0x55d2390398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e480 tx=0x7f124c005f90 comp rx=0 tx=0).stop 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 shutdown_connections 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d239049af0 0x55d23904bf40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d2390373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239036d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 --2- 172.21.15.96:0/5054885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d2390398b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 >> 172.21.15.96:0/5054885 conn(0x55d23902b480 msgr2=0x55d239031020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f1241ffb640 0 client.25352 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 shutdown_connections 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 1 -- 172.21.15.96:0/5054885 wait complete. 2024-09-10T22:49:03.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.508+0000 7f126312b740 20 client.25352 trim_cache size 0 max 16384 2024-09-10T22:49:03.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 Processor -- start 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 -- start start 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d23902c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23902ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1b02d0 con 0x55d23902df70 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1b1ed0 con 0x55d23902d650 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.532+0000 7f126312b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1b09a0 con 0x55d239039f00 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f1253fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d23902c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60766/0 (socket says 172.21.15.96:60766) 2024-09-10T22:49:03.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 -- 172.21.15.96:0/924726497 learned_addr learned my addr 172.21.15.96:0/924726497 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125ad9d640 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23902ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 -- 172.21.15.96:0/924726497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 msgr2=0x55d23902ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23902ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 -- 172.21.15.96:0/924726497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 msgr2=0x55d23902c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d23902c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 -- 172.21.15.96:0/924726497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d239037130 con 0x55d23902d650 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f125a59c640 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f124c032920 tx=0x7f124c0056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=456afba24fb1f0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f1243fff640 1 -- 172.21.15.96:0/924726497 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c032b20 con 0x55d23902d650 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f1243fff640 1 -- 172.21.15.96:0/924726497 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124c061070 con 0x55d23902d650 2024-09-10T22:49:03.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.533+0000 7f1243fff640 1 -- 172.21.15.96:0/924726497 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c004ed0 con 0x55d23902d650 2024-09-10T22:49:03.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 -- 172.21.15.96:0/924726497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 msgr2=0x55d239045db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f124c032920 tx=0x7f124c0056e0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 -- 172.21.15.96:0/924726497 shutdown_connections 2024-09-10T22:49:03.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d239039f00 0x55d23902ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d23902df70 0x55d23902c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 --2- 172.21.15.96:0/924726497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239045db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.534+0000 7f126312b740 1 -- 172.21.15.96:0/924726497 >> 172.21.15.96:0/924726497 conn(0x55d23902b480 msgr2=0x55d239031f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.535+0000 7f126312b740 1 -- 172.21.15.96:0/924726497 shutdown_connections 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.535+0000 7f126312b740 1 -- 172.21.15.96:0/924726497 wait complete. 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.535+0000 7f126312b740 1 -- start start 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.535+0000 7f126312b740 1 Processor -- start 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239042090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239042610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1b1740 con 0x55d239039f00 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1b1070 con 0x55d23902d650 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1ae2b0 con 0x55d23902df70 2024-09-10T22:49:03.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f1253fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239042610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 10 client.0 _ll_register_callbacks cb 0x7f124c005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51272/0 (socket says 172.21.15.96:51272) 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 -- 172.21.15.96:0/3094853505 learned_addr learned my addr 172.21.15.96:0/3094853505 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125a59c640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239042090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 -- 172.21.15.96:0/3094853505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 msgr2=0x55d239042610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239042610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 -- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 msgr2=0x55d239042090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239042090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 -- 172.21.15.96:0/3094853505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2390517c0 con 0x55d239039f00 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125a59c640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239042090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f125ad9d640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1254009270 tx=0x7f12540a80d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73732b874e92aa56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:03.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f1241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12540ac820 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124c061070 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.536+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f12540ab380 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12540d30c0 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2390419c0 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124c061070 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f12540cdb00 con 0x55d239039f00 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 20 client.15783 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d2390b6e40 0x55d2390b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 4 client.15783 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:49:03.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d238fa1850 con 0x55d2390b6e40 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.537+0000 7f125a59c640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d2390b6e40 0x55d2390b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f1241ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f125a59c640 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d2390b6e40 0x55d2390b9260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d28895d6c35511 server_cookie=64798a83bd0503d1 in_seq=0 out_seq=0 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f1241ffb640 1 -- 172.21.15.96:0/3094853505 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55d238fa1850 con 0x55d2390b6e40 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f1241ffb640 10 client.15783 handle_command_reply: tid=0 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 client.15783 shutdown 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 msgr2=0x55d239050f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1254009270 tx=0x7f12540a80d0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 shutdown_connections 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d2390b6e40 0x55d2390b9260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d239039f00 0x55d239050f90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d23902df70 0x55d239042610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f1241ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 --2- 172.21.15.96:0/3094853505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d23902d650 0x55d239042090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.538+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 >> 172.21.15.96:0/3094853505 conn(0x55d23902b480 msgr2=0x55d23902c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.539+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 shutdown_connections 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.539+0000 7f126312b740 1 -- 172.21.15.96:0/3094853505 wait complete. 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.540+0000 7f126312b740 20 client.15783 trim_cache size 0 max 16384 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "125bbf28-a536-4687-81aa-19c7bbe31e42", 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:03.549 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:49:03.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 msgr2=0x7f12480b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 0x7f12480b7850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e4b0 tx=0x7f124c00e410 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 msgr2=0x7f125c1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f125400eaa0 tx=0x7f125400eea0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 shutdown_connections 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125c107510 0x7f125c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12480b5390 0x7f12480b7850 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f125c107e90 0x7f125c1a0a90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 --2- 172.21.15.96:0/3496909826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f125c101b00 0x7f125c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 >> 172.21.15.96:0/3496909826 conn(0x7f125c0fd5b0 msgr2=0x7f125c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 shutdown_connections 2024-09-10T22:49:03.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:03.541+0000 7f125b59e640 1 -- 172.21.15.96:0/3496909826 wait complete. 2024-09-10T22:49:04.279 INFO:tasks.cephfs.filesystem:scrub status for tag:125bbf28-a536-4687-81aa-19c7bbe31e42 - {'path': '/', 'tag': '125bbf28-a536-4687-81aa-19c7bbe31e42', 'options': 'recursive,force'} 2024-09-10T22:49:05.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:04 smithi096 ceph-mon[30869]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 820 op/s 2024-09-10T22:49:05.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:04 smithi121 ceph-mon[35119]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 820 op/s 2024-09-10T22:49:05.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:04 smithi117 ceph-mon[35081]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 820 op/s 2024-09-10T22:49:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 90.0 s, 3102.0 tps, lat 32.212 ms stddev 35.486 2024-09-10T22:49:07.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:06 smithi096 ceph-mon[30869]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2024-09-10T22:49:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:06 smithi121 ceph-mon[35119]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2024-09-10T22:49:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:06 smithi117 ceph-mon[35081]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2024-09-10T22:49:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.888+0000 7f94acf7c640 1 -- 172.21.15.96:0/2090073299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810a150 msgr2=0x7f94a810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.888+0000 7f94acf7c640 1 --2- 172.21.15.96:0/2090073299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810a150 0x7f94a810a5d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c02a400 comp rx=0 tx=0).stop 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 -- 172.21.15.96:0/2090073299 shutdown_connections 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 --2- 172.21.15.96:0/2090073299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810ab10 0x7f94a81136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 --2- 172.21.15.96:0/2090073299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810a150 0x7f94a810a5d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 --2- 172.21.15.96:0/2090073299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 0x7f94a8074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 -- 172.21.15.96:0/2090073299 >> 172.21.15.96:0/2090073299 conn(0x7f94a806fa40 msgr2=0x7f94a8071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 -- 172.21.15.96:0/2090073299 shutdown_connections 2024-09-10T22:49:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.889+0000 7f94acf7c640 1 -- 172.21.15.96:0/2090073299 wait complete. 2024-09-10T22:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 Processor -- start 2024-09-10T22:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 -- start start 2024-09-10T22:49:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 0x7f94a81a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 0x7f94a81a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8116e10 con 0x7f94a810ab10 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8116c90 con 0x7f94a810a150 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94acf7c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8116f90 con 0x7f94a8074470 2024-09-10T22:49:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.890+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 0x7f94a81a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56890/0 (socket says 172.21.15.96:56890) 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a6d9d640 1 -- 172.21.15.96:0/505335231 learned_addr learned my addr 172.21.15.96:0/505335231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a659c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 0x7f94a81a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 -- 172.21.15.96:0/505335231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 msgr2=0x7f94a81a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 0x7f94a81a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 -- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 msgr2=0x7f94a81a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 -- 172.21.15.96:0/505335231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81a30f0 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.891+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 0x7f94a81a1c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f949c006010 tx=0x7f949c005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.892+0000 7f94977fe640 1 -- 172.21.15.96:0/505335231 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c035070 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.892+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81a3380 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.892+0000 7f94977fe640 1 -- 172.21.15.96:0/505335231 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949c004470 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.892+0000 7f94977fe640 1 -- 172.21.15.96:0/505335231 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c003df0 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.892+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a810b440 con 0x7f94a810a150 2024-09-10T22:49:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.895+0000 7f94afe25740 1 Processor -- start 2024-09-10T22:49:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.895+0000 7f94afe25740 1 -- start start 2024-09-10T22:49:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94afe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56898/0 (socket says 172.21.15.96:56898) 2024-09-10T22:49:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94a6d9d640 1 -- 172.21.15.96:0/1838016696 learned_addr learned my addr 172.21.15.96:0/1838016696 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e944c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e944cbd0 0x5641e944f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641e93b96f0 con 0x5641e9447860 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.896+0000 7f94977fe640 1 -- 172.21.15.96:0/505335231 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f949c03d760 con 0x7f94a810a150 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a659c640 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e944c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641e93a8b20 con 0x5641e944cbd0 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641e93ac930 con 0x5641e944a190 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e944cbd0 0x5641e944f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 -- 172.21.15.96:0/1838016696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 msgr2=0x5641e944c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e944c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 -- 172.21.15.96:0/1838016696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e944cbd0 msgr2=0x5641e944f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e944cbd0 0x5641e944f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 -- 172.21.15.96:0/1838016696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e936e530 con 0x5641e9447860 2024-09-10T22:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.897+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011890 tx=0x7f94a0011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363453b071f43d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.898+0000 7f9494ff9640 1 -- 172.21.15.96:0/1838016696 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0011310 con 0x5641e9447860 2024-09-10T22:49:07.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.898+0000 7f9494ff9640 1 -- 172.21.15.96:0/1838016696 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a000f9a0 con 0x5641e9447860 2024-09-10T22:49:07.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.898+0000 7f9494ff9640 1 -- 172.21.15.96:0/1838016696 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0005600 con 0x5641e9447860 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.898+0000 7f94977fe640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 0x7f94900b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.898+0000 7f94a659c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 0x7f94900b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 msgr2=0x5641e9449c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011890 tx=0x7f94a0011d50 comp rx=0 tx=0).stop 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94977fe640 1 -- 172.21.15.96:0/505335231 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f949c10a3c0 con 0x7f94a810a150 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 shutdown_connections 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e944cbd0 0x5641e944f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e944c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 --2- 172.21.15.96:0/1838016696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e9447860 0x5641e9449c50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 >> 172.21.15.96:0/1838016696 conn(0x5641e94429b0 msgr2=0x5641e9443740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94a659c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 0x7f94900b78a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9498009da0 tx=0x7f9498005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 shutdown_connections 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- 172.21.15.96:0/1838016696 wait complete. 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.899+0000 7f94afe25740 1 -- start start 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 Processor -- start 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9450e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 0x5641e9451500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944cbd0 0x5641e94520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641e92e0ab0 con 0x5641e944a190 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641e934e730 con 0x5641e9447860 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641e944fbb0 con 0x5641e944cbd0 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 0x5641e9451500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94afe25740 10 client.0 _ll_register_callbacks cb 0x5641e93a26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9450e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 0x5641e9451500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56902/0 (socket says 172.21.15.96:56902) 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94a5d9b640 1 -- 172.21.15.96:0/1104052438 learned_addr learned my addr 172.21.15.96:0/1104052438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.900+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944cbd0 0x5641e94520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 -- 172.21.15.96:0/1104052438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944cbd0 msgr2=0x5641e94520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944cbd0 0x5641e94520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 -- 172.21.15.96:0/1104052438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 msgr2=0x5641e9451500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 0x5641e9451500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 -- 172.21.15.96:0/1104052438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e910a790 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94a659c640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9450e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9498012cb0 tx=0x7f9498004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a4e7ebd05c9a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f948dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498004030 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a000f9a0 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9498018030 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94980041d0 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94afe25740 10 client.25370 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.901+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641e9183620 con 0x5641e9447860 2024-09-10T22:49:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a000f9a0 con 0x5641e9447860 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9498029410 con 0x5641e9447860 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 10 client.25370 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 20 client.25370 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e945ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 4 client.25370 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e93bb640 con 0x5641e945c8b0 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e945ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f948dffb640 10 client.25370 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.902+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e945ed10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43baf4fe3f6a177e server_cookie=65cdc8e9bb4d40aa in_seq=0 out_seq=0 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f948dffb640 1 -- 172.21.15.96:0/1104052438 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641e93bb640 con 0x5641e945c8b0 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f948dffb640 10 client.25370 handle_command_reply: tid=0 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 client.25370 shutdown 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 msgr2=0x5641e9450e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9450e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9498012cb0 tx=0x7f9498004730 comp rx=0 tx=0).stop 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 shutdown_connections 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e945ed10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944cbd0 0x5641e94520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f948dffb640 0 client.25370 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944a190 0x5641e9451500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 --2- 172.21.15.96:0/1104052438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9450e60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 >> 172.21.15.96:0/1104052438 conn(0x5641e94429b0 msgr2=0x5641e9441bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 shutdown_connections 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 1 -- 172.21.15.96:0/1104052438 wait complete. 2024-09-10T22:49:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.903+0000 7f94afe25740 20 client.25370 trim_cache size 0 max 16384 2024-09-10T22:49:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.924+0000 7f94afe25740 1 Processor -- start 2024-09-10T22:49:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.924+0000 7f94afe25740 1 -- start start 2024-09-10T22:49:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.924+0000 7f94afe25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e94432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.924+0000 7f94afe25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94afe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9443830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59874/0 (socket says 172.21.15.96:59874) 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e94432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 -- 172.21.15.96:0/1195563073 learned_addr learned my addr 172.21.15.96:0/1195563073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b0340 con 0x5641e944cbd0 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b1f40 con 0x5641e9447860 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b0a10 con 0x5641e944a190 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9443830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 -- 172.21.15.96:0/1195563073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 msgr2=0x5641e94432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e94432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 -- 172.21.15.96:0/1195563073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 msgr2=0x5641e9443830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9443830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 -- 172.21.15.96:0/1195563073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e943e2d0 con 0x5641e944a190 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.925+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f949c02a920 tx=0x7f949c03f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5e4a95476cd2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f948ffff640 1 -- 172.21.15.96:0/1195563073 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0cca20 con 0x5641e944a190 2024-09-10T22:49:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f948ffff640 1 -- 172.21.15.96:0/1195563073 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949c0d0690 con 0x5641e944a190 2024-09-10T22:49:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f948ffff640 1 -- 172.21.15.96:0/1195563073 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0cc2f0 con 0x5641e944a190 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 msgr2=0x5641e9459150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f949c02a920 tx=0x7f949c03f040 comp rx=0 tx=0).stop 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 shutdown_connections 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9443830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9459150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 --2- 172.21.15.96:0/1195563073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e94432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 >> 172.21.15.96:0/1195563073 conn(0x5641e94429b0 msgr2=0x5641e9450a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 shutdown_connections 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.926+0000 7f94afe25740 1 -- 172.21.15.96:0/1195563073 wait complete. 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 -- start start 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 Processor -- start 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9457810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9457d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b17b0 con 0x5641e944cbd0 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81b10e0 con 0x5641e9447860 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81ae320 con 0x5641e944a190 2024-09-10T22:49:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94afe25740 10 client.0 _ll_register_callbacks cb 0x7f949801e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9457d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:59660/0 (socket says 172.21.15.96:59660) 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94a659c640 1 -- 172.21.15.96:0/907517601 learned_addr learned my addr 172.21.15.96:0/907517601 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.927+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9457810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a659c640 1 -- 172.21.15.96:0/907517601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 msgr2=0x5641e9457810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a659c640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9457810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a659c640 1 -- 172.21.15.96:0/907517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 msgr2=0x5641e9457d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a659c640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9457d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a659c640 1 -- 172.21.15.96:0/907517601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e9458290 con 0x5641e9447860 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.928+0000 7f94a6d9d640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9457d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f94a659c640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9498005c20 tx=0x7f9498029d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7c3b90c4652b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f948dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498042020 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9498029620 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c0d0690 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9498038070 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f94afe25740 10 client.25376 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641e9457110 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.929+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94980054d0 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f949803e440 con 0x5641e9447860 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 10 client.25376 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 20 client.25376 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e94cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 4 client.25376 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:49:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f949c0d0690 con 0x5641e945c8b0 2024-09-10T22:49:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e94cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.930+0000 7f948dffb640 10 client.25376 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.931+0000 7f94a5d9b640 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e94cc120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c00e858ef53d60 server_cookie=6889994d213a6abb in_seq=0 out_seq=0 2024-09-10T22:49:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f948dffb640 1 -- 172.21.15.96:0/907517601 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f949c0d0690 con 0x5641e945c8b0 2024-09-10T22:49:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f948dffb640 10 client.25376 handle_command_reply: tid=0 2024-09-10T22:49:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 client.25376 shutdown 2024-09-10T22:49:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 msgr2=0x5641e9458b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9498005c20 tx=0x7f9498029d20 comp rx=0 tx=0).stop 2024-09-10T22:49:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 shutdown_connections 2024-09-10T22:49:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641e945c8b0 0x5641e94cc120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641e944cbd0 0x5641e9457d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641e944a190 0x5641e9457810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 --2- 172.21.15.96:0/907517601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641e9447860 0x5641e9458b10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.952+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 >> 172.21.15.96:0/907517601 conn(0x5641e94429b0 msgr2=0x5641e94432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f948dffb640 0 client.25376 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 shutdown_connections 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f94afe25740 1 -- 172.21.15.96:0/907517601 wait complete. 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f94afe25740 20 client.25376 trim_cache size 0 max 16384 2024-09-10T22:49:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:49:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:49:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:49:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 msgr2=0x7f94900b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.953+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 0x7f94900b78a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9498009da0 tx=0x7f9498005f90 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 msgr2=0x7f94a81a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 0x7f94a81a1c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f949c006010 tx=0x7f949c005e70 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 shutdown_connections 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a810a150 0x7f94a81a1c00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94900b53e0 0x7f94900b78a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94a810ab10 0x7f94a81a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 --2- 172.21.15.96:0/505335231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8074470 0x7f94a81a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 >> 172.21.15.96:0/505335231 conn(0x7f94a806fa40 msgr2=0x7f94a810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 shutdown_connections 2024-09-10T22:49:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:07.954+0000 7f94acf7c640 1 -- 172.21.15.96:0/505335231 wait complete. 2024-09-10T22:49:09.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:08 smithi096 ceph-mon[30869]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.8 MiB/s wr, 830 op/s 2024-09-10T22:49:09.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:08 smithi121 ceph-mon[35119]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.8 MiB/s wr, 830 op/s 2024-09-10T22:49:09.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:08 smithi117 ceph-mon[35081]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.8 MiB/s wr, 830 op/s 2024-09-10T22:49:10.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:09 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:49:10.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:09 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:49:10.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:09 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:49:10.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:10 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:49:10.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:10 smithi117 ceph-mon[35081]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.7 MiB/s wr, 917 op/s 2024-09-10T22:49:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 95.0 s, 3623.0 tps, lat 27.588 ms stddev 28.718 2024-09-10T22:49:11.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:10 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:49:11.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:10 smithi096 ceph-mon[30869]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.7 MiB/s wr, 917 op/s 2024-09-10T22:49:11.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:10 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:49:11.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:10 smithi121 ceph-mon[35119]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.7 MiB/s wr, 917 op/s 2024-09-10T22:49:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:12 smithi096 ceph-mon[30869]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.7 MiB/s wr, 810 op/s 2024-09-10T22:49:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:12 smithi121 ceph-mon[35119]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.7 MiB/s wr, 810 op/s 2024-09-10T22:49:13.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:12 smithi117 ceph-mon[35081]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.7 MiB/s wr, 810 op/s 2024-09-10T22:49:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:15 smithi096 ceph-mon[30869]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 956 op/s 2024-09-10T22:49:15.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:15 smithi121 ceph-mon[35119]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 956 op/s 2024-09-10T22:49:15.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:15 smithi117 ceph-mon[35081]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 956 op/s 2024-09-10T22:49:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 100.0 s, 3470.8 tps, lat 28.861 ms stddev 29.254 2024-09-10T22:49:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:17 smithi096 ceph-mon[30869]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T22:49:17.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:17 smithi121 ceph-mon[35119]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T22:49:17.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:17 smithi117 ceph-mon[35081]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T22:49:19.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:19 smithi096 ceph-mon[30869]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 768 op/s 2024-09-10T22:49:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:19 smithi121 ceph-mon[35119]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 768 op/s 2024-09-10T22:49:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:19 smithi117 ceph-mon[35081]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 768 op/s 2024-09-10T22:49:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 105.0 s, 3810.4 tps, lat 26.150 ms stddev 26.230 2024-09-10T22:49:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 -- 172.21.15.96:0/386456152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c10103380 msgr2=0x7f5c10109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 --2- 172.21.15.96:0/386456152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c10103380 0x7f5c10109d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009cd0 tx=0x7f5c0802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 -- 172.21.15.96:0/386456152 shutdown_connections 2024-09-10T22:49:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 --2- 172.21.15.96:0/386456152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c101029c0 0x7f5c10102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 --2- 172.21.15.96:0/386456152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c10103380 0x7f5c10109d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 --2- 172.21.15.96:0/386456152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c10101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 -- 172.21.15.96:0/386456152 >> 172.21.15.96:0/386456152 conn(0x7f5c100fd130 msgr2=0x7f5c100ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 -- 172.21.15.96:0/386456152 shutdown_connections 2024-09-10T22:49:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.074+0000 7f5c15ced640 1 -- 172.21.15.96:0/386456152 wait complete. 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.075+0000 7f5c15ced640 1 Processor -- start 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 -- start start 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c101029c0 0x7f5c1019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10103380 0x7f5c101a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:34384/0 (socket says 172.21.15.96:34384) 2024-09-10T22:49:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/957272700 learned_addr learned my addr 172.21.15.96:0/957272700 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010d4e0 con 0x7f5c101029c0 2024-09-10T22:49:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010d360 con 0x7f5c10103380 2024-09-10T22:49:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.076+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010d660 con 0x7f5c10101870 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0effd640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c101029c0 0x7f5c1019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/957272700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10103380 msgr2=0x7f5c101a05a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10103380 0x7f5c101a05a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c101029c0 msgr2=0x7f5c1019f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c101029c0 0x7f5c1019f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101a0e60 con 0x7f5c10101870 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.077+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000d770 tx=0x7f5c0000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.078+0000 7f5c0cff9640 1 -- 172.21.15.96:0/957272700 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00017070 con 0x7f5c10101870 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.078+0000 7f5c0cff9640 1 -- 172.21.15.96:0/957272700 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c000054d0 con 0x7f5c10101870 2024-09-10T22:49:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.078+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101a1150 con 0x7f5c10101870 2024-09-10T22:49:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.078+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a7820 con 0x7f5c10101870 2024-09-10T22:49:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.078+0000 7f5c0cff9640 1 -- 172.21.15.96:0/957272700 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00004d00 con 0x7f5c10101870 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.081+0000 7f5c18ac7740 1 Processor -- start 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.081+0000 7f5c0cff9640 1 -- 172.21.15.96:0/957272700 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5c00010460 con 0x7f5c10101870 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.081+0000 7f5c18ac7740 1 -- start start 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f5652b6f0 con 0x558f565ca870 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f5651ab20 con 0x558f565cd1a0 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558f5651e930 con 0x558f565cfbe0 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57086/0 (socket says 172.21.15.96:57086) 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c0ffff640 1 -- 172.21.15.96:0/2432326721 learned_addr learned my addr 172.21.15.96:0/2432326721 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.082+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2432326721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 msgr2=0x558f565d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2432326721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 msgr2=0x558f565ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2432326721 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f564e0530 con 0x558f565cd1a0 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0cff9640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 0x7f5bf80b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565cf590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5c000e3080 tx=0x7f5c000ebac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2016f17705ee7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 0x7f5bf80b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5c0cff9640 1 -- 172.21.15.96:0/957272700 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c000e55e0 con 0x7f5c10101870 2024-09-10T22:49:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5bf5ffb640 1 -- 172.21.15.96:0/2432326721 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000af020 con 0x558f565cd1a0 2024-09-10T22:49:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5bf5ffb640 1 -- 172.21.15.96:0/2432326721 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c000ab8b0 con 0x558f565cd1a0 2024-09-10T22:49:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.083+0000 7f5bf5ffb640 1 -- 172.21.15.96:0/2432326721 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000a7c40 con 0x558f565cd1a0 2024-09-10T22:49:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2432326721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 msgr2=0x558f565cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 0x7f5bf80b7970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006010 tx=0x7f5c080314d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565cf590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5c000e3080 tx=0x7f5c000ebac0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2432326721 shutdown_connections 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565cf590 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2432326721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2432326721 >> 172.21.15.96:0/2432326721 conn(0x558f565b49b0 msgr2=0x558f565b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2432326721 shutdown_connections 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.084+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2432326721 wait complete. 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 -- start start 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 Processor -- start 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57094/0 (socket says 172.21.15.96:57094) 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2466901385 learned_addr learned my addr 172.21.15.96:0/2466901385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f56452ab0 con 0x558f565ca870 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f564c0730 con 0x558f565cd1a0 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558f565d2bc0 con 0x558f565cfbe0 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.085+0000 7f5c18ac7740 10 client.0 _ll_register_callbacks cb 0x558f565146b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2466901385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 msgr2=0x558f565d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 msgr2=0x558f565d4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565d4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5627c790 con 0x558f565ca870 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c000ab8b0 con 0x558f565ca870 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5c000e3050 tx=0x7f5c000e8e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fe2d6f3abaab3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5beeffd640 10 client.15813 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5beeffd640 1 -- 172.21.15.96:0/2466901385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00020050 con 0x558f565ca870 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.086+0000 7f5beeffd640 1 -- 172.21.15.96:0/2466901385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c00009c30 con 0x558f565ca870 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.087+0000 7f5beeffd640 1 -- 172.21.15.96:0/2466901385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c000af070 con 0x558f565ca870 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.087+0000 7f5c18ac7740 10 client.15813 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.087+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558f562f5620 con 0x558f565ca870 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.088+0000 7f5beeffd640 1 -- 172.21.15.96:0/2466901385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5c000ac000 con 0x558f565ca870 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.088+0000 7f5c18ac7740 10 client.15813 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.088+0000 7f5c18ac7740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.088+0000 7f5c18ac7740 20 client.15813 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f565df220 0x558f565e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5c18ac7740 4 client.15813 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f5652d640 con 0x558f565df220 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f565df220 0x558f565e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5beeffd640 10 client.15813 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.089+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f565df220 0x558f565e1610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a40dd07a7dd1406 server_cookie=6761342c9fce100 in_seq=0 out_seq=0 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5beeffd640 1 -- 172.21.15.96:0/2466901385 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f5652d640 con 0x558f565df220 2024-09-10T22:49:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5beeffd640 10 client.15813 handle_command_reply: tid=0 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 client.15813 shutdown 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 msgr2=0x558f565d3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5c000e3050 tx=0x7f5c000e8e00 comp rx=0 tx=0).stop 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 shutdown_connections 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f565df220 0x558f565e1610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5beeffd640 0 client.15813 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cd1a0 0x558f565d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cfbe0 0x558f565d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2466901385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565ca870 0x558f565d3e80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 >> 172.21.15.96:0/2466901385 conn(0x558f565b49b0 msgr2=0x558f565b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 shutdown_connections 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2466901385 wait complete. 2024-09-10T22:49:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.090+0000 7f5c18ac7740 20 client.15813 trim_cache size 0 max 16384 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.112+0000 7f5c18ac7740 1 Processor -- start 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 -- start start 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cd1a0 0x558f565b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 0x558f565b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:53558/0 (socket says 172.21.15.96:53558) 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2135166671 learned_addr learned my addr 172.21.15.96:0/2135166671 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 0x558f565b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c0effd640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cd1a0 0x558f565b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00e3d0 con 0x558f565cfbe0 2024-09-10T22:49:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00ffd0 con 0x558f565ca870 2024-09-10T22:49:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.113+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00eaa0 con 0x558f565cd1a0 2024-09-10T22:49:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.114+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2135166671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cd1a0 msgr2=0x558f565b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.114+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cd1a0 0x558f565b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.114+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2135166671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 msgr2=0x558f565b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.114+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 0x558f565b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.114+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/2135166671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f565ccfd0 con 0x558f565ca870 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.115+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 0x558f565b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.115+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c000a8370 tx=0x7f5c00012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccaccea217b40343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.115+0000 7f5bf57fa640 1 -- 172.21.15.96:0/2135166671 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00020040 con 0x558f565ca870 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.115+0000 7f5bf57fa640 1 -- 172.21.15.96:0/2135166671 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c000ab8b0 con 0x558f565ca870 2024-09-10T22:49:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.115+0000 7f5bf57fa640 1 -- 172.21.15.96:0/2135166671 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000d1030 con 0x558f565ca870 2024-09-10T22:49:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 msgr2=0x558f565b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c000a8370 tx=0x7f5c00012000 comp rx=0 tx=0).stop 2024-09-10T22:49:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 shutdown_connections 2024-09-10T22:49:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cfbe0 0x558f565b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565cd1a0 0x558f565b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/2135166671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565ca870 0x558f565b8c60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 >> 172.21.15.96:0/2135166671 conn(0x558f565b49b0 msgr2=0x558f565cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 shutdown_connections 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.116+0000 7f5c18ac7740 1 -- 172.21.15.96:0/2135166671 wait complete. 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 -- start start 2024-09-10T22:49:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 Processor -- start 2024-09-10T22:49:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 0x558f565dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 0x558f565da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cfbe0 0x558f565dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00f840 con 0x558f565cd1a0 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00f170 con 0x558f565cfbe0 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc00c3b0 con 0x558f565ca870 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c18ac7740 10 client.0 _ll_register_callbacks cb 0x7f5c08009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c0effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 0x558f565da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c0effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 0x558f565da7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57114/0 (socket says 172.21.15.96:57114) 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c0effd640 1 -- 172.21.15.96:0/1723520993 learned_addr learned my addr 172.21.15.96:0/1723520993 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.117+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 0x558f565dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0ffff640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cfbe0 0x558f565dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cfbe0 msgr2=0x558f565dad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cfbe0 0x558f565dad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/1723520993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 msgr2=0x558f565da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 0x558f565da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 -- 172.21.15.96:0/1723520993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f565db260 con 0x558f565ca870 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5c0f7fe640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 0x558f565dbae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5c000abaa0 tx=0x7f5c00009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b66b99b3cc4916 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.118+0000 7f5beeffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000d1060 con 0x558f565ca870 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c000e5110 con 0x558f565ca870 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c000ab8b0 con 0x558f565ca870 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c00020040 con 0x558f565ca870 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5c18ac7740 10 client.25405 fetch_fsmap learned FSMap version 30 2024-09-10T22:49:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.119+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f565db260 con 0x558f565ca870 2024-09-10T22:49:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.120+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c000b0880 con 0x558f565ca870 2024-09-10T22:49:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.120+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5c000a76f0 con 0x558f565ca870 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 10 client.25405 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 20 client.25405 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f5664c4b0 0x558f5664e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 4 client.25405 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:49:21.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.121+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5c000ab8b0 con 0x558f5664c4b0 2024-09-10T22:49:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.125+0000 7f5c0effd640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f5664c4b0 0x558f5664e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.130+0000 7f5c0effd640 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f5664c4b0 0x558f5664e8d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b9031a721328d3 server_cookie=b8ce2bc0ebff338b in_seq=0 out_seq=0 2024-09-10T22:49:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.131+0000 7f5beeffd640 10 client.25405 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.151+0000 7f5beeffd640 1 -- 172.21.15.96:0/1723520993 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5c000ab8b0 con 0x558f5664c4b0 2024-09-10T22:49:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.151+0000 7f5beeffd640 10 client.25405 handle_command_reply: tid=0 2024-09-10T22:49:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.151+0000 7f5c18ac7740 1 client.25405 shutdown 2024-09-10T22:49:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 msgr2=0x558f565dbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 0x558f565dbae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5c000abaa0 tx=0x7f5c00009e60 comp rx=0 tx=0).stop 2024-09-10T22:49:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 shutdown_connections 2024-09-10T22:49:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558f5664c4b0 0x558f5664e8d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f565cfbe0 0x558f565dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5beeffd640 0 client.25405 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.152+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f565cd1a0 0x558f565da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c18ac7740 1 --2- 172.21.15.96:0/1723520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f565ca870 0x558f565dbae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 >> 172.21.15.96:0/1723520993 conn(0x558f565b49b0 msgr2=0x558f565b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 shutdown_connections 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c18ac7740 1 -- 172.21.15.96:0/1723520993 wait complete. 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c18ac7740 20 client.25405 trim_cache size 0 max 16384 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:49:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:49:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:49:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:49:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 msgr2=0x7f5bf80b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 0x7f5bf80b7970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006010 tx=0x7f5c080314d0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 msgr2=0x7f5c1019f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.153+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000d770 tx=0x7f5c0000dc30 comp rx=0 tx=0).stop 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 shutdown_connections 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c10103380 0x7f5c101a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bf80b54b0 0x7f5bf80b7970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c101029c0 0x7f5c1019f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 --2- 172.21.15.96:0/957272700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c10101870 0x7f5c1019f320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 >> 172.21.15.96:0/957272700 conn(0x7f5c100fd130 msgr2=0x7f5c100ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 shutdown_connections 2024-09-10T22:49:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:21.154+0000 7f5c15ced640 1 -- 172.21.15.96:0/957272700 wait complete. 2024-09-10T22:49:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:21 smithi096 ceph-mon[30869]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 757 op/s 2024-09-10T22:49:21.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:21 smithi121 ceph-mon[35119]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 757 op/s 2024-09-10T22:49:21.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:21 smithi117 ceph-mon[35081]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 757 op/s 2024-09-10T22:49:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:23 smithi096 ceph-mon[30869]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T22:49:23.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:23 smithi121 ceph-mon[35119]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T22:49:23.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:23 smithi117 ceph-mon[35081]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T22:49:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:25 smithi096 ceph-mon[30869]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-09-10T22:49:25.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:25 smithi121 ceph-mon[35119]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-09-10T22:49:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:25 smithi117 ceph-mon[35081]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 777 op/s 2024-09-10T22:49:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 110.0 s, 3174.6 tps, lat 31.603 ms stddev 34.473 2024-09-10T22:49:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:26 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:26.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:26 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:26.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:26 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:27.414 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:27 smithi096 ceph-mon[30869]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:49:27.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:27 smithi121 ceph-mon[35119]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:49:27.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:27 smithi117 ceph-mon[35081]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:49:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:29 smithi096 ceph-mon[30869]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-09-10T22:49:29.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:29 smithi121 ceph-mon[35119]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-09-10T22:49:29.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:29 smithi117 ceph-mon[35081]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 645 op/s 2024-09-10T22:49:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 115.0 s, 3590.8 tps, lat 27.815 ms stddev 27.895 2024-09-10T22:49:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:31 smithi096 ceph-mon[30869]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T22:49:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:31 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:31 smithi121 ceph-mon[35119]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T22:49:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:31 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:31 smithi117 ceph-mon[35081]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T22:49:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:31 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:49:32.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:32.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:32.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:49:32.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:32.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:32.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:49:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:49:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:49:32.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:33 smithi096 ceph-mon[30869]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-09-10T22:49:33.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:33 smithi121 ceph-mon[35119]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-09-10T22:49:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:33 smithi117 ceph-mon[35081]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 MiB/s wr, 813 op/s 2024-09-10T22:49:34.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:49:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 -- 172.21.15.96:0/3084137238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444101860 msgr2=0x7f5444101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 --2- 172.21.15.96:0/3084137238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444101860 0x7f5444101c60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5438009930 tx=0x7f543802a420 comp rx=0 tx=0).stop 2024-09-10T22:49:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 -- 172.21.15.96:0/3084137238 shutdown_connections 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 --2- 172.21.15.96:0/3084137238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 --2- 172.21.15.96:0/3084137238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444103370 0x7f5444109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 --2- 172.21.15.96:0/3084137238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444101860 0x7f5444101c60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 -- 172.21.15.96:0/3084137238 >> 172.21.15.96:0/3084137238 conn(0x7f54440fd120 msgr2=0x7f54440ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 -- 172.21.15.96:0/3084137238 shutdown_connections 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.279+0000 7f544b12c640 1 -- 172.21.15.96:0/3084137238 wait complete. 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 Processor -- start 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 -- start start 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 0x7f5444198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f544410d4d0 con 0x7f5444103370 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f544410d350 con 0x7f54441029b0 2024-09-10T22:49:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f544b12c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f544410d650 con 0x7f5444101860 2024-09-10T22:49:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f54496c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f54496c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39360/0 (socket says 172.21.15.96:39360) 2024-09-10T22:49:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.280+0000 7f54496c9640 1 -- 172.21.15.96:0/1791463866 learned_addr learned my addr 172.21.15.96:0/1791463866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5443fff640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 0x7f5444198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 -- 172.21.15.96:0/1791463866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 msgr2=0x7f5444198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 0x7f5444198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 -- 172.21.15.96:0/1791463866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 msgr2=0x7f5444199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 -- 172.21.15.96:0/1791463866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544419a8c0 con 0x7f5444103370 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 0x7f5444198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5443fff640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f54496c9640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f543c00eaa0 tx=0x7f543c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5441ffb640 1 -- 172.21.15.96:0/1791463866 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00cdb0 con 0x7f5444103370 2024-09-10T22:49:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5441ffb640 1 -- 172.21.15.96:0/1791463866 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543c005020 con 0x7f5444103370 2024-09-10T22:49:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f5441ffb640 1 -- 172.21.15.96:0/1791463866 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c0116d0 con 0x7f5444103370 2024-09-10T22:49:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.281+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544419abb0 con 0x7f5444103370 2024-09-10T22:49:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.282+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441a1280 con 0x7f5444103370 2024-09-10T22:49:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.285+0000 7f5441ffb640 1 -- 172.21.15.96:0/1791463866 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f543c00f040 con 0x7f5444103370 2024-09-10T22:49:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5441ffb640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 0x7f54340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 Processor -- start 2024-09-10T22:49:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 -- start start 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5441ffb640 1 -- 172.21.15.96:0/1791463866 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f543c0e5a50 con 0x7f5444103370 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 0x7f54340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 0x5584e7364c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e7366070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584e72d46f0 con 0x5584e73651b0 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5584e72c3b20 con 0x5584e7365bc0 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f544dfcf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5584e72c7930 con 0x5584e7362860 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5443fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5443fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39374/0 (socket says 172.21.15.96:39374) 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5443fff640 1 -- 172.21.15.96:0/1029672105 learned_addr learned my addr 172.21.15.96:0/1029672105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f54496c9640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e7366070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.287+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 0x5584e7364c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 -- 172.21.15.96:0/1029672105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 msgr2=0x5584e7364c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 0x5584e7364c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 -- 172.21.15.96:0/1029672105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 msgr2=0x5584e7366070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e7366070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 -- 172.21.15.96:0/1029672105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e7289530 con 0x5584e73651b0 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 0x5584e7364c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5443fff640 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5430009c30 tx=0x7f5430010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9f899d958b67b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f542effd640 1 -- 172.21.15.96:0/1029672105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430004030 con 0x5584e73651b0 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 0x7f54340b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f54380096c0 tx=0x7f5438009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f542effd640 1 -- 172.21.15.96:0/1029672105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54300041d0 con 0x5584e73651b0 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.288+0000 7f542effd640 1 -- 172.21.15.96:0/1029672105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54300126b0 con 0x5584e73651b0 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- 172.21.15.96:0/1029672105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 msgr2=0x5584e7365630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5430009c30 tx=0x7f5430010740 comp rx=0 tx=0).stop 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- 172.21.15.96:0/1029672105 shutdown_connections 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e7366070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e7365630 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1029672105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7362860 0x5584e7364c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- 172.21.15.96:0/1029672105 >> 172.21.15.96:0/1029672105 conn(0x5584e735d9b0 msgr2=0x5584e735e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- 172.21.15.96:0/1029672105 shutdown_connections 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- 172.21.15.96:0/1029672105 wait complete. 2024-09-10T22:49:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 -- start start 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.290+0000 7f544dfcf740 1 Processor -- start 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e736c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e736d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584e71fbab0 con 0x5584e73651b0 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5584e7269730 con 0x5584e7362860 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5584e7366b90 con 0x5584e7365bc0 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f5443fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e736c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f544dfcf740 10 client.0 _ll_register_callbacks cb 0x5584e72bd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35580/0 (socket says 172.21.15.96:35580) 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f5448ec8640 1 -- 172.21.15.96:0/306129035 learned_addr learned my addr 172.21.15.96:0/306129035 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f5443fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e736c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39384/0 (socket says 172.21.15.96:39384) 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.291+0000 7f54496c9640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e736d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 -- 172.21.15.96:0/306129035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 msgr2=0x5584e736d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e736d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 -- 172.21.15.96:0/306129035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 msgr2=0x5584e736c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e736c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 -- 172.21.15.96:0/306129035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e7025790 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.292+0000 7f5448ec8640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5438033000 tx=0x7f5438000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7866126208eb3a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.293+0000 7f5423fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.293+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543804c020 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54300041d0 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f54380046c0 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5438035030 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f544dfcf740 10 client.25400 fetch_fsmap learned FSMap version 31 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584e709e620 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438041030 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.294+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f54380038f0 con 0x5584e7362860 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 10 client.25400 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 10 client.25400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 20 client.25400 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e738b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 4 client.25400 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f54300041d0 con 0x5584e7388bb0 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f5443fff640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e738b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f5423fff640 10 client.25400 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.295+0000 7f5443fff640 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e738b010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dda73f18a7acaa4 server_cookie=e1a2471bc34f5a07 in_seq=0 out_seq=0 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f5423fff640 1 -- 172.21.15.96:0/306129035 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f54300041d0 con 0x5584e7388bb0 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f5423fff640 10 client.25400 handle_command_reply: tid=0 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 client.25400 shutdown 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 msgr2=0x5584e736bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5438033000 tx=0x7f5438000bc0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 shutdown_connections 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e738b010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e736d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e73651b0 0x5584e736c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f5423fff640 0 client.25400 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 --2- 172.21.15.96:0/306129035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7362860 0x5584e736bf00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 >> 172.21.15.96:0/306129035 conn(0x5584e735d9b0 msgr2=0x5584e735cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 shutdown_connections 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 1 -- 172.21.15.96:0/306129035 wait complete. 2024-09-10T22:49:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.296+0000 7f544dfcf740 20 client.25400 trim_cache size 0 max 16384 2024-09-10T22:49:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.321+0000 7f544dfcf740 1 Processor -- start 2024-09-10T22:49:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.321+0000 7f544dfcf740 1 -- start start 2024-09-10T22:49:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e73651b0 0x5584e7358bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39400/0 (socket says 172.21.15.96:39400) 2024-09-10T22:49:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f5448ec8640 1 -- 172.21.15.96:0/1439461615 learned_addr learned my addr 172.21.15.96:0/1439461615 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e736baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.322+0000 7f5443fff640 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e73651b0 0x5584e7358bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a7dc0 con 0x5584e7362860 2024-09-10T22:49:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a9930 con 0x5584e7365bc0 2024-09-10T22:49:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a8490 con 0x5584e73651b0 2024-09-10T22:49:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 -- 172.21.15.96:0/1439461615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e73651b0 msgr2=0x5584e7358bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e73651b0 0x5584e7358bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 -- 172.21.15.96:0/1439461615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 msgr2=0x5584e736baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e736baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 -- 172.21.15.96:0/1439461615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e736a150 con 0x5584e7362860 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.323+0000 7f5448ec8640 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f543802a940 tx=0x7f543803a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421fba1ad968a595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f542e7fc640 1 -- 172.21.15.96:0/1439461615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543804c040 con 0x5584e7362860 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f542e7fc640 1 -- 172.21.15.96:0/1439461615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438041040 con 0x5584e7362860 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f542e7fc640 1 -- 172.21.15.96:0/1439461615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543803abf0 con 0x5584e7362860 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 msgr2=0x5584e7364c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f543802a940 tx=0x7f543803a680 comp rx=0 tx=0).stop 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 shutdown_connections 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e7365bc0 0x5584e736baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e73651b0 0x5584e7358bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 --2- 172.21.15.96:0/1439461615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e7364c50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.324+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 >> 172.21.15.96:0/1439461615 conn(0x5584e735d9b0 msgr2=0x5584e736b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.325+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 shutdown_connections 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.325+0000 7f544dfcf740 1 -- 172.21.15.96:0/1439461615 wait complete. 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.325+0000 7f544dfcf740 1 -- start start 2024-09-10T22:49:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.325+0000 7f544dfcf740 1 Processor -- start 2024-09-10T22:49:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.325+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e73651b0 0x5584e739d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e7398c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a9230 con 0x5584e7362860 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39404/0 (socket says 172.21.15.96:39404) 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a8b60 con 0x5584e73651b0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54441a5da0 con 0x5584e7365bc0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 -- 172.21.15.96:0/3508671647 learned_addr learned my addr 172.21.15.96:0/3508671647 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 10 client.0 _ll_register_callbacks cb 0x7f5430005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5443fff640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e73651b0 0x5584e739d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f54496c9640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e7398c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 -- 172.21.15.96:0/3508671647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 msgr2=0x5584e7398c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e7398c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 -- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e73651b0 msgr2=0x5584e739d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e73651b0 0x5584e739d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 -- 172.21.15.96:0/3508671647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e73994b0 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5448ec8640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5438003b90 tx=0x7f5438003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766b26790faacbb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5423fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543806a060 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f543803ab70 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5438041040 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f543804c040 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 10 client.15837 fetch_fsmap learned FSMap version 31 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.326+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584e73f5a70 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438041040 con 0x5584e7362860 2024-09-10T22:49:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f54380046c0 con 0x5584e7362860 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f544dfcf740 10 client.15837 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f544dfcf740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f544dfcf740 20 client.15837 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.327+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e73f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.328+0000 7f544dfcf740 4 client.15837 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.328+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584e72d6640 con 0x5584e7388bb0 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.328+0000 7f5443fff640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e73f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.329+0000 7f5443fff640 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e73f8170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1798c8fe012c2cd server_cookie=6d6cf301e1d9fd11 in_seq=0 out_seq=0 2024-09-10T22:49:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.329+0000 7f5423fff640 10 client.15837 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.346+0000 7f5423fff640 1 -- 172.21.15.96:0/3508671647 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584e72d6640 con 0x5584e7388bb0 2024-09-10T22:49:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.346+0000 7f5423fff640 10 client.15837 handle_command_reply: tid=0 2024-09-10T22:49:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.346+0000 7f544dfcf740 1 client.15837 shutdown 2024-09-10T22:49:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 msgr2=0x5584e739cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5438003b90 tx=0x7f5438003bc0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 shutdown_connections 2024-09-10T22:49:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5584e7388bb0 0x5584e73f8170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584e73651b0 0x5584e739d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584e7365bc0 0x5584e7398c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 --2- 172.21.15.96:0/3508671647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584e7362860 0x5584e739cbc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.347+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 >> 172.21.15.96:0/3508671647 conn(0x5584e735d9b0 msgr2=0x5584e736baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.348+0000 7f5423fff640 0 client.15837 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.348+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 shutdown_connections 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.348+0000 7f544dfcf740 1 -- 172.21.15.96:0/3508671647 wait complete. 2024-09-10T22:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.348+0000 7f544dfcf740 20 client.15837 trim_cache size 0 max 16384 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.349+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 msgr2=0x7f54340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.349+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 0x7f54340b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f54380096c0 tx=0x7f5438009560 comp rx=0 tx=0).stop 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.349+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 msgr2=0x7f544419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.349+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f543c00eaa0 tx=0x7f543c00ef60 comp rx=0 tx=0).stop 2024-09-10T22:49:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 shutdown_connections 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54441029b0 0x7f5444199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54340b52e0 0x7f54340b77a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5444103370 0x7f544419a000 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 --2- 172.21.15.96:0/1791463866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5444101860 0x7f5444198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.351+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 >> 172.21.15.96:0/1791463866 conn(0x7f54440fd120 msgr2=0x7f54440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.352+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 shutdown_connections 2024-09-10T22:49:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:34.352+0000 7f544b12c640 1 -- 172.21.15.96:0/1791463866 wait complete. 2024-09-10T22:49:34.654 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:35.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:35 smithi096 ceph-mon[30869]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-09-10T22:49:35.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:35 smithi121 ceph-mon[35119]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-09-10T22:49:35.525 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:35 smithi117 ceph-mon[35081]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 713 op/s 2024-09-10T22:49:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 120.0 s, 3183.6 tps, lat 31.342 ms stddev 33.798 2024-09-10T22:49:36.299 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:36 smithi096 ceph-mon[30869]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 -- 172.21.15.96:0/2867529370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004106e60 msgr2=0x7fb0041072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 --2- 172.21.15.96:0/2867529370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004106e60 0x7fb0041072c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb000009860 tx=0x7fb00002a340 comp rx=0 tx=0).stop 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 -- 172.21.15.96:0/2867529370 shutdown_connections 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 --2- 172.21.15.96:0/2867529370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004106e60 0x7fb0041072c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 --2- 172.21.15.96:0/2867529370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004107800 0x7fb00410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 --2- 172.21.15.96:0/2867529370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004101450 0x7fb004101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.312+0000 7fb00b539640 1 -- 172.21.15.96:0/2867529370 >> 172.21.15.96:0/2867529370 conn(0x7fb004078140 msgr2=0x7fb0040ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.313+0000 7fb00b539640 1 -- 172.21.15.96:0/2867529370 shutdown_connections 2024-09-10T22:49:36.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.313+0000 7fb00b539640 1 -- 172.21.15.96:0/2867529370 wait complete. 2024-09-10T22:49:36.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 Processor -- start 2024-09-10T22:49:36.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 -- start start 2024-09-10T22:49:36.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 0x7fb004116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 0x7fb004117220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004107800 0x7fb0041110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb00410f8f0 con 0x7fb004101450 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb00410f770 con 0x7fb004107800 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb00b539640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb00410fa70 con 0x7fb004106e60 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 0x7fb004116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 0x7fb004116ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58602/0 (socket says 172.21.15.96:58602) 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb0092d5640 1 -- 172.21.15.96:0/4174914665 learned_addr learned my addr 172.21.15.96:0/4174914665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb008ad4640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 0x7fb004117220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.314+0000 7fb009ad6640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004107800 0x7fb0041110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 -- 172.21.15.96:0/4174914665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004107800 msgr2=0x7fb0041110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004107800 0x7fb0041110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 -- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 msgr2=0x7fb004116ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 0x7fb004116ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 -- 172.21.15.96:0/4174914665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0041119b0 con 0x7fb004106e60 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7fb008ad4640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 0x7fb004117220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb000035040 tx=0x7fb000032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.315+0000 7faffa7fc640 1 -- 172.21.15.96:0/4174914665 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000037040 con 0x7fb004106e60 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.316+0000 7faffa7fc640 1 -- 172.21.15.96:0/4174914665 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000030c10 con 0x7fb004106e60 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.316+0000 7faffa7fc640 1 -- 172.21.15.96:0/4174914665 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000035f0 con 0x7fb004106e60 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.316+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb004111c40 con 0x7fb004106e60 2024-09-10T22:49:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.316+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0041b4a30 con 0x7fb004106e60 2024-09-10T22:49:36.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.320+0000 7faffa7fc640 1 -- 172.21.15.96:0/4174914665 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb000004050 con 0x7fb004106e60 2024-09-10T22:49:36.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 Processor -- start 2024-09-10T22:49:36.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 -- start start 2024-09-10T22:49:36.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7faffa7fc640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 0x7faff00b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf9c0 0x5560093db410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556009345e50 con 0x5560093cf0a0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556009335280 con 0x5560093cf9c0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb00d90e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556009339090 con 0x5560093db950 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb009ad6640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 0x7faff00b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7faffa7fc640 1 -- 172.21.15.96:0/4174914665 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb000104de0 con 0x7fb004106e60 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58616/0 (socket says 172.21.15.96:58616) 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb0092d5640 1 -- 172.21.15.96:0/3975234867 learned_addr learned my addr 172.21.15.96:0/3975234867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.322+0000 7fb009ad6640 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 -- 172.21.15.96:0/3975234867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 msgr2=0x5560093ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb008ad4640 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf9c0 0x5560093db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093ddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 -- 172.21.15.96:0/3975234867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf9c0 msgr2=0x5560093db410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:36.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf9c0 0x5560093db410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 -- 172.21.15.96:0/3975234867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556009093650 con 0x5560093cf0a0 2024-09-10T22:49:36.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb009ad6640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 0x7faff00b7890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faff4005fd0 tx=0x7faff4011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.323+0000 7fb0092d5640 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faffc0115f0 tx=0x7faffc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2cd40d3379ff19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.324+0000 7fafef7fe640 1 -- 172.21.15.96:0/3975234867 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc004b20 con 0x5560093cf0a0 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.324+0000 7fafef7fe640 1 -- 172.21.15.96:0/3975234867 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faffc004cc0 con 0x5560093cf0a0 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fafef7fe640 1 -- 172.21.15.96:0/3975234867 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc0053b0 con 0x5560093cf0a0 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 -- 172.21.15.96:0/3975234867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 msgr2=0x5560093cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faffc0115f0 tx=0x7faffc011ab0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 -- 172.21.15.96:0/3975234867 shutdown_connections 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf9c0 0x5560093db410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 --2- 172.21.15.96:0/3975234867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cf480 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.325+0000 7fb00d90e740 1 -- 172.21.15.96:0/3975234867 >> 172.21.15.96:0/3975234867 conn(0x5560093cced0 msgr2=0x5560093ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.326+0000 7fb00d90e740 1 -- 172.21.15.96:0/3975234867 shutdown_connections 2024-09-10T22:49:36.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.326+0000 7fb00d90e740 1 -- 172.21.15.96:0/3975234867 wait complete. 2024-09-10T22:49:36.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.326+0000 7fb00d90e740 1 -- start start 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.326+0000 7fb00d90e740 1 Processor -- start 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556009196700 con 0x5560093cf0a0 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5560092d0110 con 0x5560093db950 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560093de620 con 0x5560093cf9c0 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb00d90e740 10 client.0 _ll_register_callbacks cb 0x5560092e9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58630/0 (socket says 172.21.15.96:58630) 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb0092d5640 1 -- 172.21.15.96:0/983641508 learned_addr learned my addr 172.21.15.96:0/983641508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.327+0000 7fb008ad4640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 -- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 msgr2=0x5560093d8a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 -- 172.21.15.96:0/983641508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 msgr2=0x5560093db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 -- 172.21.15.96:0/983641508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560092fdc40 con 0x5560093cf9c0 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb0092d5640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb008ad4640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093d8530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb0001028e0 tx=0x7fb00002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca6cbd662867104 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fafecff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000cb0b0 con 0x5560093cf9c0 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb0000ca800 con 0x5560093cf9c0 2024-09-10T22:49:36.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.328+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc004cc0 con 0x5560093cf9c0 2024-09-10T22:49:36.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.329+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb000030dc0 con 0x5560093cf9c0 2024-09-10T22:49:36.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.329+0000 7fb00d90e740 10 client.25429 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:36.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.329+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560093e9e70 con 0x5560093cf9c0 2024-09-10T22:49:36.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.329+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faffc004cc0 con 0x5560093cf9c0 2024-09-10T22:49:36.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.329+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb0000ca270 con 0x5560093cf9c0 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 10 client.25429 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 20 client.25429 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5560093eb280 0x5560093ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 4 client.25429 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556009347da0 con 0x5560093eb280 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb0092d5640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5560093eb280 0x5560093ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fb0092d5640 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5560093eb280 0x5560093ed6b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2c5643caf67ba7 server_cookie=ef87409473287042 in_seq=0 out_seq=0 2024-09-10T22:49:36.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.330+0000 7fafecff9640 10 client.25429 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fafecff9640 1 -- 172.21.15.96:0/983641508 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556009347da0 con 0x5560093eb280 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fafecff9640 10 client.25429 handle_command_reply: tid=0 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 client.25429 shutdown 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 msgr2=0x5560093d8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093d8530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb0001028e0 tx=0x7fb00002a340 comp rx=0 tx=0).stop 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 shutdown_connections 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5560093eb280 0x5560093ed6b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093d8530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 --2- 172.21.15.96:0/983641508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fafecff9640 0 client.25429 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.332+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 >> 172.21.15.96:0/983641508 conn(0x5560093cced0 msgr2=0x5560093d2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.333+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 shutdown_connections 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.333+0000 7fb00d90e740 1 -- 172.21.15.96:0/983641508 wait complete. 2024-09-10T22:49:36.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.333+0000 7fb00d90e740 20 client.25429 trim_cache size 0 max 16384 2024-09-10T22:49:36.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.358+0000 7fb00d90e740 1 Processor -- start 2024-09-10T22:49:36.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.358+0000 7fb00d90e740 1 -- start start 2024-09-10T22:49:36.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041bd080 con 0x5560093cf0a0 2024-09-10T22:49:36.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041bebc0 con 0x5560093db950 2024-09-10T22:49:36.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb00d90e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041bd750 con 0x5560093cf9c0 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb009ad6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb009ad6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40432/0 (socket says 172.21.15.96:40432) 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.359+0000 7fb009ad6640 1 -- 172.21.15.96:0/140741172 learned_addr learned my addr 172.21.15.96:0/140741172 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 -- 172.21.15.96:0/140741172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 msgr2=0x5560093cdf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093cdf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 -- 172.21.15.96:0/140741172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 msgr2=0x5560093cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb0092d5640 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 -- 172.21.15.96:0/140741172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560093de4b0 con 0x5560093db950 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb0092d5640 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.360+0000 7fb009ad6640 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faff40098c0 tx=0x7faff401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74d75b27fa115e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.361+0000 7fafeeffd640 1 -- 172.21.15.96:0/140741172 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff401e720 con 0x5560093db950 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.361+0000 7fafeeffd640 1 -- 172.21.15.96:0/140741172 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff401ed70 con 0x5560093db950 2024-09-10T22:49:36.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.361+0000 7fafeeffd640 1 -- 172.21.15.96:0/140741172 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4026da0 con 0x5560093db950 2024-09-10T22:49:36.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.361+0000 7fb00d90e740 1 -- 172.21.15.96:0/140741172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 msgr2=0x5560093d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.361+0000 7fb00d90e740 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faff40098c0 tx=0x7faff401c000 comp rx=0 tx=0).stop 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 -- 172.21.15.96:0/140741172 shutdown_connections 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093db950 0x5560093d5740 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093cf9c0 0x5560093cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 --2- 172.21.15.96:0/140741172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf0a0 0x5560093cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 -- 172.21.15.96:0/140741172 >> 172.21.15.96:0/140741172 conn(0x5560093cced0 msgr2=0x5560093d6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 -- 172.21.15.96:0/140741172 shutdown_connections 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 -- 172.21.15.96:0/140741172 wait complete. 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 -- start start 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.362+0000 7fb00d90e740 1 Processor -- start 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf9c0 0x5560093e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041be4f0 con 0x5560093cf9c0 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041bde20 con 0x5560093cf0a0 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb0041bb080 con 0x5560093db950 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb00d90e740 10 client.0 _ll_register_callbacks cb 0x7fb0000c7bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb0092d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40448/0 (socket says 172.21.15.96:40448) 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb0092d5640 1 -- 172.21.15.96:0/1701445714 learned_addr learned my addr 172.21.15.96:0/1701445714 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:36.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.363+0000 7fb009ad6640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.364+0000 7fb0092d5640 1 -- 172.21.15.96:0/1701445714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 msgr2=0x5560093e0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:36.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.364+0000 7fb0092d5640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093e0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.364+0000 7fb0092d5640 1 -- 172.21.15.96:0/1701445714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf9c0 msgr2=0x5560093e0690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.364+0000 7fb0092d5640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf9c0 0x5560093e0690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.364+0000 7fb0092d5640 1 -- 172.21.15.96:0/1701445714 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560093e1480 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fb0092d5640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faffc0049d0 tx=0x7faffc0114f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46eab15b189a24a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fafecff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc031060 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff401ed70 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faffc016040 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faffc004cc0 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fb00d90e740 10 client.25415 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.365+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55600946a1d0 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff401ed70 con 0x5560093cf0a0 2024-09-10T22:49:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faffc036070 con 0x5560093cf0a0 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 10 client.25415 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 10 client.25415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 20 client.25415 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55600946a970 0x55600946cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 4 client.25415 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.366+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556009343850 con 0x55600946a970 2024-09-10T22:49:36.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fb008ad4640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55600946a970 0x55600946cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fb008ad4640 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55600946a970 0x55600946cd90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3834ea6953ff05 server_cookie=8c2061cc936a45e4 in_seq=0 out_seq=0 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fafecff9640 10 client.25415 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fafecff9640 1 -- 172.21.15.96:0/1701445714 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x556009343850 con 0x55600946a970 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fafecff9640 10 client.25415 handle_command_reply: tid=0 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fb00d90e740 1 client.25415 shutdown 2024-09-10T22:49:36.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 msgr2=0x5560093e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.368+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faffc0049d0 tx=0x7faffc0114f0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 shutdown_connections 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55600946a970 0x55600946cd90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560093db950 0x5560093e0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560093cf9c0 0x5560093e0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 --2- 172.21.15.96:0/1701445714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560093cf0a0 0x5560093e4160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 >> 172.21.15.96:0/1701445714 conn(0x5560093cced0 msgr2=0x5560093d9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fafecff9640 0 client.25415 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 shutdown_connections 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 1 -- 172.21.15.96:0/1701445714 wait complete. 2024-09-10T22:49:36.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.369+0000 7fb00d90e740 20 client.25415 trim_cache size 0 max 16384 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 msgr2=0x7faff00b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 0x7faff00b7890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faff4005fd0 tx=0x7faff4011810 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 msgr2=0x7fb004117220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 0x7fb004117220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb000035040 tx=0x7fb000032d50 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 shutdown_connections 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb004107800 0x7fb0041110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faff00b53d0 0x7faff00b7890 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb004106e60 0x7fb004117220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 --2- 172.21.15.96:0/4174914665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb004101450 0x7fb004116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 >> 172.21.15.96:0/4174914665 conn(0x7fb004078140 msgr2=0x7fb004103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 shutdown_connections 2024-09-10T22:49:36.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:36.370+0000 7fb00b539640 1 -- 172.21.15.96:0/4174914665 wait complete. 2024-09-10T22:49:36.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:36 smithi121 ceph-mon[35119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:36.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:36 smithi117 ceph-mon[35081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:49:37.422 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:49:37.422 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:49:37.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:37 smithi096 ceph-mon[30869]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T22:49:37.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:37 smithi121 ceph-mon[35119]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T22:49:37.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:37 smithi117 ceph-mon[35081]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T22:49:37.811 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:38.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 -- 172.21.15.96:0/1047787950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038109170 msgr2=0x7fc03810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 --2- 172.21.15.96:0/1047787950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038109170 0x7fc03810f970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc030009be0 tx=0x7fc03002a6e0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 -- 172.21.15.96:0/1047787950 shutdown_connections 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 --2- 172.21.15.96:0/1047787950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038109170 0x7fc03810f970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 --2- 172.21.15.96:0/1047787950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0381087f0 0x7fc038108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 --2- 172.21.15.96:0/1047787950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc038102de0 0x7fc0381031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 -- 172.21.15.96:0/1047787950 >> 172.21.15.96:0/1047787950 conn(0x7fc0380fe830 msgr2=0x7fc038100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 -- 172.21.15.96:0/1047787950 shutdown_connections 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.360+0000 7fc03ec58640 1 -- 172.21.15.96:0/1047787950 wait complete. 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 Processor -- start 2024-09-10T22:49:38.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 -- start start 2024-09-10T22:49:38.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0381087f0 0x7fc038199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc038109170 0x7fc03819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc038112e30 con 0x7fc038109170 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc038112cb0 con 0x7fc0381087f0 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.361+0000 7fc03ec58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc038112fb0 con 0x7fc038102de0 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47316/0 (socket says 172.21.15.96:47316) 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 -- 172.21.15.96:0/4285883754 learned_addr learned my addr 172.21.15.96:0/4285883754 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03d1f5640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc038109170 0x7fc03819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 -- 172.21.15.96:0/4285883754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0381087f0 msgr2=0x7fc038199780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0381087f0 0x7fc038199780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 -- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc038109170 msgr2=0x7fc03819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc038109170 0x7fc03819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.362+0000 7fc03c9f4640 1 -- 172.21.15.96:0/4285883754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03819f110 con 0x7fc038102de0 2024-09-10T22:49:38.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc03c9f4640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9c0 tx=0x7fc02c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00ccc0 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03819f400 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381a3c90 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc02c0054d0 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.363+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c010430 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.364+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c0052c0 con 0x7fc038102de0 2024-09-10T22:49:38.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.365+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc02c0105d0 con 0x7fc038102de0 2024-09-10T22:49:38.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.366+0000 7fc035ffb640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 0x7fc0280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:38.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.366+0000 7fc037fff640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 0x7fc0280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:38.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.366+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc02c0e5df0 con 0x7fc038102de0 2024-09-10T22:49:38.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.367+0000 7fc037fff640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 0x7fc0280b7820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc038100fd0 tx=0x7fc024005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:38.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.370+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc02c0acd50 con 0x7fc038102de0 2024-09-10T22:49:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.699+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc01c005880 con 0x7fc038102de0 2024-09-10T22:49:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.700+0000 7fc035ffb640 1 -- 172.21.15.96:0/4285883754 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fc02c0ac650 con 0x7fc038102de0 2024-09-10T22:49:38.702 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:49:38.702 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:49:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.705+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 msgr2=0x7fc0280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.705+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 0x7fc0280b7820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc038100fd0 tx=0x7fc024005f90 comp rx=0 tx=0).stop 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.705+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 msgr2=0x7fc038199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.705+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9c0 tx=0x7fc02c00ee80 comp rx=0 tx=0).stop 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 shutdown_connections 2024-09-10T22:49:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0381087f0 0x7fc038199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0280b5360 0x7fc0280b7820 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc038109170 0x7fc03819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 --2- 172.21.15.96:0/4285883754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc038102de0 0x7fc038199240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.706+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 >> 172.21.15.96:0/4285883754 conn(0x7fc0380fe830 msgr2=0x7fc038104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.708+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 shutdown_connections 2024-09-10T22:49:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:38.708+0000 7fc03ec58640 1 -- 172.21.15.96:0/4285883754 wait complete. 2024-09-10T22:49:39.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:49:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:39 smithi121 ceph-mon[35119]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-09-10T22:49:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4285883754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:39 smithi117 ceph-mon[35081]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-09-10T22:49:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4285883754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:39.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:39 smithi096 ceph-mon[30869]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 732 op/s 2024-09-10T22:49:39.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4285883754' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:49:39.830 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 -- 172.21.15.96:0/1608638204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868101c90 msgr2=0x7f5868102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 --2- 172.21.15.96:0/1608638204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868101c90 0x7f5868102070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 -- 172.21.15.96:0/1608638204 shutdown_connections 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 --2- 172.21.15.96:0/1608638204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868108020 0x7f586810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 --2- 172.21.15.96:0/1608638204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f5868107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 --2- 172.21.15.96:0/1608638204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868101c90 0x7f5868102070 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 -- 172.21.15.96:0/1608638204 >> 172.21.15.96:0/1608638204 conn(0x7f58680fd700 msgr2=0x7f58680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 -- 172.21.15.96:0/1608638204 shutdown_connections 2024-09-10T22:49:40.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.361+0000 7f586cb41640 1 -- 172.21.15.96:0/1608638204 wait complete. 2024-09-10T22:49:40.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.362+0000 7f586cb41640 1 Processor -- start 2024-09-10T22:49:40.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.363+0000 7f586cb41640 1 -- start start 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 0x7f586810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f586810a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868108020 0x7f586810afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110c40 con 0x7f5868108020 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110ac0 con 0x7f5868101c90 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586cb41640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110dc0 con 0x7f58681076a0 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f585ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f586810a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 0x7f586810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f585ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f586810a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47336/0 (socket says 172.21.15.96:47336) 2024-09-10T22:49:40.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f5866d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868108020 0x7f586810afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f585ffff640 1 -- 172.21.15.96:0/4069540904 learned_addr learned my addr 172.21.15.96:0/4069540904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 -- 172.21.15.96:0/4069540904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 msgr2=0x7f586810a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f586810a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 -- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868108020 msgr2=0x7f586810afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868108020 0x7f586810afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.364+0000 7f586659c640 1 -- 172.21.15.96:0/4069540904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681abb80 con 0x7f5868101c90 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.365+0000 7f586659c640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 0x7f586810e6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f585802aa00 tx=0x7f5858003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.365+0000 7f585f7fe640 1 -- 172.21.15.96:0/4069540904 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585803f070 con 0x7f5868101c90 2024-09-10T22:49:40.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.365+0000 7f585f7fe640 1 -- 172.21.15.96:0/4069540904 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5858031d60 con 0x7f5868101c90 2024-09-10T22:49:40.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.366+0000 7f585f7fe640 1 -- 172.21.15.96:0/4069540904 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858035030 con 0x7f5868101c90 2024-09-10T22:49:40.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.366+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681abd50 con 0x7f5868101c90 2024-09-10T22:49:40.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.366+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681ac1b0 con 0x7f5868101c90 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.369+0000 7f586ef16740 1 Processor -- start 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.369+0000 7f586ef16740 1 -- start start 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849cea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849ced180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c849c43ea0 con 0x55c849ccddf0 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c849c332d0 con 0x55c849cead90 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f586ef16740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c849c370e0 con 0x55c849ccd4d0 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f585f7fe640 1 -- 172.21.15.96:0/4069540904 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f585804c060 con 0x7f5868101c90 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47340/0 (socket says 172.21.15.96:47340) 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 -- 172.21.15.96:0/781711666 learned_addr learned my addr 172.21.15.96:0/781711666 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f585ffff640 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849ced180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 -- 172.21.15.96:0/781711666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 msgr2=0x55c849ced180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849ced180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 -- 172.21.15.96:0/781711666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 msgr2=0x55c849cea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849cea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.370+0000 7f5866d9d640 1 -- 172.21.15.96:0/781711666 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c849991650 con 0x55c849ccd4d0 2024-09-10T22:49:40.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.371+0000 7f5866d9d640 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58600103a0 tx=0x7f5860010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f5d1b2b760a4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.371+0000 7f585cff9640 1 -- 172.21.15.96:0/781711666 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860016070 con 0x55c849ccd4d0 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.371+0000 7f585cff9640 1 -- 172.21.15.96:0/781711666 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860004d20 con 0x55c849ccd4d0 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.372+0000 7f585cff9640 1 -- 172.21.15.96:0/781711666 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860005420 con 0x55c849ccd4d0 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- 172.21.15.96:0/781711666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 msgr2=0x55c849ccd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58600103a0 tx=0x7f5860010860 comp rx=0 tx=0).stop 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f585f7fe640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 0x7f58540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- 172.21.15.96:0/781711666 shutdown_connections 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849ced180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849cea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 --2- 172.21.15.96:0/781711666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccd8b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- 172.21.15.96:0/781711666 >> 172.21.15.96:0/781711666 conn(0x55c849ccb300 msgr2=0x55c849ccc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- 172.21.15.96:0/781711666 shutdown_connections 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f5866d9d640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 0x7f58540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- 172.21.15.96:0/781711666 wait complete. 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f585f7fe640 1 -- 172.21.15.96:0/4069540904 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5858105df0 con 0x7f5868101c90 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.373+0000 7f586ef16740 1 -- start start 2024-09-10T22:49:40.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 Processor -- start 2024-09-10T22:49:40.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849ce7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849cead90 0x55c849ce8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c849a94700 con 0x55c849cead90 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c849bce160 con 0x55c849ccd4d0 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c849ceda90 con 0x55c849ccddf0 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586ef16740 10 client.0 _ll_register_callbacks cb 0x55c849be79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40504/0 (socket says 172.21.15.96:40504) 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.374+0000 7f586659c640 1 -- 172.21.15.96:0/3641043457 learned_addr learned my addr 172.21.15.96:0/3641043457 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f5866d9d640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 0x7f58540b7820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009db0 tx=0x7f5860000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 -- 172.21.15.96:0/3641043457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 msgr2=0x55c849ce7be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849ce7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 -- 172.21.15.96:0/3641043457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849cead90 msgr2=0x55c849ce8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849cead90 0x55c849ce8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 -- 172.21.15.96:0/3641043457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c849bfbc90 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586659c640 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58580cc690 tx=0x7f58580c8660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903b3e7d021c669f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f584dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049070 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f585803b460 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.375+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5860004d20 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f58580cc210 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 10 client.25430 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c849cd3800 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860004d20 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f58580cc960 con 0x55c849ccd4d0 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 10 client.25430 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 10 client.25430 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 20 client.25430 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849cd4e20 0x55c849cd72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 4 client.25430 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:40.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.376+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c849c45df0 con 0x55c849cd4e20 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.377+0000 7f585ffff640 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849cd4e20 0x55c849cd72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.377+0000 7f584dffb640 10 client.25430 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.377+0000 7f585ffff640 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849cd4e20 0x55c849cd72e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abfd86947329a209 server_cookie=12fc2b90872fde65 in_seq=0 out_seq=0 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.378+0000 7f584dffb640 1 -- 172.21.15.96:0/3641043457 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c849c45df0 con 0x55c849cd4e20 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.378+0000 7f584dffb640 10 client.25430 handle_command_reply: tid=0 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.378+0000 7f586ef16740 1 client.25430 shutdown 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 msgr2=0x55c849cea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58580cc690 tx=0x7f58580c8660 comp rx=0 tx=0).stop 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 shutdown_connections 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849cd4e20 0x55c849cd72e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849cead90 0x55c849ce8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849ce7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 --2- 172.21.15.96:0/3641043457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849ccd4d0 0x55c849cea790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 >> 172.21.15.96:0/3641043457 conn(0x55c849ccb300 msgr2=0x55c849cd0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f584dffb640 0 client.25430 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:49:40.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 shutdown_connections 2024-09-10T22:49:40.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 1 -- 172.21.15.96:0/3641043457 wait complete. 2024-09-10T22:49:40.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.379+0000 7f586ef16740 20 client.25430 trim_cache size 0 max 16384 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 Processor -- start 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 -- start start 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849ccc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f586811ea30 con 0x55c849ccddf0 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58681cfba0 con 0x55c849cead90 2024-09-10T22:49:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586ef16740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58681ce730 con 0x55c849ccd4d0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47356/0 (socket says 172.21.15.96:47356) 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.400+0000 7f586659c640 1 -- 172.21.15.96:0/2450158739 learned_addr learned my addr 172.21.15.96:0/2450158739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f585ffff640 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849ccc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f5866d9d640 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 -- 172.21.15.96:0/2450158739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 msgr2=0x55c849cebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cebcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 -- 172.21.15.96:0/2450158739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 msgr2=0x55c849ccc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849ccc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 -- 172.21.15.96:0/2450158739 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c849cf71c0 con 0x55c849ccd4d0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f586659c640 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5858000c00 tx=0x7f58580c86d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d6eb3aea80e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f584ffff640 1 -- 172.21.15.96:0/2450158739 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049070 con 0x55c849ccd4d0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f584ffff640 1 -- 172.21.15.96:0/2450158739 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58580cc700 con 0x55c849ccd4d0 2024-09-10T22:49:40.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.401+0000 7f584ffff640 1 -- 172.21.15.96:0/2450158739 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585803b460 con 0x55c849ccd4d0 2024-09-10T22:49:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- 172.21.15.96:0/2450158739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 msgr2=0x55c849ccc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5858000c00 tx=0x7f58580c86d0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- 172.21.15.96:0/2450158739 shutdown_connections 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccddf0 0x55c849ccc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 --2- 172.21.15.96:0/2450158739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccd4d0 0x55c849ccc430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- 172.21.15.96:0/2450158739 >> 172.21.15.96:0/2450158739 conn(0x55c849ccb300 msgr2=0x55c849cd6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- 172.21.15.96:0/2450158739 shutdown_connections 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- 172.21.15.96:0/2450158739 wait complete. 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.402+0000 7f586ef16740 1 -- start start 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 Processor -- start 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849cedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58681cf4d0 con 0x55c849ccd4d0 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58681cee00 con 0x55c849cead90 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f586811ca30 con 0x55c849ccddf0 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.403+0000 7f586ef16740 10 client.0 _ll_register_callbacks cb 0x7f5858003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.404+0000 7f586659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.404+0000 7f586659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58724/0 (socket says 172.21.15.96:58724) 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.404+0000 7f586659c640 1 -- 172.21.15.96:0/2837655191 learned_addr learned my addr 172.21.15.96:0/2837655191 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.406+0000 7f586659c640 1 -- 172.21.15.96:0/2837655191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 msgr2=0x55c849cedd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:49:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.406+0000 7f586659c640 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849cedd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.406+0000 7f586659c640 1 -- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 msgr2=0x55c849cee3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:49:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.406+0000 7f586659c640 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cee3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.406+0000 7f586659c640 1 -- 172.21.15.96:0/2837655191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c849d556f0 con 0x55c849ccd4d0 2024-09-10T22:49:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.407+0000 7f586659c640 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f58580c8ec0 tx=0x7f585802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f79e9ec3c2b564 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.407+0000 7f584dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:40.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.409+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049070 con 0x55c849ccd4d0 2024-09-10T22:49:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.409+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f58580318b0 con 0x55c849ccd4d0 2024-09-10T22:49:40.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.412+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58580cc700 con 0x55c849ccd4d0 2024-09-10T22:49:40.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.413+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f58580d1130 con 0x55c849ccd4d0 2024-09-10T22:49:40.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.413+0000 7f586ef16740 10 client.15867 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:40.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.413+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c849d556f0 con 0x55c849ccd4d0 2024-09-10T22:49:40.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58580e7450 con 0x55c849ccd4d0 2024-09-10T22:49:40.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f58580cb3d0 con 0x55c849ccd4d0 2024-09-10T22:49:40.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 10 client.15867 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:40.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 10 client.15867 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:49:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 20 client.15867 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849d56af0 0x55c849d58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 4 client.15867 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:49:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.414+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f58580cc700 con 0x55c849d56af0 2024-09-10T22:49:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.415+0000 7f585ffff640 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849d56af0 0x55c849d58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:40.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.420+0000 7f585ffff640 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849d56af0 0x55c849d58ee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f4ff39c16c5f2ec server_cookie=8c111d833f0eec15 in_seq=0 out_seq=0 2024-09-10T22:49:40.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.420+0000 7f584dffb640 10 client.15867 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:49:40.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.421+0000 7f584dffb640 1 -- 172.21.15.96:0/2837655191 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f58580cc700 con 0x55c849d56af0 2024-09-10T22:49:40.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.421+0000 7f584dffb640 10 client.15867 handle_command_reply: tid=0 2024-09-10T22:49:40.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.421+0000 7f586ef16740 1 client.15867 shutdown 2024-09-10T22:49:40.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.426+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 msgr2=0x55c849cf3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.426+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f58580c8ec0 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-09-10T22:49:40.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 shutdown_connections 2024-09-10T22:49:40.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c849d56af0 0x55c849d58ee0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c849cead90 0x55c849cee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c849ccddf0 0x55c849cedd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 --2- 172.21.15.96:0/2837655191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c849ccd4d0 0x55c849cf3680 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 >> 172.21.15.96:0/2837655191 conn(0x55c849ccb300 msgr2=0x55c849cebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f584dffb640 0 client.15867 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.429+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 shutdown_connections 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.430+0000 7f586ef16740 1 -- 172.21.15.96:0/2837655191 wait complete. 2024-09-10T22:49:40.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.430+0000 7f586ef16740 20 client.15867 trim_cache size 0 max 16384 2024-09-10T22:49:40.433 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 msgr2=0x7f58540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 0x7f58540b7820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5860009db0 tx=0x7f5860000f40 comp rx=0 tx=0).stop 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 msgr2=0x7f586810e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 0x7f586810e6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f585802aa00 tx=0x7f5858003cd0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 shutdown_connections 2024-09-10T22:49:40.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f58540b5360 0x7f58540b7820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5868108020 0x7f586810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58681076a0 0x7f586810a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 --2- 172.21.15.96:0/4069540904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101c90 0x7f586810e6e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.432+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 >> 172.21.15.96:0/4069540904 conn(0x7f58680fd700 msgr2=0x7f58680ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.433+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 shutdown_connections 2024-09-10T22:49:40.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:40.433+0000 7f586cb41640 1 -- 172.21.15.96:0/4069540904 wait complete. 2024-09-10T22:49:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 125.0 s, 2940.8 tps, lat 34.097 ms stddev 36.653 2024-09-10T22:49:41.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:49:41.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:41 smithi121 ceph-mon[35119]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 846 op/s 2024-09-10T22:49:41.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:41 smithi117 ceph-mon[35081]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 846 op/s 2024-09-10T22:49:41.655 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:41 smithi096 ceph-mon[30869]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 846 op/s 2024-09-10T22:49:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 -- 172.21.15.96:0/335018598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 msgr2=0x7fcb801020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 --2- 172.21.15.96:0/335018598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb801020f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009900 tx=0x7fcb7002a410 comp rx=0 tx=0).stop 2024-09-10T22:49:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 -- 172.21.15.96:0/335018598 shutdown_connections 2024-09-10T22:49:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 --2- 172.21.15.96:0/335018598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb801080a0 0x7fcb8010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 --2- 172.21.15.96:0/335018598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb80107720 0x7fcb80107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 --2- 172.21.15.96:0/335018598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb801020f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.239+0000 7fcb868fa640 1 -- 172.21.15.96:0/335018598 >> 172.21.15.96:0/335018598 conn(0x7fcb800fd7c0 msgr2=0x7fcb800ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- 172.21.15.96:0/335018598 shutdown_connections 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- 172.21.15.96:0/335018598 wait complete. 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 Processor -- start 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- start start 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb80074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb80107720 0x7fcb80074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8006d430 con 0x7fcb801080a0 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8006d2b0 con 0x7fcb80101d10 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.240+0000 7fcb868fa640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8006d5b0 con 0x7fcb80107720 2024-09-10T22:49:42.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58748/0 (socket says 172.21.15.96:58748) 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 -- 172.21.15.96:0/4043003944 learned_addr learned my addr 172.21.15.96:0/4043003944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb77fff640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb80107720 0x7fcb80074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb80074740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 -- 172.21.15.96:0/4043003944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb80107720 msgr2=0x7fcb80074c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb80107720 0x7fcb80074c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 -- 172.21.15.96:0/4043003944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 msgr2=0x7fcb80074740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb80074740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 -- 172.21.15.96:0/4043003944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8006f340 con 0x7fcb801080a0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb84e97640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800cad0 tx=0x7fcb7800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb7dffb640 1 -- 172.21.15.96:0/4043003944 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800ad60 con 0x7fcb801080a0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8006f5d0 con 0x7fcb801080a0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801ac300 con 0x7fcb801080a0 2024-09-10T22:49:42.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb7dffb640 1 -- 172.21.15.96:0/4043003944 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb78005020 con 0x7fcb801080a0 2024-09-10T22:49:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.241+0000 7fcb7dffb640 1 -- 172.21.15.96:0/4043003944 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801d060 con 0x7fcb801080a0 2024-09-10T22:49:42.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.245+0000 7fcb88ccf740 1 Processor -- start 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.245+0000 7fcb7dffb640 1 -- 172.21.15.96:0/4043003944 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fcb78004a40 con 0x7fcb801080a0 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 -- start start 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a891c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a892190 0x55e47a894580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e47a7f7ea0 con 0x55e47a8814d0 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e47a7e72d0 con 0x55e47a881df0 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e47a7eb0e0 con 0x55e47a892190 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58756/0 (socket says 172.21.15.96:58756) 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 -- 172.21.15.96:0/3057076759 learned_addr learned my addr 172.21.15.96:0/3057076759 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a891c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a892190 0x55e47a894580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb7dffb640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 0x7fcb6c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 -- 172.21.15.96:0/3057076759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a892190 msgr2=0x55e47a894580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a892190 0x55e47a894580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 -- 172.21.15.96:0/3057076759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 msgr2=0x55e47a891c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a891c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb77fff640 1 -- 172.21.15.96:0/3057076759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47a545650 con 0x55e47a8814d0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a891c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb84e97640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 0x7fcb6c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.246+0000 7fcb7dffb640 1 -- 172.21.15.96:0/4043003944 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcb780e5a60 con 0x7fcb801080a0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.247+0000 7fcb77fff640 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009390 tx=0x7fcb6800c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a3365f9002fc2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.247+0000 7fcb767fc640 1 -- 172.21.15.96:0/3057076759 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800c110 con 0x55e47a8814d0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.247+0000 7fcb767fc640 1 -- 172.21.15.96:0/3057076759 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6800c2b0 con 0x55e47a8814d0 2024-09-10T22:49:42.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.247+0000 7fcb767fc640 1 -- 172.21.15.96:0/3057076759 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800a730 con 0x55e47a8814d0 2024-09-10T22:49:42.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- 172.21.15.96:0/3057076759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 msgr2=0x55e47a8818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009390 tx=0x7fcb6800c4e0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb84e97640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 0x7fcb6c0b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800caa0 tx=0x7fcb780122b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- 172.21.15.96:0/3057076759 shutdown_connections 2024-09-10T22:49:42.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a891c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a892190 0x55e47a894580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/3057076759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a8814d0 0x55e47a8818b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- 172.21.15.96:0/3057076759 >> 172.21.15.96:0/3057076759 conn(0x55e47a87f300 msgr2=0x55e47a8804a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- 172.21.15.96:0/3057076759 shutdown_connections 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- 172.21.15.96:0/3057076759 wait complete. 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 -- start start 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.248+0000 7fcb88ccf740 1 Processor -- start 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a88ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47370/0 (socket says 172.21.15.96:47370) 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb77fff640 1 -- 172.21.15.96:0/4135003424 learned_addr learned my addr 172.21.15.96:0/4135003424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:42.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a88f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e47a648700 con 0x55e47a892190 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e47a782160 con 0x55e47a881df0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.249+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e47a894e40 con 0x55e47a8814d0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb88ccf740 10 client.0 _ll_register_callbacks cb 0x55e47a79b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a88f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 -- 172.21.15.96:0/4135003424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 msgr2=0x55e47a88ee30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a88ee30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 -- 172.21.15.96:0/4135003424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 msgr2=0x55e47a88f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a88f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47a7afc90 con 0x55e47a8814d0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb77fff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb68016dc0 tx=0x7fcb68016f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72090567949bbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.250+0000 7fcb5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.251+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800c9d0 con 0x55e47a8814d0 2024-09-10T22:49:42.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.251+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcb68005250 con 0x55e47a8814d0 2024-09-10T22:49:42.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.251+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6800c2b0 con 0x55e47a8814d0 2024-09-10T22:49:42.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.251+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb68005d60 con 0x55e47a8814d0 2024-09-10T22:49:42.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.252+0000 7fcb88ccf740 10 client.25459 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:42.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.252+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e47a8a06d0 con 0x55e47a8814d0 2024-09-10T22:49:42.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6800c2b0 con 0x55e47a8814d0 2024-09-10T22:49:42.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcb6800a4b0 con 0x55e47a8814d0 2024-09-10T22:49:42.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 10 client.25459 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 10 client.25459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 20 client.25459 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a8a1b30 0x55e47a8a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 4 client.25459 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.253+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47a7f9df0 con 0x55e47a8a1b30 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.254+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a8a1b30 0x55e47a8a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.254+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a8a1b30 0x55e47a8a3f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b03eed9d25cf661 server_cookie=34bb78ca2d0101c7 in_seq=0 out_seq=0 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.254+0000 7fcb5f7fe640 10 client.25459 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/4135003424 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e47a7f9df0 con 0x55e47a8a1b30 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb5f7fe640 10 client.25459 handle_command_reply: tid=0 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 client.25459 shutdown 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 msgr2=0x55e47a8919e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb68016dc0 tx=0x7fcb68016f60 comp rx=0 tx=0).stop 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 shutdown_connections 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a8a1b30 0x55e47a8a3f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a88ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a88f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/4135003424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a8919e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb5f7fe640 0 client.25459 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 >> 172.21.15.96:0/4135003424 conn(0x55e47a87f300 msgr2=0x55e47a884e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 shutdown_connections 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 1 -- 172.21.15.96:0/4135003424 wait complete. 2024-09-10T22:49:42.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.255+0000 7fcb88ccf740 20 client.25459 trim_cache size 0 max 16384 2024-09-10T22:49:42.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.274+0000 7fcb88ccf740 1 Processor -- start 2024-09-10T22:49:42.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 -- start start 2024-09-10T22:49:42.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 0x55e47a8945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a880900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b4890 con 0x55e47a892190 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b63d0 con 0x55e47a8814d0 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b4f60 con 0x55e47a881df0 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 0x55e47a8945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47376/0 (socket says 172.21.15.96:47376) 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 -- 172.21.15.96:0/2962239885 learned_addr learned my addr 172.21.15.96:0/2962239885 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb77fff640 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a880900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 -- 172.21.15.96:0/2962239885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 msgr2=0x55e47a8945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 0x55e47a8945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 -- 172.21.15.96:0/2962239885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 msgr2=0x55e47a880900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a880900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb7ffff640 1 -- 172.21.15.96:0/2962239885 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47a88f700 con 0x55e47a881df0 2024-09-10T22:49:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.275+0000 7fcb77fff640 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 0x55e47a8945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.276+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009230 tx=0x7fcb70005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7981fa4c22a9447f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.276+0000 7fcb75ffb640 1 -- 172.21.15.96:0/2962239885 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70035040 con 0x55e47a881df0 2024-09-10T22:49:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.276+0000 7fcb75ffb640 1 -- 172.21.15.96:0/2962239885 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7004a070 con 0x55e47a881df0 2024-09-10T22:49:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.276+0000 7fcb75ffb640 1 -- 172.21.15.96:0/2962239885 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70031cb0 con 0x55e47a881df0 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 -- 172.21.15.96:0/2962239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 msgr2=0x55e47a8803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009230 tx=0x7fcb70005a20 comp rx=0 tx=0).stop 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 -- 172.21.15.96:0/2962239885 shutdown_connections 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a880900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a881df0 0x55e47a8803c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/2962239885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a8814d0 0x55e47a8945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.277+0000 7fcb88ccf740 1 -- 172.21.15.96:0/2962239885 >> 172.21.15.96:0/2962239885 conn(0x55e47a87f300 msgr2=0x55e47a893ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.278+0000 7fcb88ccf740 1 -- 172.21.15.96:0/2962239885 shutdown_connections 2024-09-10T22:49:42.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.278+0000 7fcb88ccf740 1 -- 172.21.15.96:0/2962239885 wait complete. 2024-09-10T22:49:42.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.279+0000 7fcb88ccf740 1 -- start start 2024-09-10T22:49:42.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.279+0000 7fcb88ccf740 1 Processor -- start 2024-09-10T22:49:42.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a89a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a8970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b5d00 con 0x55e47a892190 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b5630 con 0x55e47a881df0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801b2890 con 0x55e47a8814d0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb88ccf740 10 client.0 _ll_register_callbacks cb 0x7fcb70009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47382/0 (socket says 172.21.15.96:47382) 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 -- 172.21.15.96:0/1459345284 learned_addr learned my addr 172.21.15.96:0/1459345284 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a8970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a89a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 -- 172.21.15.96:0/1459345284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 msgr2=0x55e47a89a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a89a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 -- 172.21.15.96:0/1459345284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 msgr2=0x55e47a8970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a8970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 -- 172.21.15.96:0/1459345284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47a89ad00 con 0x55e47a8814d0 2024-09-10T22:49:42.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb77fff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aba0 tx=0x7fcb68016fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc429d7181d66c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:42.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:42.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.280+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800c320 con 0x55e47a8814d0 2024-09-10T22:49:42.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcb6800a440 con 0x55e47a8814d0 2024-09-10T22:49:42.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7004a070 con 0x55e47a8814d0 2024-09-10T22:49:42.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb68044050 con 0x55e47a8814d0 2024-09-10T22:49:42.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb88ccf740 10 client.25465 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e47a90ecc0 con 0x55e47a8814d0 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.281+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7004a070 con 0x55e47a8814d0 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcb680048d0 con 0x55e47a8814d0 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 10 client.25465 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 10 client.25465 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 20 client.25465 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a90f360 0x55e47a911780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 4 client.25465 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e47a7f58a0 con 0x55e47a90f360 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a90f360 0x55e47a911780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.282+0000 7fcb5f7fe640 10 client.25465 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb7ffff640 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a90f360 0x55e47a911780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7da31897c6bcad0e server_cookie=49070de5382784b3 in_seq=0 out_seq=0 2024-09-10T22:49:42.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb5f7fe640 1 -- 172.21.15.96:0/1459345284 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e47a7f58a0 con 0x55e47a90f360 2024-09-10T22:49:42.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb5f7fe640 10 client.25465 handle_command_reply: tid=0 2024-09-10T22:49:42.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb88ccf740 1 client.25465 shutdown 2024-09-10T22:49:42.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 msgr2=0x55e47a89a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.283+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800aba0 tx=0x7fcb68016fd0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 shutdown_connections 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e47a90f360 0x55e47a911780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e47a881df0 0x55e47a89a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e47a892190 0x55e47a8970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 --2- 172.21.15.96:0/1459345284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e47a8814d0 0x55e47a89a0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 >> 172.21.15.96:0/1459345284 conn(0x55e47a87f300 msgr2=0x55e47a8803c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb5f7fe640 0 client.25465 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 shutdown_connections 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 1 -- 172.21.15.96:0/1459345284 wait complete. 2024-09-10T22:49:42.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.285+0000 7fcb88ccf740 20 client.25465 trim_cache size 0 max 16384 2024-09-10T22:49:42.289 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 msgr2=0x7fcb6c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 0x7fcb6c0b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800caa0 tx=0x7fcb780122b0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 msgr2=0x7fcb8006ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800cad0 tx=0x7fcb7800ced0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 shutdown_connections 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcb6c0b52e0 0x7fcb6c0b77a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb801080a0 0x7fcb8006ea30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcb80107720 0x7fcb80074c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 --2- 172.21.15.96:0/4043003944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb80101d10 0x7fcb80074740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:42.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 >> 172.21.15.96:0/4043003944 conn(0x7fcb800fd7c0 msgr2=0x7fcb80103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:42.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 shutdown_connections 2024-09-10T22:49:42.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:42.286+0000 7fcb868fa640 1 -- 172.21.15.96:0/4043003944 wait complete. 2024-09-10T22:49:43.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:49:43.489 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:43.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:43 smithi096 ceph-mon[30869]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 702 op/s 2024-09-10T22:49:43.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:43 smithi121 ceph-mon[35119]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 702 op/s 2024-09-10T22:49:43.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:43 smithi117 ceph-mon[35081]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 702 op/s 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 -- 172.21.15.96:0/1908272428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 msgr2=0x7f2620107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 --2- 172.21.15.96:0/1908272428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 0x7f2620107b60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2614009900 tx=0x7f261402a410 comp rx=0 tx=0).stop 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 -- 172.21.15.96:0/1908272428 shutdown_connections 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 --2- 172.21.15.96:0/1908272428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26201080a0 0x7f262010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 --2- 172.21.15.96:0/1908272428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 0x7f2620107b60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 --2- 172.21.15.96:0/1908272428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2620101d10 0x7f26201020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 -- 172.21.15.96:0/1908272428 >> 172.21.15.96:0/1908272428 conn(0x7f26200fd7c0 msgr2=0x7f26200ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 -- 172.21.15.96:0/1908272428 shutdown_connections 2024-09-10T22:49:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.024+0000 7f2625757640 1 -- 172.21.15.96:0/1908272428 wait complete. 2024-09-10T22:49:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.025+0000 7f2625757640 1 Processor -- start 2024-09-10T22:49:44.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.025+0000 7f2625757640 1 -- start start 2024-09-10T22:49:44.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 0x7f262010e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26201080a0 0x7f26201a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2620112dc0 con 0x7f2620107720 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f261effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2620112c40 con 0x7f2620101d10 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f261effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40582/0 (socket says 172.21.15.96:40582) 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f261effd640 1 -- 172.21.15.96:0/3185104372 learned_addr learned my addr 172.21.15.96:0/3185104372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26201080a0 0x7f26201a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.026+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2620112f40 con 0x7f26201080a0 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 -- 172.21.15.96:0/3185104372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26201080a0 msgr2=0x7f26201a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26201080a0 0x7f26201a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 -- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 msgr2=0x7f262010e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 0x7f262010e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 -- 172.21.15.96:0/3185104372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a5a20 con 0x7f2620101d10 2024-09-10T22:49:44.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.027+0000 7f261effd640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.028+0000 7f2607fff640 1 -- 172.21.15.96:0/3185104372 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610017070 con 0x7f2620101d10 2024-09-10T22:49:44.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.028+0000 7f2607fff640 1 -- 172.21.15.96:0/3185104372 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2610009c30 con 0x7f2620101d10 2024-09-10T22:49:44.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.028+0000 7f2607fff640 1 -- 172.21.15.96:0/3185104372 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261000f620 con 0x7f2620101d10 2024-09-10T22:49:44.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.028+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a5d10 con 0x7f2620101d10 2024-09-10T22:49:44.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.028+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201ac3d0 con 0x7f2620101d10 2024-09-10T22:49:44.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.031+0000 7f2607fff640 1 -- 172.21.15.96:0/3185104372 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f2610010460 con 0x7f2620101d10 2024-09-10T22:49:44.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 Processor -- start 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 -- start start 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e3434d0 0x55bc9e3438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e352170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e2b9ea0 con 0x55bc9e3434d0 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e2a92d0 con 0x55bc9e34fd80 2024-09-10T22:49:44.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f2627b2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e2ad0e0 con 0x55bc9e343df0 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.032+0000 7f261effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47402/0 (socket says 172.21.15.96:47402) 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 -- 172.21.15.96:0/3780544624 learned_addr learned my addr 172.21.15.96:0/3780544624 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e3434d0 0x55bc9e3438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 -- 172.21.15.96:0/3780544624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 msgr2=0x55bc9e352170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e352170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 -- 172.21.15.96:0/3780544624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e3434d0 msgr2=0x55bc9e3438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e3434d0 0x55bc9e3438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 -- 172.21.15.96:0/3780544624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9e007650 con 0x55bc9e343df0 2024-09-10T22:49:44.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f261effd640 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26100ebc20 tx=0x7f26100ed670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c566365a5d7b545 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.033+0000 7f26057fa640 1 -- 172.21.15.96:0/3780544624 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100f4070 con 0x55bc9e343df0 2024-09-10T22:49:44.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.034+0000 7f26057fa640 1 -- 172.21.15.96:0/3780544624 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100e7780 con 0x55bc9e343df0 2024-09-10T22:49:44.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.034+0000 7f2607fff640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 0x7f26080b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.034+0000 7f2607fff640 1 -- 172.21.15.96:0/3185104372 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f26100e58a0 con 0x7f2620101d10 2024-09-10T22:49:44.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.034+0000 7f26057fa640 1 -- 172.21.15.96:0/3780544624 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100f0ae0 con 0x55bc9e343df0 2024-09-10T22:49:44.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 -- 172.21.15.96:0/3780544624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 msgr2=0x55bc9e34f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26100ebc20 tx=0x7f26100ed670 comp rx=0 tx=0).stop 2024-09-10T22:49:44.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 0x7f26080b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 -- 172.21.15.96:0/3780544624 shutdown_connections 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e352170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34f840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3780544624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e3434d0 0x55bc9e3438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f2627b2c740 1 -- 172.21.15.96:0/3780544624 >> 172.21.15.96:0/3780544624 conn(0x55bc9e341300 msgr2=0x55bc9e3424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.035+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 0x7f26080b78a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2618005fd0 tx=0x7f26180096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.036+0000 7f2627b2c740 1 -- 172.21.15.96:0/3780544624 shutdown_connections 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.036+0000 7f2627b2c740 1 -- 172.21.15.96:0/3780544624 wait complete. 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.036+0000 7f2627b2c740 1 -- start start 2024-09-10T22:49:44.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.036+0000 7f2627b2c740 1 Processor -- start 2024-09-10T22:49:44.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e34fd80 0x55bc9e34cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e10a700 con 0x55bc9e34fd80 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e244160 con 0x55bc9e3434d0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bc9e352aa0 con 0x55bc9e343df0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f261effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f261effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40600/0 (socket says 172.21.15.96:40600) 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f261effd640 1 -- 172.21.15.96:0/2912290449 learned_addr learned my addr 172.21.15.96:0/2912290449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f2627b2c740 10 client.0 _ll_register_callbacks cb 0x55bc9e25d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f261f7fe640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e34fd80 0x55bc9e34cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.037+0000 7f261e7fc640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 -- 172.21.15.96:0/2912290449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 msgr2=0x55bc9e34c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 -- 172.21.15.96:0/2912290449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e34fd80 msgr2=0x55bc9e34cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e34fd80 0x55bc9e34cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 -- 172.21.15.96:0/2912290449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9e271c90 con 0x55bc9e3434d0 2024-09-10T22:49:44.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f261effd640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26100ebbf0 tx=0x7f2610005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d134eeb492c026 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.038+0000 7f2602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:44.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610020020 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f26100ac2c0 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100e7780 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26100f4070 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2627b2c740 10 client.25457 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.039+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc9e35e350 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100e7780 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f26100971c0 con 0x55bc9e3434d0 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 10 client.25457 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 10 client.25457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 20 client.25457 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e35f6a0 0x55bc9e361a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 4 client.25457 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:44.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.040+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc9e2bbdf0 con 0x55bc9e35f6a0 2024-09-10T22:49:44.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.041+0000 7f261e7fc640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e35f6a0 0x55bc9e361a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.042+0000 7f261e7fc640 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e35f6a0 0x55bc9e361a90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=209867e7e93a2426 server_cookie=fdd4c4153fde15c in_seq=0 out_seq=0 2024-09-10T22:49:44.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.042+0000 7f2602ff5640 10 client.25457 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:49:44.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.042+0000 7f2602ff5640 1 -- 172.21.15.96:0/2912290449 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc9e2bbdf0 con 0x55bc9e35f6a0 2024-09-10T22:49:44.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.042+0000 7f2602ff5640 10 client.25457 handle_command_reply: tid=0 2024-09-10T22:49:44.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.043+0000 7f2627b2c740 1 client.25457 shutdown 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.043+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 msgr2=0x55bc9e34f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.043+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26100ebbf0 tx=0x7f2610005f70 comp rx=0 tx=0).stop 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 shutdown_connections 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e35f6a0 0x55bc9e361a90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e34fd80 0x55bc9e34cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e343df0 0x55bc9e34c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 --2- 172.21.15.96:0/2912290449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e3434d0 0x55bc9e34f590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 >> 172.21.15.96:0/2912290449 conn(0x55bc9e341300 msgr2=0x55bc9e346e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2602ff5640 0 client.25457 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 shutdown_connections 2024-09-10T22:49:44.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 1 -- 172.21.15.96:0/2912290449 wait complete. 2024-09-10T22:49:44.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.044+0000 7f2627b2c740 20 client.25457 trim_cache size 0 max 16384 2024-09-10T22:49:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.061+0000 7f2627b2c740 1 Processor -- start 2024-09-10T22:49:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.061+0000 7f2627b2c740 1 -- start start 2024-09-10T22:49:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.061+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e340c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e367670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e34e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f261effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e340c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f261effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e340c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47420/0 (socket says 172.21.15.96:47420) 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f261effd640 1 -- 172.21.15.96:0/3429176142 learned_addr learned my addr 172.21.15.96:0/3429176142 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b4a10 con 0x55bc9e343df0 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b6550 con 0x55bc9e34fd80 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f261e7fc640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e367670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b50e0 con 0x55bc9e3434d0 2024-09-10T22:49:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.062+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e34e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 -- 172.21.15.96:0/3429176142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 msgr2=0x55bc9e340c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e340c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 -- 172.21.15.96:0/3429176142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 msgr2=0x55bc9e34e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e34e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 -- 172.21.15.96:0/3429176142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9e343c20 con 0x55bc9e343df0 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261f7fe640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e34e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f261e7fc640 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e367670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55bc9e352b60 tx=0x7f2614032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f97e33cfb37040 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f2604ff9640 1 -- 172.21.15.96:0/3429176142 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261404b070 con 0x55bc9e343df0 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f2604ff9640 1 -- 172.21.15.96:0/3429176142 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26140054f0 con 0x55bc9e343df0 2024-09-10T22:49:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.063+0000 7f2604ff9640 1 -- 172.21.15.96:0/3429176142 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614035040 con 0x55bc9e343df0 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 msgr2=0x55bc9e367670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e367670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55bc9e352b60 tx=0x7f2614032f60 comp rx=0 tx=0).stop 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 shutdown_connections 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e34e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e367670 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 --2- 172.21.15.96:0/3429176142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e340c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.065+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 >> 172.21.15.96:0/3429176142 conn(0x55bc9e341300 msgr2=0x55bc9e34e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.066+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 shutdown_connections 2024-09-10T22:49:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.066+0000 7f2627b2c740 1 -- 172.21.15.96:0/3429176142 wait complete. 2024-09-10T22:49:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 -- start start 2024-09-10T22:49:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 Processor -- start 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e362f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e3cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b5e80 con 0x55bc9e343df0 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b57b0 con 0x55bc9e34fd80 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b29f0 con 0x55bc9e3434d0 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f2627b2c740 10 client.0 _ll_register_callbacks cb 0x7f2618009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f261e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f261e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58848/0 (socket says 172.21.15.96:58848) 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f261e7fc640 1 -- 172.21.15.96:0/1276868949 learned_addr learned my addr 172.21.15.96:0/1276868949 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f261effd640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e362f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.067+0000 7f261f7fe640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e3cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 -- 172.21.15.96:0/1276868949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 msgr2=0x55bc9e362f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e362f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 -- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 msgr2=0x55bc9e3cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e3cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 -- 172.21.15.96:0/1276868949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9e3cb900 con 0x55bc9e343df0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261effd640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e362f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f261e7fc640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2614009850 tx=0x7f26140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a61c8f70a526e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2602ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614054060 con 0x55bc9e343df0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2614004040 con 0x55bc9e343df0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26140054f0 con 0x55bc9e343df0 2024-09-10T22:49:44.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2614035070 con 0x55bc9e343df0 2024-09-10T22:49:44.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2627b2c740 10 client.15897 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:44.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.068+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc9e3ccc00 con 0x55bc9e343df0 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26140054f0 con 0x55bc9e343df0 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2614059070 con 0x55bc9e343df0 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 10 client.15897 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 10 client.15897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 20 client.15897 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e3cd2e0 0x55bc9e3cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 4 client.15897 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.069+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc9e2b78a0 con 0x55bc9e3cd2e0 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.070+0000 7f261effd640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e3cd2e0 0x55bc9e3cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.071+0000 7f261effd640 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e3cd2e0 0x55bc9e3cf700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31234070be2b295e server_cookie=a8f9a17795cb8d8f in_seq=0 out_seq=0 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.071+0000 7f2602ff5640 10 client.15897 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2602ff5640 1 -- 172.21.15.96:0/1276868949 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc9e2b78a0 con 0x55bc9e3cd2e0 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2602ff5640 10 client.15897 handle_command_reply: tid=0 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 client.15897 shutdown 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 msgr2=0x55bc9e363480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2614009850 tx=0x7f26140048f0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 shutdown_connections 2024-09-10T22:49:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bc9e3cd2e0 0x55bc9e3cf700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc9e34fd80 0x55bc9e3cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bc9e343df0 0x55bc9e363480 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 --2- 172.21.15.96:0/1276868949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc9e3434d0 0x55bc9e362f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 >> 172.21.15.96:0/1276868949 conn(0x55bc9e341300 msgr2=0x55bc9e34e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2602ff5640 0 client.15897 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 shutdown_connections 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 1 -- 172.21.15.96:0/1276868949 wait complete. 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.072+0000 7f2627b2c740 20 client.15897 trim_cache size 0 max 16384 2024-09-10T22:49:44.076 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:49:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 msgr2=0x7f26080b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 0x7f26080b78a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2618005fd0 tx=0x7f26180096e0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 msgr2=0x7f262010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de10 comp rx=0 tx=0).stop 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 shutdown_connections 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26080b53e0 0x7f26080b78a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26201080a0 0x7f26201a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2620107720 0x7f262010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 --2- 172.21.15.96:0/3185104372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2620101d10 0x7f262010dcb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 >> 172.21.15.96:0/3185104372 conn(0x7f26200fd7c0 msgr2=0x7f2620104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.073+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 shutdown_connections 2024-09-10T22:49:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:44.074+0000 7f2625757640 1 -- 172.21.15.96:0/3185104372 wait complete. 2024-09-10T22:49:44.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:49:45.250 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:45.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:45 smithi117 ceph-mon[35081]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-09-10T22:49:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:45 smithi096 ceph-mon[30869]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-09-10T22:49:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:45 smithi121 ceph-mon[35119]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-09-10T22:49:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 130.0 s, 2954.6 tps, lat 33.826 ms stddev 38.432 2024-09-10T22:49:46.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.448+0000 7fe0696d1640 1 -- 172.21.15.96:0/1221713215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 msgr2=0x7fe05c097660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.448+0000 7fe0696d1640 1 --2- 172.21.15.96:0/1221713215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c097660 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe05802a480 comp rx=0 tx=0).stop 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 -- 172.21.15.96:0/1221713215 shutdown_connections 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 --2- 172.21.15.96:0/1221713215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c0948b0 0x7fe05c094cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 --2- 172.21.15.96:0/1221713215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c095230 0x7fe05c0a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 --2- 172.21.15.96:0/1221713215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c097660 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 -- 172.21.15.96:0/1221713215 >> 172.21.15.96:0/1221713215 conn(0x7fe05c00a160 msgr2=0x7fe05c00c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 -- 172.21.15.96:0/1221713215 shutdown_connections 2024-09-10T22:49:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 -- 172.21.15.96:0/1221713215 wait complete. 2024-09-10T22:49:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 Processor -- start 2024-09-10T22:49:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.450+0000 7fe0696d1640 1 -- start start 2024-09-10T22:49:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 0x7fe05c098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe063fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0a4740 con 0x7fe05c097280 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0a45c0 con 0x7fe05c095230 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0696d1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0a48c0 con 0x7fe05c0948b0 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 0x7fe05c098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe068ed0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.451+0000 7fe063fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43474/0 (socket says 172.21.15.96:43474) 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe068ed0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41712/0 (socket says 172.21.15.96:41712) 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe068ed0640 1 -- 172.21.15.96:0/3494247345 learned_addr learned my addr 172.21.15.96:0/3494247345 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 -- 172.21.15.96:0/3494247345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 msgr2=0x7fe05c098000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 -- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 msgr2=0x7fe05c098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 -- 172.21.15.96:0/3494247345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c13a1a0 con 0x7fe05c095230 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe068ed0640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe063fff640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:46.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.452+0000 7fe0637fe640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 0x7fe05c098540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d6d0 tx=0x7fe05000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.453+0000 7fe0617fa640 1 -- 172.21.15.96:0/3494247345 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050017070 con 0x7fe05c095230 2024-09-10T22:49:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.453+0000 7fe0617fa640 1 -- 172.21.15.96:0/3494247345 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe050009820 con 0x7fe05c095230 2024-09-10T22:49:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.453+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c13a490 con 0x7fe05c095230 2024-09-10T22:49:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.453+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c13a9a0 con 0x7fe05c095230 2024-09-10T22:49:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.453+0000 7fe0617fa640 1 -- 172.21.15.96:0/3494247345 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004c40 con 0x7fe05c095230 2024-09-10T22:49:46.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.457+0000 7fe0617fa640 1 -- 172.21.15.96:0/3494247345 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe050010460 con 0x7fe05c095230 2024-09-10T22:49:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 Processor -- start 2024-09-10T22:49:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 -- start start 2024-09-10T22:49:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ebba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 0x562101ed0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562101e31ea0 con 0x562101ebb6a0 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562101e212d0 con 0x562101ebbfc0 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe06cd08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562101e250e0 con 0x562101ed0760 2024-09-10T22:49:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43488/0 (socket says 172.21.15.96:43488) 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe0637fe640 1 -- 172.21.15.96:0/3879939736 learned_addr learned my addr 172.21.15.96:0/3879939736 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.458+0000 7fe063fff640 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 0x562101ed0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe063fff640 1 -- 172.21.15.96:0/3879939736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 msgr2=0x562101ed2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe063fff640 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe063fff640 1 -- 172.21.15.96:0/3879939736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 msgr2=0x562101ebba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe063fff640 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ebba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe063fff640 1 -- 172.21.15.96:0/3879939736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562101b7f650 con 0x562101ebbfc0 2024-09-10T22:49:46.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.459+0000 7fe0617fa640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 0x7fe0540b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.460+0000 7fe068ed0640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 0x7fe0540b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.460+0000 7fe0617fa640 1 -- 172.21.15.96:0/3494247345 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0500e5360 con 0x7fe05c095230 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.460+0000 7fe063fff640 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 0x562101ed0220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe05802aa70 tx=0x7fe058004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f754a793d38b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.460+0000 7fe068ed0640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 0x7fe0540b78f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe064052890 tx=0x7fe06407b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.461+0000 7fe04e7fc640 1 -- 172.21.15.96:0/3879939736 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058035040 con 0x562101ebbfc0 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.461+0000 7fe04e7fc640 1 -- 172.21.15.96:0/3879939736 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe058004760 con 0x562101ebbfc0 2024-09-10T22:49:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.461+0000 7fe04e7fc640 1 -- 172.21.15.96:0/3879939736 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058003a10 con 0x562101ebbfc0 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.461+0000 7fe06cd08740 1 -- 172.21.15.96:0/3879939736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 msgr2=0x562101ed0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.461+0000 7fe06cd08740 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 0x562101ed0220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe05802aa70 tx=0x7fe058004530 comp rx=0 tx=0).stop 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.462+0000 7fe06cd08740 1 -- 172.21.15.96:0/3879939736 shutdown_connections 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.462+0000 7fe06cd08740 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ebbfc0 0x562101ed0220 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe05802aa70 tx=0x7fe058004530 comp rx=0 tx=0).stop 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.462+0000 7fe06cd08740 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.462+0000 7fe06cd08740 1 --2- 172.21.15.96:0/3879939736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ebba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.462+0000 7fe06cd08740 1 -- 172.21.15.96:0/3879939736 >> 172.21.15.96:0/3879939736 conn(0x562101eb94d0 msgr2=0x562101eba670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.463+0000 7fe06cd08740 1 -- 172.21.15.96:0/3879939736 shutdown_connections 2024-09-10T22:49:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.463+0000 7fe06cd08740 1 -- 172.21.15.96:0/3879939736 wait complete. 2024-09-10T22:49:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.463+0000 7fe06cd08740 1 -- start start 2024-09-10T22:49:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.463+0000 7fe06cd08740 1 Processor -- start 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ec5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562101c82700 con 0x562101ebb6a0 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562101dbc160 con 0x562101ec5d80 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562101ec1cf0 con 0x562101ed0760 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe06cd08740 10 client.0 _ll_register_callbacks cb 0x562101dd59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37082/0 (socket says 172.21.15.96:37082) 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 -- 172.21.15.96:0/2685059570 learned_addr learned my addr 172.21.15.96:0/2685059570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe063fff640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ec5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 -- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 msgr2=0x562101ed0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.464+0000 7fe068ed0640 1 -- 172.21.15.96:0/2685059570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 msgr2=0x562101ec5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ec5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe068ed0640 1 -- 172.21.15.96:0/2685059570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562101de9c90 con 0x562101ec5d80 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe063fff640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ec5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe064051c90 tx=0x7fe0640690b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7c7402eb597ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0437fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:46.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064093420 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0640843b0 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe058004760 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0640845b0 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe06cd08740 10 client.25475 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562101ee7d00 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.465+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe058004760 con 0x562101ec5d80 2024-09-10T22:49:46.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.466+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe06409c020 con 0x562101ec5d80 2024-09-10T22:49:46.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.466+0000 7fe06cd08740 10 client.25475 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:46.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.466+0000 7fe06cd08740 10 client.25475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:49:46.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.466+0000 7fe06cd08740 20 client.25475 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.466+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101ee9200 0x562101eeb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.467+0000 7fe06cd08740 4 client.25475 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.467+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562101e33df0 con 0x562101ee9200 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.467+0000 7fe063fff640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101ee9200 0x562101eeb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.468+0000 7fe0437fe640 10 client.25475 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.468+0000 7fe063fff640 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101ee9200 0x562101eeb630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14f0cdf547008099 server_cookie=82e3f1ca7aa0d5d7 in_seq=0 out_seq=0 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.469+0000 7fe0437fe640 1 -- 172.21.15.96:0/2685059570 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562101e33df0 con 0x562101ee9200 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.469+0000 7fe0437fe640 10 client.25475 handle_command_reply: tid=0 2024-09-10T22:49:46.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.469+0000 7fe06cd08740 1 client.25475 shutdown 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 msgr2=0x562101ecaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe064051c90 tx=0x7fe0640690b0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 shutdown_connections 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101ee9200 0x562101eeb630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101ecaec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ed0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2685059570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101ec5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 >> 172.21.15.96:0/2685059570 conn(0x562101eb94d0 msgr2=0x562101ebf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe0437fe640 0 client.25475 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:49:46.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 shutdown_connections 2024-09-10T22:49:46.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 1 -- 172.21.15.96:0/2685059570 wait complete. 2024-09-10T22:49:46.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.470+0000 7fe06cd08740 20 client.25475 trim_cache size 0 max 16384 2024-09-10T22:49:46.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.488+0000 7fe06cd08740 1 Processor -- start 2024-09-10T22:49:46.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.488+0000 7fe06cd08740 1 -- start start 2024-09-10T22:49:46.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 0x562101ec2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 0x562101ef0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe06cd08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800e320 con 0x562101ebb6a0 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41742/0 (socket says 172.21.15.96:41742) 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe063fff640 1 -- 172.21.15.96:0/2081073662 learned_addr learned my addr 172.21.15.96:0/2081073662 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 0x562101ec2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.489+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 0x562101ef0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800fe60 con 0x562101ed0760 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800e9f0 con 0x562101ec5d80 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 -- 172.21.15.96:0/2081073662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 msgr2=0x562101ec2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 0x562101ec2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 -- 172.21.15.96:0/2081073662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 msgr2=0x562101eb8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 -- 172.21.15.96:0/2081073662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562101ec3b20 con 0x562101ed0760 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 0x562101ec2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe063fff640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:46.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.490+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 0x562101ef0f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe064068db0 tx=0x7fe064069680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b79c678420f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe04dffb640 1 -- 172.21.15.96:0/2081073662 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640a4040 con 0x562101ed0760 2024-09-10T22:49:46.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe04dffb640 1 -- 172.21.15.96:0/2081073662 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06409e460 con 0x562101ed0760 2024-09-10T22:49:46.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe04dffb640 1 -- 172.21.15.96:0/2081073662 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064084e60 con 0x562101ed0760 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 msgr2=0x562101ef0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 0x562101ef0f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe064068db0 tx=0x7fe064069680 comp rx=0 tx=0).stop 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 shutdown_connections 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ed0760 0x562101ef0f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ec5d80 0x562101ec2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2081073662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101eb8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.491+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 >> 172.21.15.96:0/2081073662 conn(0x562101eb94d0 msgr2=0x562101ecadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.492+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 shutdown_connections 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.492+0000 7fe06cd08740 1 -- 172.21.15.96:0/2081073662 wait complete. 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.493+0000 7fe06cd08740 1 -- start start 2024-09-10T22:49:46.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.493+0000 7fe06cd08740 1 Processor -- start 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101efd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ef0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800f790 con 0x562101ebb6a0 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800f0c0 con 0x562101ec5d80 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe04800c300 con 0x562101ed0760 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe06cd08740 10 client.0 _ll_register_callbacks cb 0x7fe0640645a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101efd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe063fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101efd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41754/0 (socket says 172.21.15.96:41754) 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37098/0 (socket says 172.21.15.96:37098) 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe063fff640 1 -- 172.21.15.96:0/2954716391 learned_addr learned my addr 172.21.15.96:0/2954716391 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:46.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe068ed0640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ef0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 -- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 msgr2=0x562101ef0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ef0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 -- 172.21.15.96:0/2954716391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 msgr2=0x562101efd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101efd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.494+0000 7fe0637fe640 1 -- 172.21.15.96:0/2954716391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562101ef1170 con 0x562101ec5d80 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.495+0000 7fe0637fe640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0500e5600 tx=0x7fe0500a9280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d55dd82eb2e7dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.495+0000 7fe0437fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.495+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050020040 con 0x562101ec5d80 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.495+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06409e460 con 0x562101ec5d80 2024-09-10T22:49:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.495+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe050009cb0 con 0x562101ec5d80 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.496+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0500ac470 con 0x562101ec5d80 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 10 client.25481 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562101f55d60 con 0x562101ec5d80 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06409e460 con 0x562101ec5d80 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe050005220 con 0x562101ec5d80 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 10 client.25481 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 10 client.25481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 20 client.25481 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101f563b0 0x562101f587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 4 client.25481 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562101e2f8a0 con 0x562101f563b0 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.497+0000 7fe063fff640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101f563b0 0x562101f587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:46.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe063fff640 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101f563b0 0x562101f587f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e391a49ecf5a6ce1 server_cookie=d6d727963376e1de in_seq=0 out_seq=0 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe0437fe640 10 client.25481 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe0437fe640 1 -- 172.21.15.96:0/2954716391 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562101e2f8a0 con 0x562101f563b0 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe0437fe640 10 client.25481 handle_command_reply: tid=0 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 client.25481 shutdown 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 msgr2=0x562101efd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0500e5600 tx=0x7fe0500a9280 comp rx=0 tx=0).stop 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 shutdown_connections 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x562101f563b0 0x562101f587f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562101ec5d80 0x562101efd7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562101ed0760 0x562101ef0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe0437fe640 0 client.25481 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 --2- 172.21.15.96:0/2954716391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562101ebb6a0 0x562101efd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 >> 172.21.15.96:0/2954716391 conn(0x562101eb94d0 msgr2=0x562101ec6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 shutdown_connections 2024-09-10T22:49:46.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.498+0000 7fe06cd08740 1 -- 172.21.15.96:0/2954716391 wait complete. 2024-09-10T22:49:46.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe06cd08740 20 client.25481 trim_cache size 0 max 16384 2024-09-10T22:49:46.505 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 msgr2=0x7fe0540b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 0x7fe0540b78f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe064052890 tx=0x7fe06407b040 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 msgr2=0x7fe05c098540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 0x7fe05c098540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d6d0 tx=0x7fe05000db90 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 shutdown_connections 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05c095230 0x7fe05c098540 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0540b5430 0x7fe0540b78f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe05c097280 0x7fe05c098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 --2- 172.21.15.96:0/3494247345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe05c0948b0 0x7fe05c098000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.499+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 >> 172.21.15.96:0/3494247345 conn(0x7fe05c00a160 msgr2=0x7fe05c09a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:46.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.500+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 shutdown_connections 2024-09-10T22:49:46.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:46.500+0000 7fe0696d1640 1 -- 172.21.15.96:0/3494247345 wait complete. 2024-09-10T22:49:47.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:47 smithi096 ceph-mon[30869]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-09-10T22:49:47.337 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:49:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 -- 172.21.15.96:0/3027938127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34105b10 msgr2=0x7feb34107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3027938127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34105b10 0x7feb34107f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feb20009860 tx=0x7feb2002a340 comp rx=0 tx=0).stop 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 -- 172.21.15.96:0/3027938127 shutdown_connections 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3027938127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34105b10 0x7feb34107f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3027938127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb341086c0 0x7feb3410ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3027938127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb3406bf90 0x7feb34105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 -- 172.21.15.96:0/3027938127 >> 172.21.15.96:0/3027938127 conn(0x7feb340fd170 msgr2=0x7feb340ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 -- 172.21.15.96:0/3027938127 shutdown_connections 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.475+0000 7feb3b09b640 1 -- 172.21.15.96:0/3027938127 wait complete. 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 Processor -- start 2024-09-10T22:49:47.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 -- start start 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb3406bf90 0x7feb3419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 0x7feb3419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 0x7feb341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb34102930 con 0x7feb34105b10 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb341027b0 con 0x7feb341086c0 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb3b09b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb34102ab0 con 0x7feb3406bf90 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 0x7feb3419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 0x7feb3419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41760/0 (socket says 172.21.15.96:41760) 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb39638640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 0x7feb341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb33fff640 1 -- 172.21.15.96:0/3660129491 learned_addr learned my addr 172.21.15.96:0/3660129491 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.476+0000 7feb38e37640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb3406bf90 0x7feb3419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 -- 172.21.15.96:0/3660129491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb3406bf90 msgr2=0x7feb3419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb3406bf90 0x7feb3419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 -- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 msgr2=0x7feb3419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 0x7feb3419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 -- 172.21.15.96:0/3660129491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb341a0f10 con 0x7feb341086c0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb39638640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 0x7feb341a0650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb31ffb640 1 -- 172.21.15.96:0/3660129491 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00ce20 con 0x7feb341086c0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb31ffb640 1 -- 172.21.15.96:0/3660129491 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c005020 con 0x7feb341086c0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.477+0000 7feb31ffb640 1 -- 172.21.15.96:0/3660129491 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c010690 con 0x7feb341086c0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.478+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341a1200 con 0x7feb341086c0 2024-09-10T22:49:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.478+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a78d0 con 0x7feb341086c0 2024-09-10T22:49:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.481+0000 7feb3de66740 1 Processor -- start 2024-09-10T22:49:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb31ffb640 1 -- 172.21.15.96:0/3660129491 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7feb2c0054d0 con 0x7feb341086c0 2024-09-10T22:49:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 -- start start 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f619ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565f610e6f0 con 0x5565f61a1bd0 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565f60fdb20 con 0x5565f619f190 2024-09-10T22:49:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.482+0000 7feb3de66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565f6101930 con 0x5565f619c860 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb33fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f619ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37124/0 (socket says 172.21.15.96:37124) 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 -- 172.21.15.96:0/2655189408 learned_addr learned my addr 172.21.15.96:0/2655189408 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb33fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f619ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43548/0 (socket says 172.21.15.96:43548) 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb33fff640 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 -- 172.21.15.96:0/2655189408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 msgr2=0x5565f619ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f619ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 -- 172.21.15.96:0/2655189408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 msgr2=0x5565f61a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 -- 172.21.15.96:0/2655189408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f60c3530 con 0x5565f619f190 2024-09-10T22:49:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb33fff640 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb38e37640 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb28009cc0 tx=0x7feb28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8153c50740460f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb31ffb640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 0x7feb240b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb1effd640 1 -- 172.21.15.96:0/2655189408 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28005220 con 0x5565f619f190 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb1effd640 1 -- 172.21.15.96:0/2655189408 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb28004c50 con 0x5565f619f190 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb1effd640 1 -- 172.21.15.96:0/2655189408 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb280197f0 con 0x5565f619f190 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb33fff640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 0x7feb240b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.483+0000 7feb31ffb640 1 -- 172.21.15.96:0/3660129491 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7feb2c0e61f0 con 0x7feb341086c0 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 -- 172.21.15.96:0/2655189408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 msgr2=0x5565f61a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb33fff640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 0x7feb240b78f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb200057c0 tx=0x7feb20030470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb28009cc0 tx=0x7feb28011810 comp rx=0 tx=0).stop 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 -- 172.21.15.96:0/2655189408 shutdown_connections 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61a1580 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 --2- 172.21.15.96:0/2655189408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f619ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 -- 172.21.15.96:0/2655189408 >> 172.21.15.96:0/2655189408 conn(0x5565f61979b0 msgr2=0x5565f6198740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 -- 172.21.15.96:0/2655189408 shutdown_connections 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.484+0000 7feb3de66740 1 -- 172.21.15.96:0/2655189408 wait complete. 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 -- start start 2024-09-10T22:49:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 Processor -- start 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.485+0000 7feb3de66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565f6035ab0 con 0x5565f619f190 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb3de66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565f60a3730 con 0x5565f61a1bd0 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb3de66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565f61a4bb0 con 0x5565f619c860 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41782/0 (socket says 172.21.15.96:41782) 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb33fff640 1 -- 172.21.15.96:0/1534816198 learned_addr learned my addr 172.21.15.96:0/1534816198 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb39638640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb3de66740 10 client.0 _ll_register_callbacks cb 0x5565f60f76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 -- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 msgr2=0x5565f61a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 -- 172.21.15.96:0/1534816198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 msgr2=0x5565f61a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 -- 172.21.15.96:0/1534816198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f5e5f790 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb38e37640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61a5ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feb28009530 tx=0x7feb28005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eac6ff75b25ce39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.486+0000 7feb13fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2804c070 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb28004c50 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7feb28019730 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb28058050 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb3de66740 10 client.25510 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565f5ed8620 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.487+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb28019de0 con 0x5565f619c860 2024-09-10T22:49:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feb280456b0 con 0x5565f619c860 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 10 client.25510 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 20 client.25510 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f61af4c0 0x5565f61b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 4 client.25510 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb28004c50 con 0x5565f61af4c0 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb33fff640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f61af4c0 0x5565f61b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb33fff640 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f61af4c0 0x5565f61b1920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a509cae3b7aa087 server_cookie=5ebc062e99853e5c in_seq=0 out_seq=0 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.488+0000 7feb13fff640 10 client.25510 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb13fff640 1 -- 172.21.15.96:0/1534816198 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7feb28004c50 con 0x5565f61af4c0 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb13fff640 10 client.25510 handle_command_reply: tid=0 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 client.25510 shutdown 2024-09-10T22:49:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 msgr2=0x5565f61a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61a5ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feb28009530 tx=0x7feb28005030 comp rx=0 tx=0).stop 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 shutdown_connections 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f61af4c0 0x5565f61b1920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 --2- 172.21.15.96:0/1534816198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61a5ef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 >> 172.21.15.96:0/1534816198 conn(0x5565f61979b0 msgr2=0x5565f6196bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb13fff640 0 client.25510 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 shutdown_connections 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 1 -- 172.21.15.96:0/1534816198 wait complete. 2024-09-10T22:49:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.489+0000 7feb3de66740 20 client.25510 trim_cache size 0 max 16384 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.514+0000 7feb3de66740 1 Processor -- start 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.514+0000 7feb3de66740 1 -- start start 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f6198920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb341a7b40 con 0x5565f619f190 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb341a9680 con 0x5565f61a1bd0 2024-09-10T22:49:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb3de66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb341a8210 con 0x5565f619c860 2024-09-10T22:49:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37152/0 (socket says 172.21.15.96:37152) 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 -- 172.21.15.96:0/1096738495 learned_addr learned my addr 172.21.15.96:0/1096738495 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb38e37640 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f6198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 -- 172.21.15.96:0/1096738495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 msgr2=0x5565f6198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f6198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 -- 172.21.15.96:0/1096738495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 msgr2=0x5565f61be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.515+0000 7feb39638640 1 -- 172.21.15.96:0/1096738495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f619b970 con 0x5565f61a1bd0 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.516+0000 7feb39638640 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0a8f80 tx=0x7feb2c0a9080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557bddb2184189bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.516+0000 7feb1e7fc640 1 -- 172.21.15.96:0/1096738495 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b1490 con 0x5565f61a1bd0 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.516+0000 7feb1e7fc640 1 -- 172.21.15.96:0/1096738495 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c0b5f40 con 0x5565f61a1bd0 2024-09-10T22:49:47.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.516+0000 7feb1e7fc640 1 -- 172.21.15.96:0/1096738495 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b5d10 con 0x5565f61a1bd0 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 -- 172.21.15.96:0/1096738495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 msgr2=0x5565f61be690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0a8f80 tx=0x7feb2c0a9080 comp rx=0 tx=0).stop 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 -- 172.21.15.96:0/1096738495 shutdown_connections 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f61a1bd0 0x5565f61be690 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f619f190 0x5565f61be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 --2- 172.21.15.96:0/1096738495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f6198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 -- 172.21.15.96:0/1096738495 >> 172.21.15.96:0/1096738495 conn(0x5565f61979b0 msgr2=0x5565f6198210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.517+0000 7feb3de66740 1 -- 172.21.15.96:0/1096738495 shutdown_connections 2024-09-10T22:49:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.518+0000 7feb3de66740 1 -- 172.21.15.96:0/1096738495 wait complete. 2024-09-10T22:49:47.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.518+0000 7feb3de66740 1 -- start start 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.518+0000 7feb3de66740 1 Processor -- start 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb341a8fb0 con 0x5565f61a1bd0 2024-09-10T22:49:47.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb341a88e0 con 0x5565f619f190 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb34117d20 con 0x5565f619c860 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb3de66740 10 client.0 _ll_register_callbacks cb 0x7feb28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43572/0 (socket says 172.21.15.96:43572) 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 -- 172.21.15.96:0/739891670 learned_addr learned my addr 172.21.15.96:0/739891670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb39638640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb33fff640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 -- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 msgr2=0x5565f61ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 -- 172.21.15.96:0/739891670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 msgr2=0x5565f61abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.519+0000 7feb38e37640 1 -- 172.21.15.96:0/739891670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f61ac3f0 con 0x5565f619c860 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.520+0000 7feb38e37640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb28019470 tx=0x7feb28005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1aae060ac39f702 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:47.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.520+0000 7feb13fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.520+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2804c070 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.520+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7feb28004220 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.521+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c0b5f40 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.521+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb28064050 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.521+0000 7feb3de66740 10 client.25513 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.521+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565f621de30 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c0b5f40 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7feb280035f0 con 0x5565f619c860 2024-09-10T22:49:47.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 10 client.25513 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 20 client.25513 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f621e7b0 0x5565f6220bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 4 client.25513 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565f6110640 con 0x5565f621e7b0 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.522+0000 7feb33fff640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f621e7b0 0x5565f6220bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.523+0000 7feb33fff640 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f621e7b0 0x5565f6220bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e5851b8f75bb871 server_cookie=762620e8a9de7fca in_seq=0 out_seq=0 2024-09-10T22:49:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.523+0000 7feb13fff640 10 client.25513 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:47.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:47 smithi117 ceph-mon[35081]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-09-10T22:49:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.546+0000 7feb13fff640 1 -- 172.21.15.96:0/739891670 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565f6110640 con 0x5565f621e7b0 2024-09-10T22:49:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.546+0000 7feb13fff640 10 client.25513 handle_command_reply: tid=0 2024-09-10T22:49:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.546+0000 7feb3de66740 1 client.25513 shutdown 2024-09-10T22:49:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 msgr2=0x5565f61ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb28019470 tx=0x7feb28005660 comp rx=0 tx=0).stop 2024-09-10T22:49:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 shutdown_connections 2024-09-10T22:49:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565f621e7b0 0x5565f6220bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565f619f190 0x5565f61ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565f61a1bd0 0x5565f61abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 --2- 172.21.15.96:0/739891670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565f619c860 0x5565f61ad3e0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb28019470 tx=0x7feb28005660 comp rx=0 tx=0).stop 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 >> 172.21.15.96:0/739891670 conn(0x5565f61979b0 msgr2=0x5565f61963d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.547+0000 7feb13fff640 0 client.25513 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.548+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 shutdown_connections 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.548+0000 7feb3de66740 1 -- 172.21.15.96:0/739891670 wait complete. 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.548+0000 7feb3de66740 20 client.25513 trim_cache size 0 max 16384 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:49:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 msgr2=0x7feb240b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 0x7feb240b78f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb200057c0 tx=0x7feb20030470 comp rx=0 tx=0).stop 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 msgr2=0x7feb341a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 0x7feb341a0650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00ef10 comp rx=0 tx=0).stop 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 shutdown_connections 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb341086c0 0x7feb341a0650 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7feb240b5430 0x7feb240b78f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb34105b10 0x7feb3419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 --2- 172.21.15.96:0/3660129491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb3406bf90 0x7feb3419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 >> 172.21.15.96:0/3660129491 conn(0x7feb340fd170 msgr2=0x7feb340ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 shutdown_connections 2024-09-10T22:49:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:49:47.549+0000 7feb3b09b640 1 -- 172.21.15.96:0/3660129491 wait complete. 2024-09-10T22:49:47.715 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:47 smithi121 ceph-mon[35119]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 889 op/s 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.502+0000 7fcc7f699640 1 -- 172.21.15.96:0/2609811176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78075e00 msgr2=0x7fcc7810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.502+0000 7fcc7f699640 1 --2- 172.21.15.96:0/2609811176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78075e00 0x7fcc7810ee90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009900 tx=0x7fcc6802a410 comp rx=0 tx=0).stop 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 -- 172.21.15.96:0/2609811176 shutdown_connections 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 --2- 172.21.15.96:0/2609811176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc7810f3d0 0x7fcc781117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 --2- 172.21.15.96:0/2609811176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78075e00 0x7fcc7810ee90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 --2- 172.21.15.96:0/2609811176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc780758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 -- 172.21.15.96:0/2609811176 >> 172.21.15.96:0/2609811176 conn(0x7fcc780fe830 msgr2=0x7fcc78100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.503+0000 7fcc7f699640 1 -- 172.21.15.96:0/2609811176 shutdown_connections 2024-09-10T22:49:48.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.504+0000 7fcc7f699640 1 -- 172.21.15.96:0/2609811176 wait complete. 2024-09-10T22:49:48.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.504+0000 7fcc7f699640 1 Processor -- start 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.504+0000 7fcc7f699640 1 -- start start 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 0x7fcc7819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7810f3d0 0x7fcc781a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78114f80 con 0x7fcc7810f3d0 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78114e00 con 0x7fcc78075e00 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7f699640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78115100 con 0x7fcc780754e0 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7cc34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 0x7fcc7819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7cc34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 0x7fcc7819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37188/0 (socket says 172.21.15.96:37188) 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43596/0 (socket says 172.21.15.96:43596) 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7cc34640 1 -- 172.21.15.96:0/473394127 learned_addr learned my addr 172.21.15.96:0/473394127 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:48.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7810f3d0 0x7fcc781a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 -- 172.21.15.96:0/473394127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 msgr2=0x7fcc7819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 0x7fcc7819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 -- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7810f3d0 msgr2=0x7fcc781a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7810f3d0 0x7fcc781a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.505+0000 7fcc7d435640 1 -- 172.21.15.96:0/473394127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc781a15a0 con 0x7fcc780754e0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.506+0000 7fcc7d435640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d6d0 tx=0x7fcc7400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.506+0000 7fcc6e7fc640 1 -- 172.21.15.96:0/473394127 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74017070 con 0x7fcc780754e0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.506+0000 7fcc6e7fc640 1 -- 172.21.15.96:0/473394127 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc74009820 con 0x7fcc780754e0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.506+0000 7fcc6e7fc640 1 -- 172.21.15.96:0/473394127 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74004c40 con 0x7fcc780754e0 2024-09-10T22:49:48.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.506+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc781a1890 con 0x7fcc780754e0 2024-09-10T22:49:48.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.507+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc78102a20 con 0x7fcc780754e0 2024-09-10T22:49:48.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.509+0000 7fcc6e7fc640 1 -- 172.21.15.96:0/473394127 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fcc74010460 con 0x7fcc780754e0 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 Processor -- start 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc6e7fc640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 0x7fcc640b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 -- start start 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc6e7fc640 1 -- 172.21.15.96:0/473394127 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcc740e5540 con 0x7fcc780754e0 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894b4d0 0x556df894b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 0x556df8957840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df895a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556df88c1ea0 con 0x556df894bdf0 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556df88b12d0 con 0x556df894b4d0 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556df88b50e0 con 0x556df8957d80 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.510+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 0x7fcc640b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894b4d0 0x556df894b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7cc34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 0x556df8957840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7cc34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 0x556df8957840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41820/0 (socket says 172.21.15.96:41820) 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7cc34640 1 -- 172.21.15.96:0/676371038 learned_addr learned my addr 172.21.15.96:0/676371038 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df895a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 -- 172.21.15.96:0/676371038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894b4d0 msgr2=0x556df894b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894b4d0 0x556df894b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 -- 172.21.15.96:0/676371038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 msgr2=0x556df8957840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 0x556df8957840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.511+0000 7fcc7dc36640 1 -- 172.21.15.96:0/676371038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df860f650 con 0x556df8957d80 2024-09-10T22:49:48.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.512+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 0x7fcc640b78f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc780764c0 tx=0x7fcc70010a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.513+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df895a170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005bd0 tx=0x7fcc70022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2737002c7afd7de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.513+0000 7fcc637fe640 1 -- 172.21.15.96:0/676371038 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7001b040 con 0x556df8957d80 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.513+0000 7fcc637fe640 1 -- 172.21.15.96:0/676371038 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc70021bf0 con 0x556df8957d80 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.513+0000 7fcc637fe640 1 -- 172.21.15.96:0/676371038 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002a300 con 0x556df8957d80 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 -- 172.21.15.96:0/676371038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 msgr2=0x556df895a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df895a170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005bd0 tx=0x7fcc70022000 comp rx=0 tx=0).stop 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 -- 172.21.15.96:0/676371038 shutdown_connections 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df895a170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894bdf0 0x556df8957840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.514+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/676371038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894b4d0 0x556df894b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.515+0000 7fcc81a6e740 1 -- 172.21.15.96:0/676371038 >> 172.21.15.96:0/676371038 conn(0x556df8949300 msgr2=0x556df894a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.515+0000 7fcc81a6e740 1 -- 172.21.15.96:0/676371038 shutdown_connections 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.515+0000 7fcc81a6e740 1 -- 172.21.15.96:0/676371038 wait complete. 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.515+0000 7fcc81a6e740 1 -- start start 2024-09-10T22:49:48.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.515+0000 7fcc81a6e740 1 Processor -- start 2024-09-10T22:49:48.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8954b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df8955220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556df8712700 con 0x556df894b4d0 2024-09-10T22:49:48.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556df884c160 con 0x556df894bdf0 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556df895a9a0 con 0x556df8957d80 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc81a6e740 10 client.0 _ll_register_callbacks cb 0x556df88659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41828/0 (socket says 172.21.15.96:41828) 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 -- 172.21.15.96:0/3821920711 learned_addr learned my addr 172.21.15.96:0/3821920711 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8954b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df8955220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 -- 172.21.15.96:0/3821920711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 msgr2=0x556df8955220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df8955220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 -- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 msgr2=0x556df8954b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8954b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 -- 172.21.15.96:0/3821920711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df8879c90 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.516+0000 7fcc7d435640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcc740e2fb0 tx=0x7fcc740aba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400df02625ec2549 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74098020 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcc740ab2b0 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc70021bf0 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc74020070 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc81a6e740 10 client.15942 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.517+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556df8966410 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc740981c0 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcc740b3b60 con 0x556df894b4d0 2024-09-10T22:49:48.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 10 client.15942 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 20 client.15942 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89677d0 0x556df8969bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 4 client.15942 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.518+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc70021bf0 con 0x556df89677d0 2024-09-10T22:49:48.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.519+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89677d0 0x556df8969bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.519+0000 7fcc60ff9640 10 client.15942 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:48.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.519+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89677d0 0x556df8969bf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca306361350d001 server_cookie=f60ec457aec197f8 in_seq=0 out_seq=0 2024-09-10T22:49:48.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.520+0000 7fcc60ff9640 1 -- 172.21.15.96:0/3821920711 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcc70021bf0 con 0x556df89677d0 2024-09-10T22:49:48.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.520+0000 7fcc60ff9640 10 client.15942 handle_command_reply: tid=0 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 client.15942 shutdown 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 msgr2=0x556df8957720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcc740e2fb0 tx=0x7fcc740aba10 comp rx=0 tx=0).stop 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 shutdown_connections 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89677d0 0x556df8969bf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8954b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df8957d80 0x556df8955220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/3821920711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df8957720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 >> 172.21.15.96:0/3821920711 conn(0x556df8949300 msgr2=0x556df894ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc60ff9640 0 client.15942 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.521+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 shutdown_connections 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.522+0000 7fcc81a6e740 1 -- 172.21.15.96:0/3821920711 wait complete. 2024-09-10T22:49:48.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.522+0000 7fcc81a6e740 20 client.15942 trim_cache size 0 max 16384 2024-09-10T22:49:48.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 Processor -- start 2024-09-10T22:49:48.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 -- start start 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df894f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 0x556df8963c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00e3f0 con 0x556df894b4d0 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00ff30 con 0x556df8957d80 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.538+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00eac0 con 0x556df894bdf0 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37214/0 (socket says 172.21.15.96:37214) 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 -- 172.21.15.96:0/1562970451 learned_addr learned my addr 172.21.15.96:0/1562970451 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7d435640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df894f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 0x556df8963c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 -- 172.21.15.96:0/1562970451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 msgr2=0x556df8963c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 0x556df8963c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 -- 172.21.15.96:0/1562970451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 msgr2=0x556df894f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df894f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 -- 172.21.15.96:0/1562970451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df8957350 con 0x556df8957d80 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 0x556df8963c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:48.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcc700121b0 tx=0x7fcc700121e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc60429656245ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc62ffd640 1 -- 172.21.15.96:0/1562970451 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7003a070 con 0x556df8957d80 2024-09-10T22:49:48.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc62ffd640 1 -- 172.21.15.96:0/1562970451 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc70010e50 con 0x556df8957d80 2024-09-10T22:49:48.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.539+0000 7fcc62ffd640 1 -- 172.21.15.96:0/1562970451 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002c660 con 0x556df8957d80 2024-09-10T22:49:48.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.540+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1562970451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 msgr2=0x556df8959170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.540+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcc700121b0 tx=0x7fcc700121e0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1562970451 shutdown_connections 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df8957d80 0x556df8959170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894bdf0 0x556df8963c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1562970451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df894b4d0 0x556df894f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1562970451 >> 172.21.15.96:0/1562970451 conn(0x556df8949300 msgr2=0x556df894fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1562970451 shutdown_connections 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1562970451 wait complete. 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.541+0000 7fcc81a6e740 1 -- start start 2024-09-10T22:49:48.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 Processor -- start 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8960210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 0x556df895ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00f860 con 0x556df8957d80 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00f190 con 0x556df894bdf0 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c00c3d0 con 0x556df894b4d0 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.542+0000 7fcc81a6e740 10 client.0 _ll_register_callbacks cb 0x7fcc740046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:48.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43634/0 (socket says 172.21.15.96:43634) 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 -- 172.21.15.96:0/1402560598 learned_addr learned my addr 172.21.15.96:0/1402560598 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 0x556df895ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 -- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 msgr2=0x556df8960210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8960210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 -- 172.21.15.96:0/1402560598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 msgr2=0x556df895ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 0x556df895ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 -- 172.21.15.96:0/1402560598 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df8960850 con 0x556df894b4d0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7dc36640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 0x556df895ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc7d435640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b3d90 tx=0x7fcc740b3dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2defc69cfa2ffe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.543+0000 7fcc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74098040 con 0x556df894b4d0 2024-09-10T22:49:48.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcc740a8740 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc70010e50 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc740b19c0 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 10 client.25528 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556df89d4aa0 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc70010e50 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcc740a8940 con 0x556df894b4d0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 10 client.25528 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 20 client.25528 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89d5280 0x556df89d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 4 client.25528 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.544+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556df88c3df0 con 0x556df89d5280 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89d5280 0x556df89d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc60ff9640 10 client.25528 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc7cc34640 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89d5280 0x556df89d76a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2cd6160084dc97 server_cookie=a735ede56e55ed8c in_seq=0 out_seq=0 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc60ff9640 1 -- 172.21.15.96:0/1402560598 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556df88c3df0 con 0x556df89d5280 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc60ff9640 10 client.25528 handle_command_reply: tid=0 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 client.25528 shutdown 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 msgr2=0x556df895fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b3d90 tx=0x7fcc740b3dc0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 shutdown_connections 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556df89d5280 0x556df89d76a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556df894bdf0 0x556df8960210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556df8957d80 0x556df895ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 --2- 172.21.15.96:0/1402560598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556df894b4d0 0x556df895fcd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 >> 172.21.15.96:0/1402560598 conn(0x556df8949300 msgr2=0x556df89561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.545+0000 7fcc60ff9640 0 client.25528 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:48.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.546+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 shutdown_connections 2024-09-10T22:49:48.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.546+0000 7fcc81a6e740 1 -- 172.21.15.96:0/1402560598 wait complete. 2024-09-10T22:49:48.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.546+0000 7fcc81a6e740 20 client.25528 trim_cache size 0 max 16384 2024-09-10T22:49:48.553 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 msgr2=0x7fcc640b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 0x7fcc640b78f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc780764c0 tx=0x7fcc70010a40 comp rx=0 tx=0).stop 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 msgr2=0x7fcc7819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d6d0 tx=0x7fcc7400db90 comp rx=0 tx=0).stop 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 shutdown_connections 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc78075e00 0x7fcc7819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fcc640b5430 0x7fcc640b78f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc7810f3d0 0x7fcc781a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 --2- 172.21.15.96:0/473394127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc780754e0 0x7fcc7819b4f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.547+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 >> 172.21.15.96:0/473394127 conn(0x7fcc780fe830 msgr2=0x7fcc780ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.548+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 shutdown_connections 2024-09-10T22:49:48.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:48.548+0000 7fcc7f699640 1 -- 172.21.15.96:0/473394127 wait complete. 2024-09-10T22:49:49.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:49 smithi117 ceph-mon[35081]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 736 op/s 2024-09-10T22:49:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:49 smithi096 ceph-mon[30869]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 736 op/s 2024-09-10T22:49:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:49 smithi121 ceph-mon[35119]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 736 op/s 2024-09-10T22:49:50.363 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:49:50.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:49:50.711 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 135.0 s, 3342.4 tps, lat 29.903 ms stddev 35.078 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.266+0000 7f10374ae640 1 -- 172.21.15.96:0/1721309428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a150 msgr2=0x7f10301111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.266+0000 7f10374ae640 1 --2- 172.21.15.96:0/1721309428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a150 0x7f10301111b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02a480 comp rx=0 tx=0).stop 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.267+0000 7f10374ae640 1 -- 172.21.15.96:0/1721309428 shutdown_connections 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.267+0000 7f10374ae640 1 --2- 172.21.15.96:0/1721309428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a530 0x7f103010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.267+0000 7f10374ae640 1 --2- 172.21.15.96:0/1721309428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 0x7f103011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.267+0000 7f10374ae640 1 --2- 172.21.15.96:0/1721309428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a150 0x7f10301111b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.267+0000 7f10374ae640 1 -- 172.21.15.96:0/1721309428 >> 172.21.15.96:0/1721309428 conn(0x7f103006dfc0 msgr2=0x7f103006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.268+0000 7f10374ae640 1 -- 172.21.15.96:0/1721309428 shutdown_connections 2024-09-10T22:49:51.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.268+0000 7f10374ae640 1 -- 172.21.15.96:0/1721309428 wait complete. 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 Processor -- start 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 -- start start 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a150 0x7f10301a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 0x7f10301ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1030121b60 con 0x7f103011c0a0 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10301219e0 con 0x7f103010a150 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.269+0000 7f10374ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1030121ce0 con 0x7f103010a530 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43642/0 (socket says 172.21.15.96:43642) 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 -- 172.21.15.96:0/3499603364 learned_addr learned my addr 172.21.15.96:0/3499603364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f103524a640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a150 0x7f10301a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1035a4b640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 0x7f10301ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 -- 172.21.15.96:0/3499603364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a150 msgr2=0x7f10301a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a150 0x7f10301a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 -- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 msgr2=0x7f10301ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 0x7f10301ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 -- 172.21.15.96:0/3499603364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10301ae140 con 0x7f103010a530 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.270+0000 7f1034a49640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.271+0000 7f10267fc640 1 -- 172.21.15.96:0/3499603364 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102000cd80 con 0x7f103010a530 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.271+0000 7f10267fc640 1 -- 172.21.15.96:0/3499603364 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10200054d0 con 0x7f103010a530 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.271+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10301ae430 con 0x7f103010a530 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.271+0000 7f10267fc640 1 -- 172.21.15.96:0/3499603364 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020010690 con 0x7f103010a530 2024-09-10T22:49:51.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.271+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10301b4af0 con 0x7f103010a530 2024-09-10T22:49:51.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.275+0000 7f1039883740 1 Processor -- start 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.275+0000 7f1039883740 1 -- start start 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 0x5573b5dd3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573b5d3be60 con 0x5573b5dc4ee0 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5573b5d2b290 con 0x5573b5dc5800 2024-09-10T22:49:51.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1039883740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5573b5d2f0a0 con 0x5573b5dd1790 2024-09-10T22:49:51.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1035a4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f103524a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1035a4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41890/0 (socket says 172.21.15.96:41890) 2024-09-10T22:49:51.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f1035a4b640 1 -- 172.21.15.96:0/2209814685 learned_addr learned my addr 172.21.15.96:0/2209814685 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f103524a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37250/0 (socket says 172.21.15.96:37250) 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.276+0000 7f10267fc640 1 -- 172.21.15.96:0/3499603364 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1020004e90 con 0x7f103010a530 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1034a49640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 0x5573b5dd3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 -- 172.21.15.96:0/2209814685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 msgr2=0x5573b5dd3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 0x5573b5dd3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 -- 172.21.15.96:0/2209814685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 msgr2=0x5573b5dd1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 -- 172.21.15.96:0/2209814685 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b5a89930 con 0x5573b5dc4ee0 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f103524a640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1034a49640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 0x5573b5dd3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:51.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f1035a4b640 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1028011440 tx=0x7f1028011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13db15c8597a3b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f101b7fe640 1 -- 172.21.15.96:0/2209814685 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028004d20 con 0x5573b5dc4ee0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f10267fc640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 0x7f101c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f101b7fe640 1 -- 172.21.15.96:0/2209814685 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1028004050 con 0x5573b5dc4ee0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.277+0000 7f101b7fe640 1 -- 172.21.15.96:0/2209814685 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102800f780 con 0x5573b5dc4ee0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f103524a640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 0x7f101c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f10267fc640 1 -- 172.21.15.96:0/3499603364 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f10200e7230 con 0x7f103010a530 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 -- 172.21.15.96:0/2209814685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 msgr2=0x5573b5dc52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1028011440 tx=0x7f1028011900 comp rx=0 tx=0).stop 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f103524a640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 0x7f101c0b7820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f102c031440 tx=0x7f102c032c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 -- 172.21.15.96:0/2209814685 shutdown_connections 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc5800 0x5573b5dd1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd1790 0x5573b5dd3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 --2- 172.21.15.96:0/2209814685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc4ee0 0x5573b5dc52c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 -- 172.21.15.96:0/2209814685 >> 172.21.15.96:0/2209814685 conn(0x5573b5dc2d10 msgr2=0x5573b5dc3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 -- 172.21.15.96:0/2209814685 shutdown_connections 2024-09-10T22:49:51.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.278+0000 7f1039883740 1 -- 172.21.15.96:0/2209814685 wait complete. 2024-09-10T22:49:51.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 -- start start 2024-09-10T22:49:51.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 Processor -- start 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.279+0000 7f1039883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573b5b8c890 con 0x5573b5dd1790 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1039883740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5573b5cc5d40 con 0x5573b5dc4ee0 2024-09-10T22:49:51.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1039883740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5573b5dd4440 con 0x5573b5dc5800 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1039883740 10 client.0 _ll_register_callbacks cb 0x5573b5d29c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43662/0 (socket says 172.21.15.96:43662) 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 -- 172.21.15.96:0/1014184275 learned_addr learned my addr 172.21.15.96:0/1014184275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f103524a640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1035a4b640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 -- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 msgr2=0x5573b5dce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 -- 172.21.15.96:0/1014184275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 msgr2=0x5573b5dcf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.280+0000 7f1034a49640 1 -- 172.21.15.96:0/1014184275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b5ddd870 con 0x5573b5dc5800 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1034a49640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10200ac520 tx=0x7f10200048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3acd2d00002446 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:51.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200d3020 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f10200b5b20 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1028004050 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10200ac330 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 10 client.25537 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573b5de01f0 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1028004050 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f10200cfb70 con 0x5573b5dc5800 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 10 client.25537 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 10 client.25537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 20 client.25537 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5de16b0 0x5573b5de3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 4 client.25537 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.281+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b5d3ddb0 con 0x5573b5de16b0 2024-09-10T22:49:51.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.282+0000 7f103524a640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5de16b0 0x5573b5de3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.283+0000 7f103524a640 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5de16b0 0x5573b5de3aa0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e135abef93f6ac server_cookie=d8d2703cdc471787 in_seq=0 out_seq=0 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1018ff9640 10 client.25537 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1018ff9640 1 -- 172.21.15.96:0/1014184275 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573b5d3ddb0 con 0x5573b5de16b0 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1018ff9640 10 client.25537 handle_command_reply: tid=0 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 client.25537 shutdown 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 msgr2=0x5573b5dcece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10200ac520 tx=0x7f10200048f0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 shutdown_connections 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5de16b0 0x5573b5de3aa0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcece0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 --2- 172.21.15.96:0/1014184275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1018ff9640 0 client.25537 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:51.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.284+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 >> 172.21.15.96:0/1014184275 conn(0x5573b5dc2d10 msgr2=0x5573b5dc1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.285+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 shutdown_connections 2024-09-10T22:49:51.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.285+0000 7f1039883740 1 -- 172.21.15.96:0/1014184275 wait complete. 2024-09-10T22:49:51.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.285+0000 7f1039883740 20 client.25537 trim_cache size 0 max 16384 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 Processor -- start 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 -- start start 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10301bd0f0 con 0x5573b5dd1790 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10301bec30 con 0x5573b5dc4ee0 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1039883740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10301bd7c0 con 0x5573b5dc5800 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1035a4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1034a49640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37268/0 (socket says 172.21.15.96:37268) 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 -- 172.21.15.96:0/640028892 learned_addr learned my addr 172.21.15.96:0/640028892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 -- 172.21.15.96:0/640028892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 msgr2=0x5573b5dcb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 -- 172.21.15.96:0/640028892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 msgr2=0x5573b5dcbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f103524a640 1 -- 172.21.15.96:0/640028892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b5dc5630 con 0x5573b5dc4ee0 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.307+0000 7f1034a49640 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.308+0000 7f103524a640 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f102c02a9a0 tx=0x7f102c03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854d8d43b7b6b44a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.308+0000 7f101affd640 1 -- 172.21.15.96:0/640028892 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c035040 con 0x5573b5dc4ee0 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.308+0000 7f101affd640 1 -- 172.21.15.96:0/640028892 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f102c0054d0 con 0x5573b5dc4ee0 2024-09-10T22:49:51.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.308+0000 7f101affd640 1 -- 172.21.15.96:0/640028892 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c004bc0 con 0x5573b5dc4ee0 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 -- 172.21.15.96:0/640028892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 msgr2=0x5573b5dd04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f102c02a9a0 tx=0x7f102c03e000 comp rx=0 tx=0).stop 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 -- 172.21.15.96:0/640028892 shutdown_connections 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dd1790 0x5573b5dcbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dc5800 0x5573b5dcb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 --2- 172.21.15.96:0/640028892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dc4ee0 0x5573b5dd04e0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f102c02a9a0 tx=0x7f102c03e000 comp rx=0 tx=0).stop 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.309+0000 7f1039883740 1 -- 172.21.15.96:0/640028892 >> 172.21.15.96:0/640028892 conn(0x5573b5dc2d10 msgr2=0x5573b5dcfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.310+0000 7f1039883740 1 -- 172.21.15.96:0/640028892 shutdown_connections 2024-09-10T22:49:51.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.310+0000 7f1039883740 1 -- 172.21.15.96:0/640028892 wait complete. 2024-09-10T22:49:51.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 -- start start 2024-09-10T22:49:51.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 Processor -- start 2024-09-10T22:49:51.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dd1790 0x5573b5dda0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 0x5573b5dd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10301be560 con 0x5573b5dc5800 2024-09-10T22:49:51.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10301bde90 con 0x5573b5dd1790 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10301bb0d0 con 0x5573b5dd61e0 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1039883740 10 client.0 _ll_register_callbacks cb 0x7f102000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f103524a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f103524a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41920/0 (socket says 172.21.15.96:41920) 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f103524a640 1 -- 172.21.15.96:0/1235480886 learned_addr learned my addr 172.21.15.96:0/1235480886 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1035a4b640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 0x5573b5dd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.311+0000 7f1034a49640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dd1790 0x5573b5dda0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 -- 172.21.15.96:0/1235480886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 msgr2=0x5573b5dd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 0x5573b5dd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 -- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dd1790 msgr2=0x5573b5dda0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dd1790 0x5573b5dda0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 -- 172.21.15.96:0/1235480886 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b5dd6e90 con 0x5573b5dc5800 2024-09-10T22:49:51.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1035a4b640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 0x5573b5dd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f103524a640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f102c0057c0 tx=0x7f102c03c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbeefd7e1ee453f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c035050 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f102c03cd50 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c0054d0 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f102c068440 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1039883740 10 client.15966 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573b5e4f6c0 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.312+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f102c0054d0 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f102c058030 con 0x5573b5dc5800 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 10 client.15966 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:51.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 20 client.15966 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5e4fe60 0x5573b5e52280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 4 client.15966 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573b5d39860 con 0x5573b5e4fe60 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1034a49640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5e4fe60 0x5573b5e52280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1018ff9640 10 client.15966 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:51.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.313+0000 7f1034a49640 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5e4fe60 0x5573b5e52280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cc38b29a0f146d server_cookie=90d075c70889409c in_seq=0 out_seq=0 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1018ff9640 1 -- 172.21.15.96:0/1235480886 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573b5d39860 con 0x5573b5e4fe60 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1018ff9640 10 client.15966 handle_command_reply: tid=0 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 client.15966 shutdown 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 msgr2=0x5573b5dd9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f102c0057c0 tx=0x7f102c03c7d0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 shutdown_connections 2024-09-10T22:49:51.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5573b5e4fe60 0x5573b5e52280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b5dd1790 0x5573b5dda0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573b5dd61e0 0x5573b5dd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 --2- 172.21.15.96:0/1235480886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573b5dc5800 0x5573b5dd9b70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 >> 172.21.15.96:0/1235480886 conn(0x5573b5dc2d10 msgr2=0x5573b5dc1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1018ff9640 0 client.15966 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 shutdown_connections 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.315+0000 7f1039883740 1 -- 172.21.15.96:0/1235480886 wait complete. 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f1039883740 20 client.15966 trim_cache size 0 max 16384 2024-09-10T22:49:51.320 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "a9b1e2a5-38f5-4899-a8a8-92c463641aa7", 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 msgr2=0x7f101c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 0x7f101c0b7820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f102c031440 tx=0x7f102c032c00 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 msgr2=0x7f10301a85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9f0 tx=0x7f102000edf0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 shutdown_connections 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f101c0b5360 0x7f101c0b7820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f103011c0a0 0x7f10301ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f103010a530 0x7f10301a85d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 --2- 172.21.15.96:0/3499603364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f103010a150 0x7f10301a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 >> 172.21.15.96:0/3499603364 conn(0x7f103006dfc0 msgr2=0x7f103011c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 shutdown_connections 2024-09-10T22:49:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:51.316+0000 7f10374ae640 1 -- 172.21.15.96:0/3499603364 wait complete. 2024-09-10T22:49:51.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:51 smithi096 ceph-mon[30869]: pgmap v423: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T22:49:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:51 smithi117 ceph-mon[35081]: pgmap v423: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T22:49:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:51 smithi121 ceph-mon[35119]: pgmap v423: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T22:49:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:52 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:49:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:52 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:52.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:52 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:49:52.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:52 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:52 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:49:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:52 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:52.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:49:53.116 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:49:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:53 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:49:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:53 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:53 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:49:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:53 smithi096 ceph-mon[30869]: pgmap v424: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-10T22:49:53.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:53 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:53 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:49:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:53 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:53 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:49:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:53 smithi121 ceph-mon[35119]: pgmap v424: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-10T22:49:53.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:53 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:53 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:49:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:53 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:53 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:49:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:53 smithi117 ceph-mon[35081]: pgmap v424: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 719 op/s 2024-09-10T22:49:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:53 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:49:54.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 -- 172.21.15.96:0/2197826835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c108020 msgr2=0x7f601c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 --2- 172.21.15.96:0/2197826835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c108020 0x7f601c10e820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6014009be0 tx=0x7f601402a6c0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 -- 172.21.15.96:0/2197826835 shutdown_connections 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 --2- 172.21.15.96:0/2197826835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 --2- 172.21.15.96:0/2197826835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c108020 0x7f601c10e820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.246+0000 7f6022d1e640 1 --2- 172.21.15.96:0/2197826835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c101c90 0x7f601c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- 172.21.15.96:0/2197826835 >> 172.21.15.96:0/2197826835 conn(0x7f601c0fd700 msgr2=0x7f601c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- 172.21.15.96:0/2197826835 shutdown_connections 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- 172.21.15.96:0/2197826835 wait complete. 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 Processor -- start 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- start start 2024-09-10T22:49:54.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c108020 0x7f601c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f601c111ce0 con 0x7f601c108020 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f601c111b60 con 0x7f601c1076a0 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.247+0000 7f6022d1e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f601c111e60 con 0x7f601c101c90 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43684/0 (socket says 172.21.15.96:43684) 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 -- 172.21.15.96:0/4006365025 learned_addr learned my addr 172.21.15.96:0/4006365025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f60212bb640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c108020 0x7f601c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f601bfff640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 -- 172.21.15.96:0/4006365025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 msgr2=0x7f601c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 -- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c108020 msgr2=0x7f601c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c108020 0x7f601c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 -- 172.21.15.96:0/4006365025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c19e240 con 0x7f601c101c90 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f601bfff640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6020aba640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6019ffb640 1 -- 172.21.15.96:0/4006365025 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000ccc0 con 0x7f601c101c90 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6019ffb640 1 -- 172.21.15.96:0/4006365025 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60100054d0 con 0x7f601c101c90 2024-09-10T22:49:54.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c19e530 con 0x7f601c101c90 2024-09-10T22:49:54.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.248+0000 7f6019ffb640 1 -- 172.21.15.96:0/4006365025 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010010430 con 0x7f601c101c90 2024-09-10T22:49:54.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.249+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c06ad00 con 0x7f601c101c90 2024-09-10T22:49:54.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.253+0000 7f6019ffb640 1 -- 172.21.15.96:0/4006365025 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f6010004f80 con 0x7f601c101c90 2024-09-10T22:49:54.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.254+0000 7f60250f3740 1 Processor -- start 2024-09-10T22:49:54.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.254+0000 7f6019ffb640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 0x7f600c0b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.254+0000 7f60250f3740 1 -- start start 2024-09-10T22:49:54.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6019ffb640 1 -- 172.21.15.96:0/4006365025 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f60100e5f20 con 0x7f601c101c90 2024-09-10T22:49:54.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60212bb640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 0x7f600c0b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c2629410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c262bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5586c2593e50 con 0x5586c261d9c0 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586c2583280 con 0x5586c2629950 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60250f3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5586c2587090 con 0x5586c261d0a0 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43698/0 (socket says 172.21.15.96:43698) 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 -- 172.21.15.96:0/1983023471 learned_addr learned my addr 172.21.15.96:0/1983023471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:54.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f601bfff640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c2629410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f60212bb640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c262bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 -- 172.21.15.96:0/1983023471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 msgr2=0x5586c262bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c262bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 -- 172.21.15.96:0/1983023471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 msgr2=0x5586c2629410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c2629410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f6020aba640 1 -- 172.21.15.96:0/1983023471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c22e1650 con 0x5586c261d0a0 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.255+0000 7f601bfff640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c2629410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.256+0000 7f6020aba640 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60100e3a60 tx=0x7f6010009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe8302c89c65cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.256+0000 7f6006ffd640 1 -- 172.21.15.96:0/1983023471 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100ac290 con 0x5586c261d0a0 2024-09-10T22:49:54.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.256+0000 7f6006ffd640 1 -- 172.21.15.96:0/1983023471 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60100ace80 con 0x5586c261d0a0 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.257+0000 7f6006ffd640 1 -- 172.21.15.96:0/1983023471 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100a80d0 con 0x5586c261d0a0 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- 172.21.15.96:0/1983023471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 msgr2=0x5586c261d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60100e3a60 tx=0x7f6010009f90 comp rx=0 tx=0).stop 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60212bb640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 0x7f600c0b9a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6014004af0 tx=0x7f60140305e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- 172.21.15.96:0/1983023471 shutdown_connections 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c262bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c2629410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 --2- 172.21.15.96:0/1983023471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c261d480 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- 172.21.15.96:0/1983023471 >> 172.21.15.96:0/1983023471 conn(0x5586c261aed0 msgr2=0x5586c261c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- 172.21.15.96:0/1983023471 shutdown_connections 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- 172.21.15.96:0/1983023471 wait complete. 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 -- start start 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.258+0000 7f60250f3740 1 Processor -- start 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2629180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2626b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5586c23e4700 con 0x5586c2629950 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586c251e110 con 0x5586c261d0a0 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f60250f3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5586c262c670 con 0x5586c261d9c0 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f601bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f601bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43710/0 (socket says 172.21.15.96:43710) 2024-09-10T22:49:54.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.259+0000 7f601bfff640 1 -- 172.21.15.96:0/3035703469 learned_addr learned my addr 172.21.15.96:0/3035703469 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:54.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f6020aba640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2629180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f60250f3740 10 client.0 _ll_register_callbacks cb 0x5586c2537990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:54.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 -- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 msgr2=0x5586c2629180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2629180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 -- 172.21.15.96:0/3035703469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 msgr2=0x5586c2626b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2626b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 -- 172.21.15.96:0/3035703469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c254bc40 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f6020aba640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2629180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.260+0000 7f601bfff640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f600800e470 tx=0x7f600800e930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e88138f555a73ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008017020 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6008013610 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60100ace80 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f600800f040 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 10 client.25555 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586c2637ee0 con 0x5586c261d9c0 2024-09-10T22:49:54.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60100ace80 con 0x5586c261d9c0 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6008012050 con 0x5586c261d9c0 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 10 client.25555 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 20 client.25555 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.261+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c2639490 0x5586c263b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.262+0000 7f60250f3740 4 client.25555 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.262+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586c2595da0 con 0x5586c2639490 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.262+0000 7f6020aba640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c2639490 0x5586c263b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.262+0000 7f5ffbfff640 10 client.25555 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.262+0000 7f6020aba640 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c2639490 0x5586c263b8e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6412f6aea1f3440 server_cookie=21058c94628b03f3 in_seq=0 out_seq=0 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f5ffbfff640 1 -- 172.21.15.96:0/3035703469 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586c2595da0 con 0x5586c2639490 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f5ffbfff640 10 client.25555 handle_command_reply: tid=0 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 client.25555 shutdown 2024-09-10T22:49:54.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 msgr2=0x5586c26265d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f600800e470 tx=0x7f600800e930 comp rx=0 tx=0).stop 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 shutdown_connections 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c2639490 0x5586c263b8e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2626b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c26265d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 --2- 172.21.15.96:0/3035703469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2629180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 >> 172.21.15.96:0/3035703469 conn(0x5586c261aed0 msgr2=0x5586c2620a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f5ffbfff640 0 client.25555 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 shutdown_connections 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 1 -- 172.21.15.96:0/3035703469 wait complete. 2024-09-10T22:49:54.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.263+0000 7f60250f3740 20 client.25555 trim_cache size 0 max 16384 2024-09-10T22:49:54.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.288+0000 7f60250f3740 1 Processor -- start 2024-09-10T22:49:54.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.288+0000 7f60250f3740 1 -- start start 2024-09-10T22:49:54.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c261c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c261a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a7d00 con 0x5586c261d9c0 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a9840 con 0x5586c2629950 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60250f3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a83d0 con 0x5586c261d0a0 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f601bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c261c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43726/0 (socket says 172.21.15.96:43726) 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 -- 172.21.15.96:0/2729697437 learned_addr learned my addr 172.21.15.96:0/2729697437 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:54.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f60212bb640 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c261a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 -- 172.21.15.96:0/2729697437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 msgr2=0x5586c261a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c261a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 -- 172.21.15.96:0/2729697437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 msgr2=0x5586c261c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c261c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.289+0000 7f6020aba640 1 -- 172.21.15.96:0/2729697437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c261ac50 con 0x5586c261d0a0 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f6020aba640 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f601000c6a0 tx=0x7f6010004a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=164a54977ac652cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60067fc640 1 -- 172.21.15.96:0/2729697437 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100ac290 con 0x5586c261d0a0 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60067fc640 1 -- 172.21.15.96:0/2729697437 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60100ace80 con 0x5586c261d0a0 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60067fc640 1 -- 172.21.15.96:0/2729697437 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100a8dd0 con 0x5586c261d0a0 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 -- 172.21.15.96:0/2729697437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 msgr2=0x5586c26358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f601000c6a0 tx=0x7f6010004a20 comp rx=0 tx=0).stop 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 -- 172.21.15.96:0/2729697437 shutdown_connections 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c2629950 0x5586c261a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c261d9c0 0x5586c261c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 --2- 172.21.15.96:0/2729697437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d0a0 0x5586c26358f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 -- 172.21.15.96:0/2729697437 >> 172.21.15.96:0/2729697437 conn(0x5586c261aed0 msgr2=0x5586c261bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.290+0000 7f60250f3740 1 -- 172.21.15.96:0/2729697437 shutdown_connections 2024-09-10T22:49:54.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 -- 172.21.15.96:0/2729697437 wait complete. 2024-09-10T22:49:54.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 -- start start 2024-09-10T22:49:54.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 Processor -- start 2024-09-10T22:49:54.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c2640cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2641200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a9170 con 0x5586c2629950 2024-09-10T22:49:54.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a8aa0 con 0x5586c261d0a0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.291+0000 7f60250f3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1a5e00 con 0x5586c261d9c0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.292+0000 7f60250f3740 10 client.0 _ll_register_callbacks cb 0x7f6008009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.292+0000 7f6020aba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.292+0000 7f6020aba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37346/0 (socket says 172.21.15.96:37346) 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.292+0000 7f6020aba640 1 -- 172.21.15.96:0/145347870 learned_addr learned my addr 172.21.15.96:0/145347870 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.292+0000 7f60212bb640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2641200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 -- 172.21.15.96:0/145347870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 msgr2=0x5586c2640cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c2640cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 -- 172.21.15.96:0/145347870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 msgr2=0x5586c2641200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2641200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 -- 172.21.15.96:0/145347870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c2641910 con 0x5586c261d0a0 2024-09-10T22:49:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f6020aba640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60100ac620 tx=0x7f60100abff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568cfad3a91c5ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f5ffbfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.293+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100a8dd0 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f60100abbf0 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60100ace80 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f60100ac290 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 10 client.25538 fetch_fsmap learned FSMap version 32 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586c26a03c0 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60100ace80 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6010086190 con 0x5586c261d0a0 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 10 client.25538 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 20 client.25538 populate_metadata read hostname 'smithi096' 2024-09-10T22:49:54.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c26a0640 0x5586c26a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:49:54.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 4 client.25538 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:49:54.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.294+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586c2591850 con 0x5586c26a0640 2024-09-10T22:49:54.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f601bfff640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c26a0640 0x5586c26a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:49:54.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f601bfff640 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c26a0640 0x5586c26a2a30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f50e915d4c6ee0 server_cookie=ef9ed23c09814145 in_seq=0 out_seq=0 2024-09-10T22:49:54.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f5ffbfff640 10 client.25538 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f5ffbfff640 1 -- 172.21.15.96:0/145347870 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x5586c2591850 con 0x5586c26a0640 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f5ffbfff640 10 client.25538 handle_command_reply: tid=0 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 client.25538 shutdown 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 msgr2=0x5586c2632020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60100ac620 tx=0x7f60100abff0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 shutdown_connections 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5586c26a0640 0x5586c26a2a30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586c2629950 0x5586c2641200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f5ffbfff640 0 client.25538 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5586c261d9c0 0x5586c2640cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 --2- 172.21.15.96:0/145347870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586c261d0a0 0x5586c2632020 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.295+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 >> 172.21.15.96:0/145347870 conn(0x5586c261aed0 msgr2=0x5586c2627c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 shutdown_connections 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f60250f3740 1 -- 172.21.15.96:0/145347870 wait complete. 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f60250f3740 20 client.25538 trim_cache size 0 max 16384 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:49:54.301 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "a9b1e2a5-38f5-4899-a8a8-92c463641aa7", 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:54.302 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:54.303 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:49:54.304 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 msgr2=0x7f600c0b9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 0x7f600c0b9a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6014004af0 tx=0x7f60140305e0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 msgr2=0x7f601c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f601000e9c0 tx=0x7f601000ee80 comp rx=0 tx=0).stop 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 shutdown_connections 2024-09-10T22:49:54.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c1076a0 0x7f601c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f600c0b7570 0x7f600c0b9a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f601c108020 0x7f601c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 --2- 172.21.15.96:0/4006365025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f601c101c90 0x7f601c198190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.296+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 >> 172.21.15.96:0/4006365025 conn(0x7f601c0fd700 msgr2=0x7f601c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.297+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 shutdown_connections 2024-09-10T22:49:54.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:49:54.297+0000 7f6022d1e640 1 -- 172.21.15.96:0/4006365025 wait complete. 2024-09-10T22:49:55.039 INFO:tasks.cephfs.filesystem:scrub status for tag:a9b1e2a5-38f5-4899-a8a8-92c463641aa7 - {'path': '/', 'tag': 'a9b1e2a5-38f5-4899-a8a8-92c463641aa7', 'options': 'recursive,force'} 2024-09-10T22:49:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:55 smithi096 ceph-mon[30869]: pgmap v425: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.6 MiB/s wr, 827 op/s 2024-09-10T22:49:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:55 smithi121 ceph-mon[35119]: pgmap v425: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.6 MiB/s wr, 827 op/s 2024-09-10T22:49:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:55 smithi117 ceph-mon[35081]: pgmap v425: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.6 MiB/s wr, 827 op/s 2024-09-10T22:49:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 140.0 s, 2815.6 tps, lat 35.550 ms stddev 42.598 2024-09-10T22:49:57.551 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:57 smithi096 ceph-mon[30869]: pgmap v426: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.2 MiB/s wr, 905 op/s 2024-09-10T22:49:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:57 smithi121 ceph-mon[35119]: pgmap v426: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.2 MiB/s wr, 905 op/s 2024-09-10T22:49:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:57 smithi117 ceph-mon[35081]: pgmap v426: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 9.2 MiB/s wr, 905 op/s 2024-09-10T22:49:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:59 smithi096 ceph-mon[30869]: pgmap v427: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.3 MiB/s wr, 720 op/s 2024-09-10T22:49:59.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:49:59 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:49:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:59 smithi121 ceph-mon[35119]: pgmap v427: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.3 MiB/s wr, 720 op/s 2024-09-10T22:49:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:49:59 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:49:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:59 smithi117 ceph-mon[35081]: pgmap v427: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.3 MiB/s wr, 720 op/s 2024-09-10T22:49:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:49:59 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:50:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.681+0000 7f4195a2a640 1 -- 172.21.15.96:0/2770119948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 msgr2=0x7f4190074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.681+0000 7f4195a2a640 1 --2- 172.21.15.96:0/2770119948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f4190074690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4184009900 tx=0x7f418402a440 comp rx=0 tx=0).stop 2024-09-10T22:50:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.682+0000 7f4195a2a640 1 -- 172.21.15.96:0/2770119948 shutdown_connections 2024-09-10T22:50:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.682+0000 7f4195a2a640 1 --2- 172.21.15.96:0/2770119948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f419010a150 0x7f419010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.682+0000 7f4195a2a640 1 --2- 172.21.15.96:0/2770119948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f419011a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.682+0000 7f4195a2a640 1 --2- 172.21.15.96:0/2770119948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f4190074690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.682+0000 7f4195a2a640 1 -- 172.21.15.96:0/2770119948 >> 172.21.15.96:0/2770119948 conn(0x7f419006fa40 msgr2=0x7f4190071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.684+0000 7f4195a2a640 1 -- 172.21.15.96:0/2770119948 shutdown_connections 2024-09-10T22:50:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.684+0000 7f4195a2a640 1 -- 172.21.15.96:0/2770119948 wait complete. 2024-09-10T22:50:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 Processor -- start 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 -- start start 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f419010a150 0x7f41901a8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f41901a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f419011d3f0 con 0x7f419010ab10 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f419011d270 con 0x7f419010a150 2024-09-10T22:50:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.685+0000 7f4195a2a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f419011d570 con 0x7f4190074290 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44236/0 (socket says 172.21.15.96:44236) 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f41901a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 -- 172.21.15.96:0/3876223886 learned_addr learned my addr 172.21.15.96:0/3876223886 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 -- 172.21.15.96:0/3876223886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f419010a150 msgr2=0x7f41901a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f419010a150 0x7f41901a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 -- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 msgr2=0x7f41901a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f41901a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418effd640 1 -- 172.21.15.96:0/3876223886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41901a9680 con 0x7f4190074290 2024-09-10T22:50:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.686+0000 7f418f7fe640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f41901a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.688+0000 7f418effd640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4184005ab0 tx=0x7f4184003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.688+0000 7f4194a28640 1 -- 172.21.15.96:0/3876223886 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418403f070 con 0x7f4190074290 2024-09-10T22:50:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.688+0000 7f4194a28640 1 -- 172.21.15.96:0/3876223886 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4184031bc0 con 0x7f4190074290 2024-09-10T22:50:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.688+0000 7f4194a28640 1 -- 172.21.15.96:0/3876223886 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184035030 con 0x7f4190074290 2024-09-10T22:50:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.689+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901a9910 con 0x7f4190074290 2024-09-10T22:50:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.689+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901afe60 con 0x7f4190074290 2024-09-10T22:50:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.693+0000 7f41987fd740 1 Processor -- start 2024-09-10T22:50:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 -- start start 2024-09-10T22:50:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91154580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e91157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e910c16f0 con 0x558e91154bd0 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e910b0b20 con 0x558e91152190 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f41987fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e910b4930 con 0x558e9114f860 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44250/0 (socket says 172.21.15.96:44250) 2024-09-10T22:50:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.694+0000 7f418f7fe640 1 -- 172.21.15.96:0/2738286569 learned_addr learned my addr 172.21.15.96:0/2738286569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.695+0000 7f418f7fe640 1 -- 172.21.15.96:0/2738286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 msgr2=0x558e91154580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.695+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91154580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.695+0000 7f418f7fe640 1 -- 172.21.15.96:0/2738286569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 msgr2=0x558e91157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.695+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e91157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.695+0000 7f418f7fe640 1 -- 172.21.15.96:0/2738286569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e91076530 con 0x558e9114f860 2024-09-10T22:50:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.696+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4180011470 tx=0x7f4180011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab8ed5a7aeca7f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.696+0000 7f4194a28640 1 -- 172.21.15.96:0/3876223886 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f418403b460 con 0x7f4190074290 2024-09-10T22:50:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.696+0000 7f4175ffb640 1 -- 172.21.15.96:0/2738286569 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180004020 con 0x558e9114f860 2024-09-10T22:50:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.696+0000 7f4175ffb640 1 -- 172.21.15.96:0/2738286569 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800041c0 con 0x558e9114f860 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.697+0000 7f4175ffb640 1 -- 172.21.15.96:0/2738286569 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418000f8b0 con 0x558e9114f860 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.697+0000 7f41987fd740 1 -- 172.21.15.96:0/2738286569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 msgr2=0x558e91151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.697+0000 7f41987fd740 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4180011470 tx=0x7f4180011930 comp rx=0 tx=0).stop 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 -- 172.21.15.96:0/2738286569 shutdown_connections 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e91157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 --2- 172.21.15.96:0/2738286569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91151c50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 -- 172.21.15.96:0/2738286569 >> 172.21.15.96:0/2738286569 conn(0x558e9114a9b0 msgr2=0x558e9114b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f4194a28640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 0x7f41780b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f418f7fe640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 0x7f41780b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 -- 172.21.15.96:0/2738286569 shutdown_connections 2024-09-10T22:50:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 -- 172.21.15.96:0/2738286569 wait complete. 2024-09-10T22:50:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f4194a28640 1 -- 172.21.15.96:0/3876223886 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4184106510 con 0x7f4190074290 2024-09-10T22:50:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.698+0000 7f41987fd740 1 -- start start 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 Processor -- start 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f418f7fe640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 0x7f41780b79b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4180011440 tx=0x7f4180009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e9115a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e90fe8ab0 con 0x558e91154bd0 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e91056730 con 0x558e91152190 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e91157c00 con 0x558e9114f860 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f41987fd740 10 client.0 _ll_register_callbacks cb 0x558e910aa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f418effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f418effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44256/0 (socket says 172.21.15.96:44256) 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.699+0000 7f418effd640 1 -- 172.21.15.96:0/2500343598 learned_addr learned my addr 172.21.15.96:0/2500343598 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.700+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e9115a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 -- 172.21.15.96:0/2500343598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 msgr2=0x558e91158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 -- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 msgr2=0x558e91159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 -- 172.21.15.96:0/2500343598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e90e12790 con 0x558e91154bd0 2024-09-10T22:50:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e9115a1f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4180000c00 tx=0x7f4180011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263483386a548252 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.701+0000 7f416effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180033020 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41800041c0 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4180016040 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f418002b040 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f41987fd740 10 client.15987 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.702+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e90e8b620 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800041c0 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f418002e030 con 0x558e91154bd0 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 10 client.15987 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 20 client.15987 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e91164700 0x558e91166bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 4 client.15987 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.703+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e910c3640 con 0x558e91164700 2024-09-10T22:50:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.704+0000 7f418effd640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e91164700 0x558e91166bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.704+0000 7f418effd640 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e91164700 0x558e91166bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d19240046ec045 server_cookie=24c5b1db4b292352 in_seq=0 out_seq=0 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.704+0000 7f416effd640 10 client.15987 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.705+0000 7f416effd640 1 -- 172.21.15.96:0/2500343598 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e910c3640 con 0x558e91164700 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.705+0000 7f416effd640 10 client.15987 handle_command_reply: tid=0 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.705+0000 7f41987fd740 1 client.15987 shutdown 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 msgr2=0x558e9115a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e9115a1f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4180000c00 tx=0x7f4180011930 comp rx=0 tx=0).stop 2024-09-10T22:50:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 shutdown_connections 2024-09-10T22:50:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e91164700 0x558e91166bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e91159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91154bd0 0x558e9115a1f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 --2- 172.21.15.96:0/2500343598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.706+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 >> 172.21.15.96:0/2500343598 conn(0x558e9114a9b0 msgr2=0x558e91149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.707+0000 7f416effd640 0 client.15987 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.707+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 shutdown_connections 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.707+0000 7f41987fd740 1 -- 172.21.15.96:0/2500343598 wait complete. 2024-09-10T22:50:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.707+0000 7f41987fd740 20 client.15987 trim_cache size 0 max 16384 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.728+0000 7f41987fd740 1 Processor -- start 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.728+0000 7f41987fd740 1 -- start start 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 0x558e91161270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00e450 con 0x558e91152190 2024-09-10T22:50:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00ff90 con 0x558e91154bd0 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41987fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00eb20 con 0x558e9114f860 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 0x558e91161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48812/0 (socket says 172.21.15.96:48812) 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 -- 172.21.15.96:0/1083332184 learned_addr learned my addr 172.21.15.96:0/1083332184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418effd640 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 -- 172.21.15.96:0/1083332184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 msgr2=0x558e91158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 -- 172.21.15.96:0/1083332184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 msgr2=0x558e91161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 0x558e91161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 -- 172.21.15.96:0/1083332184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e91151fc0 con 0x558e91152190 2024-09-10T22:50:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418f7fe640 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 0x558e91161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f418e7fc640 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4188007a90 tx=0x7f4188007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaebce5f32fb201 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41757fa640 1 -- 172.21.15.96:0/1083332184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188014070 con 0x558e91152190 2024-09-10T22:50:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41757fa640 1 -- 172.21.15.96:0/1083332184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f418800f900 con 0x558e91152190 2024-09-10T22:50:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.729+0000 7f41757fa640 1 -- 172.21.15.96:0/1083332184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800e910 con 0x558e91152190 2024-09-10T22:50:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.730+0000 7f41987fd740 1 -- 172.21.15.96:0/1083332184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 msgr2=0x558e91158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.730+0000 7f41987fd740 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4188007a90 tx=0x7f4188007f50 comp rx=0 tx=0).stop 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 -- 172.21.15.96:0/1083332184 shutdown_connections 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91154bd0 0x558e91161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e91152190 0x558e91158740 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 --2- 172.21.15.96:0/1083332184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e9114f860 0x558e91158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 -- 172.21.15.96:0/1083332184 >> 172.21.15.96:0/1083332184 conn(0x558e9114a9b0 msgr2=0x558e9114da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 -- 172.21.15.96:0/1083332184 shutdown_connections 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 -- 172.21.15.96:0/1083332184 wait complete. 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.731+0000 7f41987fd740 1 -- start start 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.732+0000 7f41987fd740 1 Processor -- start 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.732+0000 7f41987fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.732+0000 7f41987fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e9115e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.732+0000 7f41987fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 0x558e9115f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.732+0000 7f418effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00f8c0 con 0x558e9114f860 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48814/0 (socket says 172.21.15.96:48814) 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00f1f0 con 0x558e91152190 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 -- 172.21.15.96:0/2485803400 learned_addr learned my addr 172.21.15.96:0/2485803400 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f417c00c430 con 0x558e91154bd0 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418e7fc640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e9115e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 10 client.0 _ll_register_callbacks cb 0x7f4180009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 0x558e9115f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 -- 172.21.15.96:0/2485803400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 msgr2=0x558e9115f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 0x558e9115f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 -- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 msgr2=0x558e9115e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e9115e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 -- 172.21.15.96:0/2485803400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9115fca0 con 0x558e9114f860 2024-09-10T22:50:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418f7fe640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 0x558e9115f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f418effd640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f41840c92a0 tx=0x7f4184009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b347bc0395dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f416effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184049050 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f41840cd130 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418800f900 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f41840b8080 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.733+0000 7f41987fd740 10 client.15993 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e911610e0 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41840cccd0 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f41840d4b30 con 0x558e9114f860 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 10 client.15993 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 20 client.15993 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e911d18a0 0x558e911d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 4 client.15993 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:50:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f418800f900 con 0x558e911d18a0 2024-09-10T22:50:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f418e7fc640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e911d18a0 0x558e911d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.734+0000 7f416effd640 10 client.15993 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.735+0000 7f418e7fc640 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e911d18a0 0x558e911d3c90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5f0e11d7afb325 server_cookie=bf209d27efdbb6cd in_seq=0 out_seq=0 2024-09-10T22:50:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:00 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:50:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:00 smithi096 ceph-mon[30869]: overall HEALTH_OK 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f416effd640 1 -- 172.21.15.96:0/2485803400 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f418800f900 con 0x558e911d18a0 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f416effd640 10 client.15993 handle_command_reply: tid=0 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 client.15993 shutdown 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 msgr2=0x558e9115e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f41840c92a0 tx=0x7f4184009260 comp rx=0 tx=0).stop 2024-09-10T22:50:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 shutdown_connections 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e911d18a0 0x558e911d3c90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e91152190 0x558e9115e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e91154bd0 0x558e9115f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 --2- 172.21.15.96:0/2485803400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e9114f860 0x558e9115e200 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f416effd640 0 client.15993 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 >> 172.21.15.96:0/2485803400 conn(0x558e9114a9b0 msgr2=0x558e91158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 shutdown_connections 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 1 -- 172.21.15.96:0/2485803400 wait complete. 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.752+0000 7f41987fd740 20 client.15993 trim_cache size 0 max 16384 2024-09-10T22:50:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:50:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:50:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:50:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 msgr2=0x7f41780b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 0x7f41780b79b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4180011440 tx=0x7f4180009310 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 msgr2=0x7f41901a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4184005ab0 tx=0x7f4184003c20 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 shutdown_connections 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f419010a150 0x7f41901a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f41780b54f0 0x7f41780b79b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f419010ab10 0x7f41901a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 --2- 172.21.15.96:0/3876223886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4190074290 0x7f41901a7b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 >> 172.21.15.96:0/3876223886 conn(0x7f419006fa40 msgr2=0x7f4190071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 shutdown_connections 2024-09-10T22:50:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:00.753+0000 7f4195a2a640 1 -- 172.21.15.96:0/3876223886 wait complete. 2024-09-10T22:50:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:00 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:50:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:00 smithi121 ceph-mon[35119]: overall HEALTH_OK 2024-09-10T22:50:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:00 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:50:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:00 smithi117 ceph-mon[35081]: overall HEALTH_OK 2024-09-10T22:50:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 145.0 s, 3196.6 tps, lat 31.300 ms stddev 33.497 2024-09-10T22:50:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:01 smithi096 ceph-mon[30869]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.0 MiB/s wr, 882 op/s 2024-09-10T22:50:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:01 smithi121 ceph-mon[35119]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.0 MiB/s wr, 882 op/s 2024-09-10T22:50:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:01 smithi117 ceph-mon[35081]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.0 MiB/s wr, 882 op/s 2024-09-10T22:50:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:03 smithi096 ceph-mon[30869]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 786 op/s 2024-09-10T22:50:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:03 smithi121 ceph-mon[35119]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 786 op/s 2024-09-10T22:50:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:03 smithi117 ceph-mon[35081]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 8.0 MiB/s wr, 786 op/s 2024-09-10T22:50:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:05 smithi096 ceph-mon[30869]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.9 MiB/s wr, 861 op/s 2024-09-10T22:50:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:05 smithi121 ceph-mon[35119]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.9 MiB/s wr, 861 op/s 2024-09-10T22:50:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:05 smithi117 ceph-mon[35081]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.9 MiB/s wr, 861 op/s 2024-09-10T22:50:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 150.0 s, 3376.6 tps, lat 29.626 ms stddev 29.058 2024-09-10T22:50:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:07 smithi096 ceph-mon[30869]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-09-10T22:50:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:07 smithi121 ceph-mon[35119]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-09-10T22:50:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:07 smithi117 ceph-mon[35081]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 811 op/s 2024-09-10T22:50:09.606 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:09 smithi096 ceph-mon[30869]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.8 MiB/s wr, 689 op/s 2024-09-10T22:50:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:09 smithi121 ceph-mon[35119]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.8 MiB/s wr, 689 op/s 2024-09-10T22:50:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:09 smithi117 ceph-mon[35081]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.8 MiB/s wr, 689 op/s 2024-09-10T22:50:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 155.0 s, 3313.2 tps, lat 30.191 ms stddev 30.977 2024-09-10T22:50:11.577 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:11 smithi117 ceph-mon[35081]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 797 op/s 2024-09-10T22:50:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:11 smithi096 ceph-mon[30869]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 797 op/s 2024-09-10T22:50:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:11 smithi121 ceph-mon[35119]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 797 op/s 2024-09-10T22:50:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:13 smithi096 ceph-mon[30869]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:50:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:13 smithi121 ceph-mon[35119]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:50:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:13 smithi117 ceph-mon[35081]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:50:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.879+0000 7f5008952640 1 -- 172.21.15.96:0/4042155447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5004106470 msgr2=0x7f5004106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.879+0000 7f5008952640 1 --2- 172.21.15.96:0/4042155447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5004106470 0x7f5004106920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009ca0 tx=0x7f4ffc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 -- 172.21.15.96:0/4042155447 shutdown_connections 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 --2- 172.21.15.96:0/4042155447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5004106470 0x7f5004106920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 --2- 172.21.15.96:0/4042155447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50041038c0 0x7f5004105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 --2- 172.21.15.96:0/4042155447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f500406bff0 0x7f5004103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 -- 172.21.15.96:0/4042155447 >> 172.21.15.96:0/4042155447 conn(0x7f50040fd130 msgr2=0x7f50040ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 -- 172.21.15.96:0/4042155447 shutdown_connections 2024-09-10T22:50:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.880+0000 7f5008952640 1 -- 172.21.15.96:0/4042155447 wait complete. 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 Processor -- start 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 -- start start 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 0x7f5004072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 0x7f5004072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5004114bf0 con 0x7f500406bff0 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5004114a70 con 0x7f50041038c0 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.882+0000 7f5008952640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5004114d70 con 0x7f5004106470 2024-09-10T22:50:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f500259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 0x7f5004072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f500259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45236/0 (socket says 172.21.15.96:45236) 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f500259c640 1 -- 172.21.15.96:0/1772918897 learned_addr learned my addr 172.21.15.96:0/1772918897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 0x7f5004072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 -- 172.21.15.96:0/1772918897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 msgr2=0x7f5004072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 0x7f5004072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 -- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 msgr2=0x7f5004071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f5002d9d640 1 -- 172.21.15.96:0/1772918897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5004073700 con 0x7f5004106470 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 0x7f5004072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.883+0000 7f500259c640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 0x7f5004072e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc031a20 tx=0x7f4ffc0317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f4ffb7fe640 1 -- 172.21.15.96:0/1772918897 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc003c30 con 0x7f5004106470 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5004073990 con 0x7f5004106470 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5004074090 con 0x7f5004106470 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f4ffb7fe640 1 -- 172.21.15.96:0/1772918897 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ffc003dd0 con 0x7f5004106470 2024-09-10T22:50:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.884+0000 7f4ffb7fe640 1 -- 172.21.15.96:0/1772918897 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc02fb60 con 0x7f5004106470 2024-09-10T22:50:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.887+0000 7f4ffb7fe640 1 -- 172.21.15.96:0/1772918897 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4ffc0305c0 con 0x7f5004106470 2024-09-10T22:50:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.887+0000 7f500b7fc740 1 Processor -- start 2024-09-10T22:50:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.887+0000 7f500b7fc740 1 -- start start 2024-09-10T22:50:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c984580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c987090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c09c8f16f0 con 0x55c09c97f860 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c09c8e0b20 con 0x55c09c982190 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f500b7fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c09c8e4930 con 0x55c09c984bd0 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45240/0 (socket says 172.21.15.96:45240) 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f4ffb7fe640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 0x7f4ff00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f4ffbfff640 1 -- 172.21.15.96:0/1716246397 learned_addr learned my addr 172.21.15.96:0/1716246397 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 0x7f4ff00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.888+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c987090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 -- 172.21.15.96:0/1716246397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 msgr2=0x55c09c987090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c987090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 -- 172.21.15.96:0/1716246397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 msgr2=0x55c09c984580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c984580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 -- 172.21.15.96:0/1716246397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09c8a6530 con 0x55c09c97f860 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.889+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005bc0 tx=0x7f4fec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0354cf635cde9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f4ffb7fe640 1 -- 172.21.15.96:0/1772918897 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4ffc106d40 con 0x7f5004106470 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f4ff8ff9640 1 -- 172.21.15.96:0/1716246397 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec013d60 con 0x55c09c97f860 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f5002d9d640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 0x7f4ff00b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009ca0 tx=0x7f4ffc041000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f4ff8ff9640 1 -- 172.21.15.96:0/1716246397 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec013170 con 0x55c09c97f860 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f4ff8ff9640 1 -- 172.21.15.96:0/1716246397 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec010d30 con 0x55c09c97f860 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 -- 172.21.15.96:0/1716246397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 msgr2=0x55c09c981c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005bc0 tx=0x7f4fec009c00 comp rx=0 tx=0).stop 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 -- 172.21.15.96:0/1716246397 shutdown_connections 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c984580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c987090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 --2- 172.21.15.96:0/1716246397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c981c50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.890+0000 7f500b7fc740 1 -- 172.21.15.96:0/1716246397 >> 172.21.15.96:0/1716246397 conn(0x55c09c97a9b0 msgr2=0x55c09c97b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- 172.21.15.96:0/1716246397 shutdown_connections 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- 172.21.15.96:0/1716246397 wait complete. 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- start start 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 Processor -- start 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c989540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c98a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c09c818ab0 con 0x55c09c97f860 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c09c886730 con 0x55c09c982190 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c09c987b50 con 0x55c09c984bd0 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500b7fc740 10 client.0 _ll_register_callbacks cb 0x55c09c8da6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45246/0 (socket says 172.21.15.96:45246) 2024-09-10T22:50:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f4ffbfff640 1 -- 172.21.15.96:0/462131368 learned_addr learned my addr 172.21.15.96:0/462131368 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f500259c640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c989540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.891+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c98a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 -- 172.21.15.96:0/462131368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 msgr2=0x55c09c98a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c98a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 -- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 msgr2=0x55c09c989540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c989540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 -- 172.21.15.96:0/462131368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09c642790 con 0x55c09c97f860 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f500259c640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c989540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0095f0 tx=0x7f4fec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91afd56097067060 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4fe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec026030 con 0x55c09c97f860 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec013170 con 0x55c09c97f860 2024-09-10T22:50:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4fec010ad0 con 0x55c09c97f860 2024-09-10T22:50:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.892+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fec010cd0 con 0x55c09c97f860 2024-09-10T22:50:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.893+0000 7f500b7fc740 10 client.16005 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.893+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c09c6bb620 con 0x55c09c97f860 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec013170 con 0x55c09c97f860 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec003960 con 0x55c09c97f860 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 20 client.16005 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09c9944c0 0x55c09c9968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 4 client.16005 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09c8f3640 con 0x55c09c9944c0 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.894+0000 7f500259c640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09c9944c0 0x55c09c9968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.895+0000 7f4fe9ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.895+0000 7f500259c640 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09c9944c0 0x55c09c9968f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51c2fd546a2db2e server_cookie=571d82e3c883e1dd in_seq=0 out_seq=0 2024-09-10T22:50:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.895+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/462131368 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c09c8f3640 con 0x55c09c9944c0 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.895+0000 7f4fe9ffb640 10 client.16005 handle_command_reply: tid=0 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.895+0000 7f500b7fc740 1 client.16005 shutdown 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 msgr2=0x55c09c988ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0095f0 tx=0x7f4fec005f90 comp rx=0 tx=0).stop 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 shutdown_connections 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09c9944c0 0x55c09c9968f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c989540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c98a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f4fe9ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 --2- 172.21.15.96:0/462131368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c97f860 0x55c09c988ea0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 >> 172.21.15.96:0/462131368 conn(0x55c09c97a9b0 msgr2=0x55c09c979bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 shutdown_connections 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 1 -- 172.21.15.96:0/462131368 wait complete. 2024-09-10T22:50:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.896+0000 7f500b7fc740 20 client.16005 trim_cache size 0 max 16384 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 Processor -- start 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 -- start start 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c97f860 0x55c09c985020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c97b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400e330 con 0x55c09c982190 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400fe70 con 0x55c09c97f860 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500b7fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400ea00 con 0x55c09c984bd0 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45258/0 (socket says 172.21.15.96:45258) 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f500259c640 1 -- 172.21.15.96:0/3083882862 learned_addr learned my addr 172.21.15.96:0/3083882862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.918+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c97b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c97f860 0x55c09c985020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 -- 172.21.15.96:0/3083882862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 msgr2=0x55c09c97b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c97b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 -- 172.21.15.96:0/3083882862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c97f860 msgr2=0x55c09c985020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c97f860 0x55c09c985020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 -- 172.21.15.96:0/3083882862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09c981fc0 con 0x55c09c982190 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f500259c640 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4012840 tx=0x7f4ff400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf85763e48198eec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f4febfff640 1 -- 172.21.15.96:0/3083882862 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4021070 con 0x55c09c982190 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f4febfff640 1 -- 172.21.15.96:0/3083882862 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff40054d0 con 0x55c09c982190 2024-09-10T22:50:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.919+0000 7f4febfff640 1 -- 172.21.15.96:0/3083882862 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4004ae0 con 0x55c09c982190 2024-09-10T22:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.920+0000 7f500b7fc740 1 -- 172.21.15.96:0/3083882862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 msgr2=0x55c09c985560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.920+0000 7f500b7fc740 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4012840 tx=0x7f4ff400f790 comp rx=0 tx=0).stop 2024-09-10T22:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 -- 172.21.15.96:0/3083882862 shutdown_connections 2024-09-10T22:50:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c984bd0 0x55c09c97b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c982190 0x55c09c985560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 --2- 172.21.15.96:0/3083882862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c97f860 0x55c09c985020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 -- 172.21.15.96:0/3083882862 >> 172.21.15.96:0/3083882862 conn(0x55c09c97a9b0 msgr2=0x55c09c97ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 -- 172.21.15.96:0/3083882862 shutdown_connections 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 -- 172.21.15.96:0/3083882862 wait complete. 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.921+0000 7f500b7fc740 1 -- start start 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.922+0000 7f500b7fc740 1 Processor -- start 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.922+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.922+0000 7f500b7fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c9909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.922+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.922+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49650/0 (socket says 172.21.15.96:49650) 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 -- 172.21.15.96:0/2618150560 learned_addr learned my addr 172.21.15.96:0/2618150560 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c984bd0 0x55c09c9ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400f7a0 con 0x55c09c984bd0 2024-09-10T22:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400f0d0 con 0x55c09c982190 2024-09-10T22:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe400c310 con 0x55c09c97f860 2024-09-10T22:50:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500b7fc740 10 client.0 _ll_register_callbacks cb 0x7f4fec013820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c984bd0 0x55c09c9ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500259c640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c9909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 -- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 msgr2=0x55c09c9909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c9909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 -- 172.21.15.96:0/2618150560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c984bd0 msgr2=0x55c09c9ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c984bd0 0x55c09c9ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09c9ffae0 con 0x55c09c97f860 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f500259c640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c9909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4ffbfff640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4fec003e00 tx=0x7f4fec00ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721522224a9542bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.923+0000 7f4fe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.924+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec026070 con 0x55c09c97f860 2024-09-10T22:50:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.924+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4fec010a70 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.924+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff40054d0 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.924+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fec03e050 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.925+0000 7f500b7fc740 10 client.25594 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.925+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c09ca00e20 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.925+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff40054d0 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec02b6f0 con 0x55c09c97f860 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 10 client.25594 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 10 client.25594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 20 client.25594 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09ca01830 0x55c09ca03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 4 client.25594 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c09c8ef0f0 con 0x55c09ca01830 2024-09-10T22:50:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500259c640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09ca01830 0x55c09ca03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f4fe9ffb640 10 client.25594 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.926+0000 7f500259c640 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09ca01830 0x55c09ca03c50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4faf5001c3664 server_cookie=33a0f77a4b2e41 in_seq=0 out_seq=0 2024-09-10T22:50:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f4fe9ffb640 1 -- 172.21.15.96:0/2618150560 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c09c8ef0f0 con 0x55c09ca01830 2024-09-10T22:50:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f4fe9ffb640 10 client.25594 handle_command_reply: tid=0 2024-09-10T22:50:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 client.25594 shutdown 2024-09-10T22:50:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 msgr2=0x55c09c990420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4fec003e00 tx=0x7f4fec00ff60 comp rx=0 tx=0).stop 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 shutdown_connections 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c09ca01830 0x55c09ca03c50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c09c982190 0x55c09c9909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c09c984bd0 0x55c09c9ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 --2- 172.21.15.96:0/2618150560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c09c97f860 0x55c09c990420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.945+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 >> 172.21.15.96:0/2618150560 conn(0x55c09c97a9b0 msgr2=0x55c09c97b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f4fe9ffb640 0 client.25594 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 shutdown_connections 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f500b7fc740 1 -- 172.21.15.96:0/2618150560 wait complete. 2024-09-10T22:50:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f500b7fc740 20 client.25594 trim_cache size 0 max 16384 2024-09-10T22:50:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:50:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:50:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:50:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 msgr2=0x7f4ff00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 0x7f4ff00b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009ca0 tx=0x7f4ffc041000 comp rx=0 tx=0).stop 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 msgr2=0x7f5004072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 0x7f5004072e40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc031a20 tx=0x7f4ffc0317a0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.946+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 shutdown_connections 2024-09-10T22:50:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50041038c0 0x7f5004072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4ff00b5360 0x7f4ff00b7820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5004106470 0x7f5004072e40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 --2- 172.21.15.96:0/1772918897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f500406bff0 0x7f5004071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 >> 172.21.15.96:0/1772918897 conn(0x7f50040fd130 msgr2=0x7f50041045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 shutdown_connections 2024-09-10T22:50:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:13.947+0000 7f5008952640 1 -- 172.21.15.96:0/1772918897 wait complete. 2024-09-10T22:50:15.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:15 smithi121 ceph-mon[35119]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-10T22:50:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:15 smithi096 ceph-mon[30869]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-10T22:50:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:15 smithi117 ceph-mon[35081]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-10T22:50:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 160.0 s, 3726.2 tps, lat 26.722 ms stddev 27.783 2024-09-10T22:50:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:17 smithi096 ceph-mon[30869]: pgmap v436: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-09-10T22:50:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:17 smithi121 ceph-mon[35119]: pgmap v436: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-09-10T22:50:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:17 smithi117 ceph-mon[35081]: pgmap v436: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 MiB/s wr, 841 op/s 2024-09-10T22:50:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:19 smithi096 ceph-mon[30869]: pgmap v437: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T22:50:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:19 smithi121 ceph-mon[35119]: pgmap v437: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T22:50:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:19 smithi117 ceph-mon[35081]: pgmap v437: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T22:50:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 165.0 s, 3255.8 tps, lat 30.801 ms stddev 31.337 2024-09-10T22:50:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:21 smithi096 ceph-mon[30869]: pgmap v438: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 839 op/s 2024-09-10T22:50:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:21 smithi121 ceph-mon[35119]: pgmap v438: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 839 op/s 2024-09-10T22:50:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:21 smithi117 ceph-mon[35081]: pgmap v438: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 839 op/s 2024-09-10T22:50:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:23 smithi096 ceph-mon[30869]: pgmap v439: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:50:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:23 smithi121 ceph-mon[35119]: pgmap v439: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:50:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:23 smithi117 ceph-mon[35081]: pgmap v439: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 759 op/s 2024-09-10T22:50:25.040 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:50:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:25 smithi096 ceph-mon[30869]: pgmap v440: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-09-10T22:50:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:25 smithi121 ceph-mon[35119]: pgmap v440: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-09-10T22:50:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:25 smithi117 ceph-mon[35081]: pgmap v440: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 720 op/s 2024-09-10T22:50:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 170.0 s, 3252.8 tps, lat 30.641 ms stddev 33.064 2024-09-10T22:50:26.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 -- 172.21.15.96:0/906699440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a4101c90 msgr2=0x7f11a4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 --2- 172.21.15.96:0/906699440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a4101c90 0x7f11a4102070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f119402a410 comp rx=0 tx=0).stop 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 -- 172.21.15.96:0/906699440 shutdown_connections 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 --2- 172.21.15.96:0/906699440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a41076a0 0x7f11a4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 --2- 172.21.15.96:0/906699440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a4108020 0x7f11a410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 --2- 172.21.15.96:0/906699440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a4101c90 0x7f11a4102070 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 -- 172.21.15.96:0/906699440 >> 172.21.15.96:0/906699440 conn(0x7f11a40fd700 msgr2=0x7f11a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 -- 172.21.15.96:0/906699440 shutdown_connections 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.000+0000 7f11aa988640 1 -- 172.21.15.96:0/906699440 wait complete. 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 Processor -- start 2024-09-10T22:50:26.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 -- start start 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a4101c90 0x7f11a410c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a4108020 0x7f11a4108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4110c20 con 0x7f11a41076a0 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4110aa0 con 0x7f11a4101c90 2024-09-10T22:50:26.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.001+0000 7f11aa988640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4110da0 con 0x7f11a4108020 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33588/0 (socket says 172.21.15.96:33588) 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 -- 172.21.15.96:0/1588388163 learned_addr learned my addr 172.21.15.96:0/1588388163 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f11a3fff640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a4101c90 0x7f11a410c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 -- 172.21.15.96:0/1588388163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a4108020 msgr2=0x7f11a4108fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a4108020 0x7f11a4108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 -- 172.21.15.96:0/1588388163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a4101c90 msgr2=0x7f11a410c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a4101c90 0x7f11a410c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 -- 172.21.15.96:0/1588388163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a41abb10 con 0x7f11a41076a0 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.002+0000 7f119bfff640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b450 tx=0x7f118c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.003+0000 7f11a1ffb640 1 -- 172.21.15.96:0/1588388163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c015020 con 0x7f11a41076a0 2024-09-10T22:50:26.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.003+0000 7f11a1ffb640 1 -- 172.21.15.96:0/1588388163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118c011d40 con 0x7f11a41076a0 2024-09-10T22:50:26.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.003+0000 7f11a1ffb640 1 -- 172.21.15.96:0/1588388163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c00f340 con 0x7f11a41076a0 2024-09-10T22:50:26.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.003+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41abe00 con 0x7f11a41076a0 2024-09-10T22:50:26.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.003+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a41ac340 con 0x7f11a41076a0 2024-09-10T22:50:26.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.008+0000 7f11acd5d740 1 Processor -- start 2024-09-10T22:50:26.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.009+0000 7f11a1ffb640 1 -- 172.21.15.96:0/1588388163 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f118c011890 con 0x7f11a41076a0 2024-09-10T22:50:26.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 -- start start 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158366480 0x559158366860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158372d30 0x559158375120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591582dce50 con 0x559158372d30 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591582cc280 con 0x559158366da0 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11acd5d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5591582d0090 con 0x559158366480 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f119bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158372d30 0x559158375120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50204/0 (socket says 172.21.15.96:50204) 2024-09-10T22:50:26.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.010+0000 7f11a3fff640 1 -- 172.21.15.96:0/2984313684 learned_addr learned my addr 172.21.15.96:0/2984313684 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a8f25640 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158366480 0x559158366860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 -- 172.21.15.96:0/2984313684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158366480 msgr2=0x559158366860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158366480 0x559158366860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 -- 172.21.15.96:0/2984313684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158372d30 msgr2=0x559158375120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158372d30 0x559158375120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 -- 172.21.15.96:0/2984313684 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915802a650 con 0x559158366da0 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a1ffb640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 0x7f11900b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f11940042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119e538b9c84dc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a1ffb640 1 -- 172.21.15.96:0/1588388163 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f118c0e5730 con 0x7f11a41076a0 2024-09-10T22:50:26.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f119a7fc640 1 -- 172.21.15.96:0/2984313684 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035030 con 0x559158366da0 2024-09-10T22:50:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f11a8f25640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 0x7f11900b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f119a7fc640 1 -- 172.21.15.96:0/2984313684 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194005d10 con 0x559158366da0 2024-09-10T22:50:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.011+0000 7f119a7fc640 1 -- 172.21.15.96:0/2984313684 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194003610 con 0x559158366da0 2024-09-10T22:50:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 -- 172.21.15.96:0/2984313684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 msgr2=0x5591583727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11a8f25640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 0x7f11900b78f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f119c005c40 tx=0x7f119c00f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f11940042d0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 -- 172.21.15.96:0/2984313684 shutdown_connections 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x5591583727f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158372d30 0x559158375120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2984313684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158366480 0x559158366860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.012+0000 7f11acd5d740 1 -- 172.21.15.96:0/2984313684 >> 172.21.15.96:0/2984313684 conn(0x5591583642b0 msgr2=0x559158365450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.013+0000 7f11acd5d740 1 -- 172.21.15.96:0/2984313684 shutdown_connections 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.013+0000 7f11acd5d740 1 -- 172.21.15.96:0/2984313684 wait complete. 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.013+0000 7f11acd5d740 1 -- start start 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.013+0000 7f11acd5d740 1 Processor -- start 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x559158372630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x55915836fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55915812d700 con 0x559158366da0 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559158267110 con 0x559158366480 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559158375950 con 0x559158372d30 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11acd5d740 10 client.0 _ll_register_callbacks cb 0x559158280990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11a8f25640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11a8f25640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40392/0 (socket says 172.21.15.96:40392) 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11a8f25640 1 -- 172.21.15.96:0/291142402 learned_addr learned my addr 172.21.15.96:0/291142402 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f119bfff640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x55915836fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11a8f25640 1 -- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 msgr2=0x559158372630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.014+0000 7f11a8f25640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x559158372630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11a8f25640 1 -- 172.21.15.96:0/291142402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 msgr2=0x55915836fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11a8f25640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x55915836fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11a8f25640 1 -- 172.21.15.96:0/291142402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559158294c40 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f119bfff640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x55915836fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11a8f25640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f119c01ac10 tx=0x7f119c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abdf3c7b6a7d6bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11837fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c01a0d0 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194005d10 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.015+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f119c01ac40 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f119c01a270 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 10 client.25603 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559158381350 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194005d10 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f119c017070 con 0x559158372d30 2024-09-10T22:50:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 10 client.25603 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 20 client.25603 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583827b0 0x559158384ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 4 client.25603 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.016+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591582deda0 con 0x5591583827b0 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.017+0000 7f11a3fff640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583827b0 0x559158384ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.018+0000 7f11a3fff640 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583827b0 0x559158384ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55bae8786cfe3ade server_cookie=fcfd3e93383d66c7 in_seq=0 out_seq=0 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.018+0000 7f11837fe640 10 client.25603 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.019+0000 7f11837fe640 1 -- 172.21.15.96:0/291142402 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591582deda0 con 0x5591583827b0 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.019+0000 7f11837fe640 10 client.25603 handle_command_reply: tid=0 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.019+0000 7f11acd5d740 1 client.25603 shutdown 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 msgr2=0x559158370090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f119c01ac10 tx=0x7f119c009500 comp rx=0 tx=0).stop 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 shutdown_connections 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583827b0 0x559158384ba0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158370090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x55915836fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 --2- 172.21.15.96:0/291142402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x559158372630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 >> 172.21.15.96:0/291142402 conn(0x5591583642b0 msgr2=0x559158369e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11837fe640 0 client.25603 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 shutdown_connections 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.020+0000 7f11acd5d740 1 -- 172.21.15.96:0/291142402 wait complete. 2024-09-10T22:50:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.021+0000 7f11acd5d740 20 client.25603 trim_cache size 0 max 16384 2024-09-10T22:50:26.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 Processor -- start 2024-09-10T22:50:26.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 -- start start 2024-09-10T22:50:26.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158363bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b48f0 con 0x559158366da0 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b6430 con 0x559158366480 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11acd5d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b4fc0 con 0x559158372d30 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f119bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50216/0 (socket says 172.21.15.96:50216) 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f119bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33620/0 (socket says 172.21.15.96:33620) 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11a3fff640 1 -- 172.21.15.96:0/2453397188 learned_addr learned my addr 172.21.15.96:0/2453397188 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.046+0000 7f11a8f25640 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158363bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 -- 172.21.15.96:0/2453397188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 msgr2=0x559158363bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158363bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 -- 172.21.15.96:0/2453397188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 msgr2=0x5591583847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 -- 172.21.15.96:0/2453397188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559158366bd0 con 0x559158366480 2024-09-10T22:50:26.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f119bfff640 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:26.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f11a3fff640 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f119402aa00 tx=0x7f1194004b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa26ad5a1a08117 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f1199ffb640 1 -- 172.21.15.96:0/2453397188 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035040 con 0x559158366480 2024-09-10T22:50:26.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f1199ffb640 1 -- 172.21.15.96:0/2453397188 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1194004940 con 0x559158366480 2024-09-10T22:50:26.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.047+0000 7f1199ffb640 1 -- 172.21.15.96:0/2453397188 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194031720 con 0x559158366480 2024-09-10T22:50:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 -- 172.21.15.96:0/2453397188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 msgr2=0x5591583658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f119402aa00 tx=0x7f1194004b90 comp rx=0 tx=0).stop 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 -- 172.21.15.96:0/2453397188 shutdown_connections 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x559158363bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366da0 0x5591583847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 --2- 172.21.15.96:0/2453397188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366480 0x5591583658d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 -- 172.21.15.96:0/2453397188 >> 172.21.15.96:0/2453397188 conn(0x5591583642b0 msgr2=0x5591583651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.048+0000 7f11acd5d740 1 -- 172.21.15.96:0/2453397188 shutdown_connections 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.049+0000 7f11acd5d740 1 -- 172.21.15.96:0/2453397188 wait complete. 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.049+0000 7f11acd5d740 1 -- start start 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.049+0000 7f11acd5d740 1 Processor -- start 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 0x55915837b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x559158375ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x5591583760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 0x55915837b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b5d60 con 0x559158366480 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b5690 con 0x559158366da0 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11a41b2910 con 0x559158372d30 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 0x55915837b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33626/0 (socket says 172.21.15.96:33626) 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a3fff640 1 -- 172.21.15.96:0/3101637577 learned_addr learned my addr 172.21.15.96:0/3101637577 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x5591583760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f119bfff640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x559158375ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11acd5d740 10 client.0 _ll_register_callbacks cb 0x7f119c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 -- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 msgr2=0x559158375ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x559158375ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 -- 172.21.15.96:0/3101637577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 msgr2=0x55915837b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 0x55915837b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.050+0000 7f11a8f25640 1 -- 172.21.15.96:0/3101637577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591583766f0 con 0x559158372d30 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11a8f25640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x5591583760e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f119c02d4e0 tx=0x7f119c01aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4f43c67298ddae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11837fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c01a7c0 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f119c02d8e0 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194004940 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.051+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f119c01a270 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.052+0000 7f11acd5d740 10 client.25609 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.052+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5591583f04d0 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.052+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119c040410 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f119c03f050 con 0x559158372d30 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 10 client.25609 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 20 client.25609 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583f0bf0 0x5591583f3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 4 client.25609 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:50:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1194004940 con 0x5591583f0bf0 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.053+0000 7f11a3fff640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583f0bf0 0x5591583f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11a3fff640 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583f0bf0 0x5591583f3010 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ca9383760d53b4 server_cookie=7740d252245ec5eb in_seq=0 out_seq=0 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11837fe640 10 client.25609 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11837fe640 1 -- 172.21.15.96:0/3101637577 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x7f1194004940 con 0x5591583f0bf0 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11837fe640 10 client.25609 handle_command_reply: tid=0 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11acd5d740 1 client.25609 shutdown 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 msgr2=0x5591583760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.054+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x5591583760e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f119c02d4e0 tx=0x7f119c01aef0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 shutdown_connections 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5591583f0bf0 0x5591583f3010 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11837fe640 0 client.25609 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559158366da0 0x559158375ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559158372d30 0x5591583760e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 --2- 172.21.15.96:0/3101637577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559158366480 0x55915837b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 >> 172.21.15.96:0/3101637577 conn(0x5591583642b0 msgr2=0x5591583710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 shutdown_connections 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 1 -- 172.21.15.96:0/3101637577 wait complete. 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.055+0000 7f11acd5d740 20 client.25609 trim_cache size 0 max 16384 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:50:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:50:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:50:26.064 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 msgr2=0x7f11900b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 0x7f11900b78f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f119c005c40 tx=0x7f119c00f040 comp rx=0 tx=0).stop 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 msgr2=0x7f11a4108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b450 tx=0x7f118c00b910 comp rx=0 tx=0).stop 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 shutdown_connections 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11900b5430 0x7f11900b78f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11a4108020 0x7f11a4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a41076a0 0x7f11a4108810 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 --2- 172.21.15.96:0/1588388163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11a4101c90 0x7f11a410c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 >> 172.21.15.96:0/1588388163 conn(0x7f11a40fd700 msgr2=0x7f11a4103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 shutdown_connections 2024-09-10T22:50:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:26.056+0000 7f11aa988640 1 -- 172.21.15.96:0/1588388163 wait complete. 2024-09-10T22:50:26.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:50:26.918 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:50:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.102+0000 7f812f1b4640 1 -- 172.21.15.96:0/352309792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8128074500 msgr2=0x7f812811a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.102+0000 7f812f1b4640 1 --2- 172.21.15.96:0/352309792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8128074500 0x7f812811a090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8120009ca0 tx=0x7f812002a8d0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.105+0000 7f812f1b4640 1 -- 172.21.15.96:0/352309792 shutdown_connections 2024-09-10T22:50:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.105+0000 7f812f1b4640 1 --2- 172.21.15.96:0/352309792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8128074500 0x7f812811a090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.105+0000 7f812f1b4640 1 --2- 172.21.15.96:0/352309792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810ad00 0x7f8128073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.105+0000 7f812f1b4640 1 --2- 172.21.15.96:0/352309792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f812810a3c0 0x7f812810a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.105+0000 7f812f1b4640 1 -- 172.21.15.96:0/352309792 >> 172.21.15.96:0/352309792 conn(0x7f812806f850 msgr2=0x7f8128071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.106+0000 7f812f1b4640 1 -- 172.21.15.96:0/352309792 shutdown_connections 2024-09-10T22:50:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.106+0000 7f812f1b4640 1 -- 172.21.15.96:0/352309792 wait complete. 2024-09-10T22:50:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.106+0000 7f812f1b4640 1 Processor -- start 2024-09-10T22:50:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 -- start start 2024-09-10T22:50:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128074500 0x7f8128118e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 0x7f81281193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 0x7f8128119910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111d50 con 0x7f8128074500 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111bd0 con 0x7f812810ad00 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812f1b4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111ed0 con 0x7f812810a3c0 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 0x7f81281193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 0x7f81281193d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52734/0 (socket says 172.21.15.96:52734) 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812d9b1640 1 -- 172.21.15.96:0/827671328 learned_addr learned my addr 172.21.15.96:0/827671328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 0x7f8128119910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.107+0000 7f812e9b3640 1 -- 172.21.15.96:0/827671328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 msgr2=0x7f81281193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 0x7f81281193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812e9b3640 1 -- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128074500 msgr2=0x7f8128118e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128074500 0x7f8128118e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812e9b3640 1 -- 172.21.15.96:0/827671328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8128117ba0 con 0x7f812810ad00 2024-09-10T22:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 0x7f8128119910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f812002aec0 tx=0x7f8120003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f811f7fe640 1 -- 172.21.15.96:0/827671328 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120004240 con 0x7f812810ad00 2024-09-10T22:50:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.108+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8128117e30 con 0x7f812810ad00 2024-09-10T22:50:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.109+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8128118380 con 0x7f812810ad00 2024-09-10T22:50:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.109+0000 7f811f7fe640 1 -- 172.21.15.96:0/827671328 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812002f620 con 0x7f812810ad00 2024-09-10T22:50:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.109+0000 7f811f7fe640 1 -- 172.21.15.96:0/827671328 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120030830 con 0x7f812810ad00 2024-09-10T22:50:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.115+0000 7f81331eb740 1 Processor -- start 2024-09-10T22:50:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.120+0000 7f811f7fe640 1 -- 172.21.15.96:0/827671328 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f812004d070 con 0x7f812810ad00 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.120+0000 7f81331eb740 1 -- start start 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930d390 0x55a14930f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930fdd0 0x55a149312290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a14927c6f0 con 0x55a14930fdd0 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a14926bb20 con 0x55a14930aa60 2024-09-10T22:50:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.121+0000 7f81331eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a14926f930 con 0x55a14930d390 2024-09-10T22:50:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f811f7fe640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 0x7f81140b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38302/0 (socket says 172.21.15.96:38302) 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 -- 172.21.15.96:0/2896797754 learned_addr learned my addr 172.21.15.96:0/2896797754 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930fdd0 0x55a149312290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 0x7f81140b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 -- 172.21.15.96:0/2896797754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930d390 msgr2=0x55a14930f780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930d390 0x55a14930f780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 -- 172.21.15.96:0/2896797754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930fdd0 msgr2=0x55a149312290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930fdd0 0x55a149312290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.122+0000 7f812d9b1640 1 -- 172.21.15.96:0/2896797754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a149231530 con 0x55a14930aa60 2024-09-10T22:50:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.123+0000 7f811f7fe640 1 -- 172.21.15.96:0/827671328 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8120107920 con 0x7f812810ad00 2024-09-10T22:50:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.125+0000 7f812e9b3640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 0x7f81140b79a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81200cebf0 tx=0x7f812002a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.127+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8118005fa0 tx=0x7f8118005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110ab707e8c25372 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.128+0000 7f811cbfe640 1 -- 172.21.15.96:0/2896797754 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118012e10 con 0x55a14930aa60 2024-09-10T22:50:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.128+0000 7f811cbfe640 1 -- 172.21.15.96:0/2896797754 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118005020 con 0x55a14930aa60 2024-09-10T22:50:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.128+0000 7f811cbfe640 1 -- 172.21.15.96:0/2896797754 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004b00 con 0x55a14930aa60 2024-09-10T22:50:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 -- 172.21.15.96:0/2896797754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 msgr2=0x55a14930ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8118005fa0 tx=0x7f8118005f30 comp rx=0 tx=0).stop 2024-09-10T22:50:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 -- 172.21.15.96:0/2896797754 shutdown_connections 2024-09-10T22:50:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930fdd0 0x55a149312290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930d390 0x55a14930f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 --2- 172.21.15.96:0/2896797754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a14930ce50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.129+0000 7f81331eb740 1 -- 172.21.15.96:0/2896797754 >> 172.21.15.96:0/2896797754 conn(0x55a149305bb0 msgr2=0x55a149306940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.130+0000 7f81331eb740 1 -- 172.21.15.96:0/2896797754 shutdown_connections 2024-09-10T22:50:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.130+0000 7f81331eb740 1 -- 172.21.15.96:0/2896797754 wait complete. 2024-09-10T22:50:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 -- start start 2024-09-10T22:50:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 Processor -- start 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a149314880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930fdd0 0x55a149315450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1491a3ab0 con 0x55a14930d390 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a149211730 con 0x55a14930fdd0 2024-09-10T22:50:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a149312db0 con 0x55a14930aa60 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f81331eb740 10 client.0 _ll_register_callbacks cb 0x55a1492656b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930fdd0 0x55a149315450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52736/0 (socket says 172.21.15.96:52736) 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.131+0000 7f812d9b1640 1 -- 172.21.15.96:0/2481059695 learned_addr learned my addr 172.21.15.96:0/2481059695 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.132+0000 7f812e1b2640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a149314880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.134+0000 7f812e1b2640 1 -- 172.21.15.96:0/2481059695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 msgr2=0x55a1493141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.134+0000 7f812e1b2640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.134+0000 7f812e1b2640 1 -- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930fdd0 msgr2=0x55a149315450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.134+0000 7f812e1b2640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930fdd0 0x55a149315450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.134+0000 7f812e1b2640 1 -- 172.21.15.96:0/2481059695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a148fcd790 con 0x55a14930d390 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.135+0000 7f8111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.135+0000 7f812e1b2640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a149314880 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f812400bb40 tx=0x7f812400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff97ddfa98ad4dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.135+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124016070 con 0x55a14930d390 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.135+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118005020 con 0x55a14930d390 2024-09-10T22:50:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.135+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f812400dd20 con 0x55a14930d390 2024-09-10T22:50:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.136+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f812401f050 con 0x55a14930d390 2024-09-10T22:50:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.138+0000 7f81331eb740 10 client.16035 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.138+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a149046620 con 0x55a14930d390 2024-09-10T22:50:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.138+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118005020 con 0x55a14930d390 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8124022020 con 0x55a14930d390 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 10 client.16035 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 20 client.16035 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14931fbf0 0x55a149322010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 4 client.16035 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a14927e640 con 0x55a14931fbf0 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.139+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14931fbf0 0x55a149322010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.140+0000 7f8111ffb640 10 client.16035 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.140+0000 7f812d9b1640 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14931fbf0 0x55a149322010 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10d8a4edbb3cec3 server_cookie=f822237bb20a8681 in_seq=0 out_seq=0 2024-09-10T22:50:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.141+0000 7f8111ffb640 1 -- 172.21.15.96:0/2481059695 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a14927e640 con 0x55a14931fbf0 2024-09-10T22:50:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.141+0000 7f8111ffb640 10 client.16035 handle_command_reply: tid=0 2024-09-10T22:50:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.144+0000 7f81331eb740 1 client.16035 shutdown 2024-09-10T22:50:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.145+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 msgr2=0x55a149314880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.145+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a149314880 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f812400bb40 tx=0x7f812400d2e0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 shutdown_connections 2024-09-10T22:50:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14931fbf0 0x55a149322010 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930fdd0 0x55a149315450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a149314880 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 --2- 172.21.15.96:0/2481059695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.147+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 >> 172.21.15.96:0/2481059695 conn(0x55a149305bb0 msgr2=0x55a149304c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.149+0000 7f8111ffb640 0 client.16035 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.149+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 shutdown_connections 2024-09-10T22:50:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.150+0000 7f81331eb740 1 -- 172.21.15.96:0/2481059695 wait complete. 2024-09-10T22:50:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.152+0000 7f81331eb740 20 client.16035 trim_cache size 0 max 16384 2024-09-10T22:50:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.194+0000 7f81331eb740 1 Processor -- start 2024-09-10T22:50:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.194+0000 7f81331eb740 1 -- start start 2024-09-10T22:50:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930d390 0x55a14931bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a149314720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f810c00e610 con 0x55a14930aa60 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f810c010150 con 0x55a14930d390 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f81331eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f810c00ece0 con 0x55a14930fdd0 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42970/0 (socket says 172.21.15.96:42970) 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f812d9b1640 1 -- 172.21.15.96:0/1437810426 learned_addr learned my addr 172.21.15.96:0/1437810426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.195+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a149314720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 -- 172.21.15.96:0/1437810426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930d390 msgr2=0x55a14931bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930d390 0x55a14931bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 -- 172.21.15.96:0/1437810426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 msgr2=0x55a1493141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 -- 172.21.15.96:0/1437810426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14930faf0 con 0x55a14930fdd0 2024-09-10T22:50:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.196+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a149314720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8118012890 tx=0x7f811800bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6aee5cbee292d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.199+0000 7f8113fff640 1 -- 172.21.15.96:0/1437810426 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004020 con 0x55a14930fdd0 2024-09-10T22:50:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.199+0000 7f8113fff640 1 -- 172.21.15.96:0/1437810426 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118010bf0 con 0x55a14930fdd0 2024-09-10T22:50:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.199+0000 7f8113fff640 1 -- 172.21.15.96:0/1437810426 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180036e0 con 0x55a14930fdd0 2024-09-10T22:50:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.200+0000 7f81331eb740 1 -- 172.21.15.96:0/1437810426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 msgr2=0x55a149314720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.200+0000 7f81331eb740 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a149314720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8118012890 tx=0x7f811800bd00 comp rx=0 tx=0).stop 2024-09-10T22:50:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.207+0000 7f81331eb740 1 -- 172.21.15.96:0/1437810426 shutdown_connections 2024-09-10T22:50:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.207+0000 7f81331eb740 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930d390 0x55a14931bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.207+0000 7f81331eb740 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a149314720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.207+0000 7f81331eb740 1 --2- 172.21.15.96:0/1437810426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930aa60 0x55a1493141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.207+0000 7f81331eb740 1 -- 172.21.15.96:0/1437810426 >> 172.21.15.96:0/1437810426 conn(0x55a149305bb0 msgr2=0x55a1493101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.212+0000 7f81331eb740 1 -- 172.21.15.96:0/1437810426 shutdown_connections 2024-09-10T22:50:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.214+0000 7f81331eb740 1 -- 172.21.15.96:0/1437810426 wait complete. 2024-09-10T22:50:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 -- start start 2024-09-10T22:50:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 Processor -- start 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a1493196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a14931a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f810c00fa80 con 0x55a14930d390 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f810c00f3b0 con 0x55a14930aa60 2024-09-10T22:50:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f810c00c590 con 0x55a14930fdd0 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.215+0000 7f81331eb740 10 client.0 _ll_register_callbacks cb 0x7f8124005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38324/0 (socket says 172.21.15.96:38324) 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 -- 172.21.15.96:0/1406706333 learned_addr learned my addr 172.21.15.96:0/1406706333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a14931a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 -- 172.21.15.96:0/1406706333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 msgr2=0x55a14931a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a14931a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 -- 172.21.15.96:0/1406706333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 msgr2=0x55a1493196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a1493196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.217+0000 7f812d9b1640 1 -- 172.21.15.96:0/1406706333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a14930d1c0 con 0x55a14930aa60 2024-09-10T22:50:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.218+0000 7f812d9b1640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8118010a20 tx=0x7f8118005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e51807d38512698 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.218+0000 7f8111ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:50:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.218+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118031680 con 0x55a14930aa60 2024-09-10T22:50:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.218+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8118031820 con 0x55a14930aa60 2024-09-10T22:50:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.218+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118010bf0 con 0x55a14930aa60 2024-09-10T22:50:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.219+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8118031a20 con 0x55a14930aa60 2024-09-10T22:50:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.220+0000 7f81331eb740 10 client.25583 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.220+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a14931bcb0 con 0x55a14930aa60 2024-09-10T22:50:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.225+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8118010bf0 con 0x55a14930aa60 2024-09-10T22:50:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.225+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8118004c60 con 0x55a14930aa60 2024-09-10T22:50:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 10 client.25583 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 20 client.25583 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14938cae0 0x55a14938eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 4 client.25583 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:50:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.226+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a14927a0f0 con 0x55a14938cae0 2024-09-10T22:50:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.230+0000 7f812e1b2640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14938cae0 0x55a14938eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.235+0000 7f812e1b2640 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14938cae0 0x55a14938eed0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1b7e9bbdbde088 server_cookie=deb39de849bd13a2 in_seq=0 out_seq=0 2024-09-10T22:50:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.236+0000 7f8111ffb640 10 client.25583 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.309+0000 7f8111ffb640 1 -- 172.21.15.96:0/1406706333 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a14927a0f0 con 0x55a14938cae0 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.309+0000 7f8111ffb640 10 client.25583 handle_command_reply: tid=0 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.309+0000 7f81331eb740 1 client.25583 shutdown 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 msgr2=0x55a149319040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8118010a20 tx=0x7f8118005970 comp rx=0 tx=0).stop 2024-09-10T22:50:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 shutdown_connections 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a14938cae0 0x55a14938eed0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a14930fdd0 0x55a14931a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a14930d390 0x55a1493196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f8111ffb640 0 client.25583 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 --2- 172.21.15.96:0/1406706333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a14930aa60 0x55a149319040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 >> 172.21.15.96:0/1406706333 conn(0x55a149305bb0 msgr2=0x55a149309b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 shutdown_connections 2024-09-10T22:50:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 1 -- 172.21.15.96:0/1406706333 wait complete. 2024-09-10T22:50:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.310+0000 7f81331eb740 20 client.25583 trim_cache size 0 max 16384 2024-09-10T22:50:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:50:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:50:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:50:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:50:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 msgr2=0x7f81140b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 0x7f81140b79a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81200cebf0 tx=0x7f812002a8d0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 msgr2=0x7f8128119910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 0x7f8128119910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f812002aec0 tx=0x7f8120003f10 comp rx=0 tx=0).stop 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 shutdown_connections 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f812810ad00 0x7f8128119910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81140b54e0 0x7f81140b79a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f812810a3c0 0x7f81281193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 --2- 172.21.15.96:0/827671328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128074500 0x7f8128118e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 >> 172.21.15.96:0/827671328 conn(0x7f812806f850 msgr2=0x7f8128070450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.311+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 shutdown_connections 2024-09-10T22:50:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:27.312+0000 7f812f1b4640 1 -- 172.21.15.96:0/827671328 wait complete. 2024-09-10T22:50:27.361 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:27.504 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:27 smithi096 ceph-mon[30869]: pgmap v441: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-09-10T22:50:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:27 smithi121 ceph-mon[35119]: pgmap v441: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-09-10T22:50:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:27 smithi117 ceph-mon[35081]: pgmap v441: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 770 op/s 2024-09-10T22:50:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/1008032379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b0108020 msgr2=0x7fe8b010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/1008032379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b0108020 0x7fe8b010e820 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009ca0 tx=0x7fe8a802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/1008032379 shutdown_connections 2024-09-10T22:50:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/1008032379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b0108020 0x7fe8b010e820 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/1008032379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b01076a0 0x7fe8b0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/1008032379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.937+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/1008032379 >> 172.21.15.96:0/1008032379 conn(0x7fe8b00fd700 msgr2=0x7fe8b00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:27.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.938+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/1008032379 shutdown_connections 2024-09-10T22:50:27.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.938+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/1008032379 wait complete. 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.938+0000 7fe8b6ce4640 1 Processor -- start 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.938+0000 7fe8b6ce4640 1 -- start start 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b01076a0 0x7fe8b019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b0108020 0x7fe8b01a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0112dd0 con 0x7fe8b01076a0 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0112c50 con 0x7fe8b0101c90 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b6ce4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0112f50 con 0x7fe8b0108020 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b01076a0 0x7fe8b019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b5281640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b0108020 0x7fe8b01a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38344/0 (socket says 172.21.15.96:38344) 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 -- 172.21.15.96:0/4211790166 learned_addr learned my addr 172.21.15.96:0/4211790166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 -- 172.21.15.96:0/4211790166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b0108020 msgr2=0x7fe8b01a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b0108020 0x7fe8b01a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 -- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b01076a0 msgr2=0x7fe8b019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b01076a0 0x7fe8b019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 -- 172.21.15.96:0/4211790166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01a15b0 con 0x7fe8b0101c90 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.939+0000 7fe8b4a80640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9f0 tx=0x7fe8a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.940+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400cd80 con 0x7fe8b0101c90 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.940+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a40108e0 con 0x7fe8b0101c90 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.940+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4017820 con 0x7fe8b0101c90 2024-09-10T22:50:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.940+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b01a18a0 con 0x7fe8b0101c90 2024-09-10T22:50:27.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.940+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a8130 con 0x7fe8b0101c90 2024-09-10T22:50:27.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.943+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b0102070 con 0x7fe8b0101c90 2024-09-10T22:50:27.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.944+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe8a4010430 con 0x7fe8b0101c90 2024-09-10T22:50:27.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.945+0000 7fe8adffb640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 0x7fe8a00b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:27.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.945+0000 7fe8affff640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 0x7fe8a00b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:27.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.946+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe8a40e61d0 con 0x7fe8b0101c90 2024-09-10T22:50:27.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.946+0000 7fe8affff640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 0x7fe8a00b7860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0103de0 tx=0x7fe89c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:27.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:27.949+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe8a40ad130 con 0x7fe8b0101c90 2024-09-10T22:50:28.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.295+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b01a8390 con 0x7fe8b0101c90 2024-09-10T22:50:28.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.296+0000 7fe8adffb640 1 -- 172.21.15.96:0/4211790166 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fe8a40ad130 con 0x7fe8b0101c90 2024-09-10T22:50:28.298 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:50:28.298 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:50:28.299 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 msgr2=0x7fe8a00b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 0x7fe8a00b7860 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0103de0 tx=0x7fe89c005f90 comp rx=0 tx=0).stop 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 msgr2=0x7fe8b019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e9f0 tx=0x7fe8a400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 shutdown_connections 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe8a00b53a0 0x7fe8a00b7860 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8b0108020 0x7fe8b01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8b01076a0 0x7fe8b019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 --2- 172.21.15.96:0/4211790166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b0101c90 0x7fe8b019b500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:28.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.299+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 >> 172.21.15.96:0/4211790166 conn(0x7fe8b00fd700 msgr2=0x7fe8b010ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:28.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.300+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 shutdown_connections 2024-09-10T22:50:28.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:28.300+0000 7fe8b6ce4640 1 -- 172.21.15.96:0/4211790166 wait complete. 2024-09-10T22:50:28.705 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:28 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4211790166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:28 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4211790166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:28 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4211790166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:50:29.140 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:50:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:29 smithi096 ceph-mon[30869]: pgmap v442: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T22:50:29.529 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:29 smithi121 ceph-mon[35119]: pgmap v442: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T22:50:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:29 smithi117 ceph-mon[35081]: pgmap v442: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-10T22:50:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 175.0 s, 3172.4 tps, lat 31.580 ms stddev 37.423 2024-09-10T22:50:31.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.108+0000 7fe6823b8640 1 -- 172.21.15.96:0/624805258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 msgr2=0x7fe67c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.108+0000 7fe6823b8640 1 --2- 172.21.15.96:0/624805258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c108bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe664009860 tx=0x7fe66402a340 comp rx=0 tx=0).stop 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 -- 172.21.15.96:0/624805258 shutdown_connections 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 --2- 172.21.15.96:0/624805258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c109130 0x7fe67c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 --2- 172.21.15.96:0/624805258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c108bf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 --2- 172.21.15.96:0/624805258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c102da0 0x7fe67c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 -- 172.21.15.96:0/624805258 >> 172.21.15.96:0/624805258 conn(0x7fe67c0fe850 msgr2=0x7fe67c100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 -- 172.21.15.96:0/624805258 shutdown_connections 2024-09-10T22:50:31.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.110+0000 7fe6823b8640 1 -- 172.21.15.96:0/624805258 wait complete. 2024-09-10T22:50:31.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.111+0000 7fe6823b8640 1 Processor -- start 2024-09-10T22:50:31.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.111+0000 7fe6823b8640 1 -- start start 2024-09-10T22:50:31.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.111+0000 7fe6823b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.111+0000 7fe6823b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.111+0000 7fe6823b8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c109130 0x7fe67c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe6823b8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c112df0 con 0x7fe67c1087b0 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe6823b8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c112c70 con 0x7fe67c102da0 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38358/0 (socket says 172.21.15.96:38358) 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe67bfff640 1 -- 172.21.15.96:0/1603330466 learned_addr learned my addr 172.21.15.96:0/1603330466 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c112f70 con 0x7fe67c109130 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe680955640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c109130 0x7fe67c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 -- 172.21.15.96:0/1603330466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c109130 msgr2=0x7fe67c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c109130 0x7fe67c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 -- 172.21.15.96:0/1603330466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 msgr2=0x7fe67c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.112+0000 7fe673fff640 1 -- 172.21.15.96:0/1603330466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c19f150 con 0x7fe67c1087b0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.113+0000 7fe673fff640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c1995e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe66402a860 tx=0x7fe664030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.113+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.113+0000 7fe679ffb640 1 -- 172.21.15.96:0/1603330466 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664040020 con 0x7fe67c1087b0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.113+0000 7fe679ffb640 1 -- 172.21.15.96:0/1603330466 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe664030900 con 0x7fe67c1087b0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.113+0000 7fe679ffb640 1 -- 172.21.15.96:0/1603330466 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664037040 con 0x7fe67c1087b0 2024-09-10T22:50:31.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.114+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c19f3e0 con 0x7fe67c1087b0 2024-09-10T22:50:31.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.114+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c19f780 con 0x7fe67c1087b0 2024-09-10T22:50:31.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.117+0000 7fe68478d740 1 Processor -- start 2024-09-10T22:50:31.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.117+0000 7fe68478d740 1 -- start start 2024-09-10T22:50:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ca410eea0 con 0x555ca41980f0 2024-09-10T22:50:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ca40fe2d0 con 0x555ca41a49a0 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe68478d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555ca41020e0 con 0x555ca4198a10 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52796/0 (socket says 172.21.15.96:52796) 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 -- 172.21.15.96:0/1018642757 learned_addr learned my addr 172.21.15.96:0/1018642757 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe680955640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe679ffb640 1 -- 172.21.15.96:0/1603330466 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe664003a50 con 0x7fe67c1087b0 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe673fff640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 -- 172.21.15.96:0/1018642757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 msgr2=0x555ca41a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 -- 172.21.15.96:0/1018642757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 msgr2=0x555ca41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.118+0000 7fe67bfff640 1 -- 172.21.15.96:0/1018642757 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca3e5c650 con 0x555ca4198a10 2024-09-10T22:50:31.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.119+0000 7fe680955640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:31.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.119+0000 7fe673fff640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:31.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.119+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009ce0 tx=0x7fe66c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc424961d6ba3138 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.119+0000 7fe679ffb640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 0x7fe6680b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.120+0000 7fe679ffb640 1 -- 172.21.15.96:0/1603330466 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe664105e80 con 0x7fe67c1087b0 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.120+0000 7fe680955640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 0x7fe6680b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.121+0000 7fe680955640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 0x7fe6680b7970 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe67400dd10 tx=0x7fe67400f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.121+0000 7fe6727fc640 1 -- 172.21.15.96:0/1018642757 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c004020 con 0x555ca4198a10 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.121+0000 7fe6727fc640 1 -- 172.21.15.96:0/1018642757 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66c0041c0 con 0x555ca4198a10 2024-09-10T22:50:31.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.121+0000 7fe6727fc640 1 -- 172.21.15.96:0/1018642757 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c012740 con 0x555ca4198a10 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 -- 172.21.15.96:0/1018642757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 msgr2=0x555ca41a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009ce0 tx=0x7fe66c010740 comp rx=0 tx=0).stop 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 -- 172.21.15.96:0/1018642757 shutdown_connections 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca41a4460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 --2- 172.21.15.96:0/1018642757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.122+0000 7fe68478d740 1 -- 172.21.15.96:0/1018642757 >> 172.21.15.96:0/1018642757 conn(0x555ca4195f20 msgr2=0x555ca41970c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.123+0000 7fe68478d740 1 -- 172.21.15.96:0/1018642757 shutdown_connections 2024-09-10T22:50:31.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.123+0000 7fe68478d740 1 -- 172.21.15.96:0/1018642757 wait complete. 2024-09-10T22:50:31.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.123+0000 7fe68478d740 1 -- start start 2024-09-10T22:50:31.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.124+0000 7fe68478d740 1 Processor -- start 2024-09-10T22:50:31.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.124+0000 7fe68478d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.124+0000 7fe68478d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.124+0000 7fe68478d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe68478d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ca3f5f700 con 0x555ca4198a10 2024-09-10T22:50:31.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52812/0 (socket says 172.21.15.96:52812) 2024-09-10T22:50:31.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 -- 172.21.15.96:0/1678801252 learned_addr learned my addr 172.21.15.96:0/1678801252 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:31.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ca4099160 con 0x555ca41a49a0 2024-09-10T22:50:31.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555ca41a7620 con 0x555ca41980f0 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe680955640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe673fff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe68478d740 10 client.0 _ll_register_callbacks cb 0x555ca40b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 -- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 msgr2=0x555ca41a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 -- 172.21.15.96:0/1678801252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 msgr2=0x555ca41a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca40c6c90 con 0x555ca41980f0 2024-09-10T22:50:31.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe680955640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:31.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66c0041c0 con 0x555ca41980f0 2024-09-10T22:50:31.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe67bfff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe66c000c00 tx=0x7fe66c003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382ee3cabc1f769c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.125+0000 7fe65b7fe640 10 client.25642 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:31.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.126+0000 7fe65b7fe640 1 -- 172.21.15.96:0/1678801252 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c029060 con 0x555ca41980f0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.126+0000 7fe65b7fe640 1 -- 172.21.15.96:0/1678801252 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe66c009820 con 0x555ca41980f0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.126+0000 7fe65b7fe640 1 -- 172.21.15.96:0/1678801252 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe66c005040 con 0x555ca41980f0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.126+0000 7fe68478d740 10 client.25642 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.126+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x555ca41b2d40 con 0x555ca41980f0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe65b7fe640 1 -- 172.21.15.96:0/1678801252 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe66c00f040 con 0x555ca41980f0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 10 client.25642 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 10 client.25642 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 20 client.25642 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca41b4090 0x555ca41b6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 4 client.25642 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.127+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca4110df0 con 0x555ca41b4090 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.128+0000 7fe673fff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca41b4090 0x555ca41b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.128+0000 7fe673fff640 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca41b4090 0x555ca41b6480 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=589b78ec2bd35b5a server_cookie=af53b8bf85c026e3 in_seq=0 out_seq=0 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe65b7fe640 10 client.25642 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe65b7fe640 1 -- 172.21.15.96:0/1678801252 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ca4110df0 con 0x555ca41b4090 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe65b7fe640 10 client.25642 handle_command_reply: tid=0 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe68478d740 1 client.25642 shutdown 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 msgr2=0x555ca41a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.129+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe66c000c00 tx=0x7fe66c003b80 comp rx=0 tx=0).stop 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 shutdown_connections 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca41b4090 0x555ca41b6480 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca41a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 --2- 172.21.15.96:0/1678801252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41980f0 0x555ca41a4210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 >> 172.21.15.96:0/1678801252 conn(0x555ca4195f20 msgr2=0x555ca419ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe65b7fe640 0 client.25642 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:50:31.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 shutdown_connections 2024-09-10T22:50:31.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 1 -- 172.21.15.96:0/1678801252 wait complete. 2024-09-10T22:50:31.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.130+0000 7fe68478d740 20 client.25642 trim_cache size 0 max 16384 2024-09-10T22:50:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.153+0000 7fe68478d740 1 Processor -- start 2024-09-10T22:50:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.153+0000 7fe68478d740 1 -- start start 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca4195760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca419c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1abf70 con 0x555ca41980f0 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1adab0 con 0x555ca41a49a0 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe68478d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1ac640 con 0x555ca4198a10 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe680955640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca4195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe680955640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38372/0 (socket says 172.21.15.96:38372) 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe680955640 1 -- 172.21.15.96:0/2974808677 learned_addr learned my addr 172.21.15.96:0/2974808677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:31.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.154+0000 7fe673fff640 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 -- 172.21.15.96:0/2974808677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 msgr2=0x555ca419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 -- 172.21.15.96:0/2974808677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 msgr2=0x555ca4195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca4195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 -- 172.21.15.96:0/2974808677 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca41a1f50 con 0x555ca41a49a0 2024-09-10T22:50:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe680955640 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe67401a520 tx=0x7fe674019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e616e8083d1737 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe671ffb640 1 -- 172.21.15.96:0/2974808677 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674024410 con 0x555ca41a49a0 2024-09-10T22:50:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe671ffb640 1 -- 172.21.15.96:0/2974808677 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe674024a60 con 0x555ca41a49a0 2024-09-10T22:50:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.155+0000 7fe671ffb640 1 -- 172.21.15.96:0/2974808677 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740189f0 con 0x555ca41a49a0 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 -- 172.21.15.96:0/2974808677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 msgr2=0x555ca419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe67401a520 tx=0x7fe674019000 comp rx=0 tx=0).stop 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 -- 172.21.15.96:0/2974808677 shutdown_connections 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41a49a0 0x555ca419ce50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca4198a10 0x555ca419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 --2- 172.21.15.96:0/2974808677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca41980f0 0x555ca4195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.156+0000 7fe68478d740 1 -- 172.21.15.96:0/2974808677 >> 172.21.15.96:0/2974808677 conn(0x555ca4195f20 msgr2=0x555ca41b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.157+0000 7fe68478d740 1 -- 172.21.15.96:0/2974808677 shutdown_connections 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.157+0000 7fe68478d740 1 -- 172.21.15.96:0/2974808677 wait complete. 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.157+0000 7fe68478d740 1 -- start start 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.158+0000 7fe68478d740 1 Processor -- start 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.158+0000 7fe68478d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41980f0 0x555ca41acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1ad3e0 con 0x555ca4198a10 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1acd10 con 0x555ca41980f0 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1a9f70 con 0x555ca41a49a0 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe68478d740 10 client.0 _ll_register_callbacks cb 0x7fe66c009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.160+0000 7fe680955640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.160+0000 7fe680955640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52834/0 (socket says 172.21.15.96:52834) 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.160+0000 7fe680955640 1 -- 172.21.15.96:0/3931493329 learned_addr learned my addr 172.21.15.96:0/3931493329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.160+0000 7fe673fff640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.159+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41980f0 0x555ca41acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 -- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41980f0 msgr2=0x555ca41acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41980f0 0x555ca41acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 -- 172.21.15.96:0/3931493329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 msgr2=0x555ca41a83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 -- 172.21.15.96:0/3931493329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca41a8fd0 con 0x555ca41a49a0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe680955640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe67401a4f0 tx=0x7fe67400b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d2fcec266cb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.161+0000 7fe65b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.162+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674033020 con 0x555ca41a49a0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.162+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe674024a60 con 0x555ca41a49a0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.162+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe674008040 con 0x555ca41a49a0 2024-09-10T22:50:31.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.162+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe674024410 con 0x555ca41a49a0 2024-09-10T22:50:31.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.164+0000 7fe68478d740 10 client.25648 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:31.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.164+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ca4229390 con 0x555ca41a49a0 2024-09-10T22:50:31.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe674024a60 con 0x555ca41a49a0 2024-09-10T22:50:31.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe67402e450 con 0x555ca41a49a0 2024-09-10T22:50:31.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 10 client.25648 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:31.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 10 client.25648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:50:31.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 20 client.25648 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:31.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca4229a00 0x555ca422bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:31.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 4 client.25648 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:50:31.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ca410c8a0 con 0x555ca4229a00 2024-09-10T22:50:31.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.169+0000 7fe67bfff640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca4229a00 0x555ca422bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:31.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.170+0000 7fe65b7fe640 10 client.25648 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:50:31.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.170+0000 7fe67bfff640 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca4229a00 0x555ca422bdf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d82839b5112bbb88 server_cookie=655d8bec2ba6ab7 in_seq=0 out_seq=0 2024-09-10T22:50:31.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.179+0000 7fe65b7fe640 1 -- 172.21.15.96:0/3931493329 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ca410c8a0 con 0x555ca4229a00 2024-09-10T22:50:31.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.179+0000 7fe65b7fe640 10 client.25648 handle_command_reply: tid=0 2024-09-10T22:50:31.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.180+0000 7fe68478d740 1 client.25648 shutdown 2024-09-10T22:50:31.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.182+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 msgr2=0x555ca41a8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.182+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe67401a4f0 tx=0x7fe67400b420 comp rx=0 tx=0).stop 2024-09-10T22:50:31.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 shutdown_connections 2024-09-10T22:50:31.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x555ca4229a00 0x555ca422bdf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ca41a49a0 0x555ca41a8930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ca4198a10 0x555ca41a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe65b7fe640 0 client.25648 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:50:31.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 --2- 172.21.15.96:0/3931493329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ca41980f0 0x555ca41acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.186+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 >> 172.21.15.96:0/3931493329 conn(0x555ca4195f20 msgr2=0x555ca4196f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.188+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 shutdown_connections 2024-09-10T22:50:31.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.188+0000 7fe68478d740 1 -- 172.21.15.96:0/3931493329 wait complete. 2024-09-10T22:50:31.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.189+0000 7fe68478d740 20 client.25648 trim_cache size 0 max 16384 2024-09-10T22:50:31.192 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:50:31.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 msgr2=0x7fe6680b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 0x7fe6680b7970 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe67400dd10 tx=0x7fe67400f040 comp rx=0 tx=0).stop 2024-09-10T22:50:31.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 msgr2=0x7fe67c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:31.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c1995e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe66402a860 tx=0x7fe664030e00 comp rx=0 tx=0).stop 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 shutdown_connections 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6680b54b0 0x7fe6680b7970 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe67c109130 0x7fe67c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67c1087b0 0x7fe67c1995e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 --2- 172.21.15.96:0/1603330466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe67c102da0 0x7fe67c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.190+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 >> 172.21.15.96:0/1603330466 conn(0x7fe67c0fe850 msgr2=0x7fe67c106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.191+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 shutdown_connections 2024-09-10T22:50:31.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:31.191+0000 7fe6823b8640 1 -- 172.21.15.96:0/1603330466 wait complete. 2024-09-10T22:50:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:31 smithi096 ceph-mon[30869]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 829 op/s 2024-09-10T22:50:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:31 smithi121 ceph-mon[35119]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 829 op/s 2024-09-10T22:50:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:31 smithi117 ceph-mon[35081]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 829 op/s 2024-09-10T22:50:32.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:50:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:50:32.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:32.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:32.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:32.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:50:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:50:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:50:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:50:32.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:50:32.849 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.494+0000 7ff07b73b640 1 -- 172.21.15.96:0/1306905309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074108020 msgr2=0x7ff07410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.494+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1306905309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074108020 0x7ff07410e820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff068009900 tx=0x7ff06802a440 comp rx=0 tx=0).stop 2024-09-10T22:50:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.496+0000 7ff07b73b640 1 -- 172.21.15.96:0/1306905309 shutdown_connections 2024-09-10T22:50:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.496+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1306905309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0741076a0 0x7ff074107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.496+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1306905309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074108020 0x7ff07410e820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.496+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1306905309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074101c90 0x7ff074102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.496+0000 7ff07b73b640 1 -- 172.21.15.96:0/1306905309 >> 172.21.15.96:0/1306905309 conn(0x7ff0740fd700 msgr2=0x7ff0740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.497+0000 7ff07b73b640 1 -- 172.21.15.96:0/1306905309 shutdown_connections 2024-09-10T22:50:33.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.497+0000 7ff07b73b640 1 -- 172.21.15.96:0/1306905309 wait complete. 2024-09-10T22:50:33.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.498+0000 7ff07b73b640 1 Processor -- start 2024-09-10T22:50:33.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff07b73b640 1 -- start start 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff07b73b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 0x7ff074112370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff07b73b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0741076a0 0x7ff0741128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff07b73b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff07b73b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff074110bb0 con 0x7ff074108020 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.499+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43034/0 (socket says 172.21.15.96:43034) 2024-09-10T22:50:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff079cd8640 1 -- 172.21.15.96:0/1613328510 learned_addr learned my addr 172.21.15.96:0/1613328510 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff074110a30 con 0x7ff0741076a0 2024-09-10T22:50:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff074110d30 con 0x7ff074101c90 2024-09-10T22:50:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 0x7ff074112370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff0794d7640 1 -- 172.21.15.96:0/1613328510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0741076a0 msgr2=0x7ff0741128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.500+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0741076a0 0x7ff0741128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.501+0000 7ff0794d7640 1 -- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 msgr2=0x7ff074117c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.501+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.501+0000 7ff0794d7640 1 -- 172.21.15.96:0/1613328510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0741183d0 con 0x7ff074101c90 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.501+0000 7ff079cd8640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.501+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 0x7ff074112370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00d6d0 tx=0x7ff06c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.505+0000 7ff0727fc640 1 -- 172.21.15.96:0/1613328510 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c017070 con 0x7ff074101c90 2024-09-10T22:50:33.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.505+0000 7ff0727fc640 1 -- 172.21.15.96:0/1613328510 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c009820 con 0x7ff074101c90 2024-09-10T22:50:33.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.505+0000 7ff0727fc640 1 -- 172.21.15.96:0/1613328510 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c004c40 con 0x7ff074101c90 2024-09-10T22:50:33.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.506+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0741186c0 con 0x7ff074101c90 2024-09-10T22:50:33.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.506+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07411cde0 con 0x7ff074101c90 2024-09-10T22:50:33.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 Processor -- start 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 -- start start 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da700840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da703170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565da66aea0 con 0x5565da700d80 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565da65a2d0 con 0x5565da6f4df0 2024-09-10T22:50:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.510+0000 7ff07db10740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565da65e0e0 con 0x5565da6f44d0 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52864/0 (socket says 172.21.15.96:52864) 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 -- 172.21.15.96:0/2520716679 learned_addr learned my addr 172.21.15.96:0/2520716679 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff078cd6640 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da703170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 -- 172.21.15.96:0/2520716679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 msgr2=0x5565da700840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da700840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 -- 172.21.15.96:0/2520716679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 msgr2=0x5565da703170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da703170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff079cd8640 1 -- 172.21.15.96:0/2520716679 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565da3b8650 con 0x5565da6f44d0 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.511+0000 7ff078cd6640 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da703170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.512+0000 7ff079cd8640 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff068004310 tx=0x7ff068004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1d18e3e2284ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.512+0000 7ff05b7fe640 1 -- 172.21.15.96:0/2520716679 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068040070 con 0x5565da6f44d0 2024-09-10T22:50:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.512+0000 7ff05b7fe640 1 -- 172.21.15.96:0/2520716679 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0680046c0 con 0x5565da6f44d0 2024-09-10T22:50:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.513+0000 7ff05b7fe640 1 -- 172.21.15.96:0/2520716679 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068035030 con 0x5565da6f44d0 2024-09-10T22:50:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.513+0000 7ff0727fc640 1 -- 172.21.15.96:0/1613328510 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff06c010460 con 0x7ff074101c90 2024-09-10T22:50:33.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.513+0000 7ff07db10740 1 -- 172.21.15.96:0/2520716679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 msgr2=0x5565da6f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.513+0000 7ff07db10740 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff068004310 tx=0x7ff068004340 comp rx=0 tx=0).stop 2024-09-10T22:50:33.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.514+0000 7ff0727fc640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 0x7ff05c0b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.514+0000 7ff0727fc640 1 -- 172.21.15.96:0/1613328510 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff06c0e5420 con 0x7ff074101c90 2024-09-10T22:50:33.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.515+0000 7ff079cd8640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 0x7ff05c0b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 -- 172.21.15.96:0/2520716679 shutdown_connections 2024-09-10T22:50:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da700840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da703170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 --2- 172.21.15.96:0/2520716679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da6f48b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 -- 172.21.15.96:0/2520716679 >> 172.21.15.96:0/2520716679 conn(0x5565da6f2300 msgr2=0x5565da6f34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff079cd8640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 0x7ff05c0b7930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff068004650 tx=0x7ff06802a440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.516+0000 7ff07db10740 1 -- 172.21.15.96:0/2520716679 shutdown_connections 2024-09-10T22:50:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.517+0000 7ff07db10740 1 -- 172.21.15.96:0/2520716679 wait complete. 2024-09-10T22:50:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.517+0000 7ff07db10740 1 -- start start 2024-09-10T22:50:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.517+0000 7ff07db10740 1 Processor -- start 2024-09-10T22:50:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da7006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da6fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565da4bb700 con 0x5565da700d80 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565da5f5160 con 0x5565da6f4df0 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565da714a50 con 0x5565da6f44d0 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff07db10740 10 client.0 _ll_register_callbacks cb 0x5565da60e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff079cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43062/0 (socket says 172.21.15.96:43062) 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff079cd8640 1 -- 172.21.15.96:0/1760323786 learned_addr learned my addr 172.21.15.96:0/1760323786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da7006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.518+0000 7ff078cd6640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da6fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 -- 172.21.15.96:0/1760323786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 msgr2=0x5565da6fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da6fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 -- 172.21.15.96:0/1760323786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 msgr2=0x5565da6fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 -- 172.21.15.96:0/1760323786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565da622c90 con 0x5565da6f44d0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff079cd8640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff0794d7640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da7006b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0e3010 tx=0x7ff06c0a73f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a4976ccb1079ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c020040 con 0x5565da6f44d0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff06c0a7ef0 con 0x5565da6f44d0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0680046c0 con 0x5565da6f44d0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff06c0abc80 con 0x5565da6f44d0 2024-09-10T22:50:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff07db10740 10 client.25660 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565da720380 con 0x5565da6f44d0 2024-09-10T22:50:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.519+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0680046c0 con 0x5565da6f44d0 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff06c0a77d0 con 0x5565da6f44d0 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 10 client.25660 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 10 client.25660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 20 client.25660 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da721840 0x5565da723c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 4 client.25660 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565da66cdf0 con 0x5565da721840 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.520+0000 7ff078cd6640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da721840 0x5565da723c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.521+0000 7ff078cd6640 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da721840 0x5565da723c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e447268f65500ea0 server_cookie=eba111e888bcea8a in_seq=0 out_seq=0 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.521+0000 7ff058ff9640 10 client.25660 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff058ff9640 1 -- 172.21.15.96:0/1760323786 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565da66cdf0 con 0x5565da721840 2024-09-10T22:50:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff058ff9640 10 client.25660 handle_command_reply: tid=0 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 client.25660 shutdown 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 msgr2=0x5565da7006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da7006b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0e3010 tx=0x7ff06c0a73f0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 shutdown_connections 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da721840 0x5565da723c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da6fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da700d80 0x5565da6fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 --2- 172.21.15.96:0/1760323786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da6f44d0 0x5565da7006b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 >> 172.21.15.96:0/1760323786 conn(0x5565da6f2300 msgr2=0x5565da6f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff058ff9640 0 client.25660 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.522+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 shutdown_connections 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.523+0000 7ff07db10740 1 -- 172.21.15.96:0/1760323786 wait complete. 2024-09-10T22:50:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.523+0000 7ff07db10740 20 client.25660 trim_cache size 0 max 16384 2024-09-10T22:50:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.542+0000 7ff07db10740 1 Processor -- start 2024-09-10T22:50:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.542+0000 7ff07db10740 1 -- start start 2024-09-10T22:50:33.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f4df0 0x5565da6fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff0794d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da702170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff0794d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38404/0 (socket says 172.21.15.96:38404) 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff0794d7640 1 -- 172.21.15.96:0/2341784094 learned_addr learned my addr 172.21.15.96:0/2341784094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff074073df0 con 0x5565da6f4df0 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741b6320 con 0x5565da6f44d0 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740744c0 con 0x5565da700d80 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.543+0000 7ff079cd8640 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da702170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 -- 172.21.15.96:0/2341784094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 msgr2=0x5565da702170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da702170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 -- 172.21.15.96:0/2341784094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f4df0 msgr2=0x5565da6fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f4df0 0x5565da6fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 -- 172.21.15.96:0/2341784094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565da6f2d80 con 0x5565da6f44d0 2024-09-10T22:50:33.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0e5740 tx=0x7ff06c0a7520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4abcfeb0d0f743c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff05affd640 1 -- 172.21.15.96:0/2341784094 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0c5020 con 0x5565da6f44d0 2024-09-10T22:50:33.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff05affd640 1 -- 172.21.15.96:0/2341784094 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c0ab790 con 0x5565da6f44d0 2024-09-10T22:50:33.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.544+0000 7ff05affd640 1 -- 172.21.15.96:0/2341784094 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c020070 con 0x5565da6f44d0 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 msgr2=0x5565da6fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0e5740 tx=0x7ff06c0a7520 comp rx=0 tx=0).stop 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 shutdown_connections 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da702170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f4df0 0x5565da6fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 --2- 172.21.15.96:0/2341784094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f44d0 0x5565da6fc380 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 >> 172.21.15.96:0/2341784094 conn(0x5565da6f2300 msgr2=0x5565da6f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 shutdown_connections 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.545+0000 7ff07db10740 1 -- 172.21.15.96:0/2341784094 wait complete. 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.546+0000 7ff07db10740 1 -- start start 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.546+0000 7ff07db10740 1 Processor -- start 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.546+0000 7ff07db10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.546+0000 7ff07db10740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.546+0000 7ff07db10740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da714ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff07db10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0741b5d70 con 0x5565da6f44d0 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff07db10740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff074074b90 con 0x5565da6f4df0 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff07db10740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff074071dd0 con 0x5565da700d80 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff07db10740 10 client.0 _ll_register_callbacks cb 0x7ff06c0057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff078cd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43088/0 (socket says 172.21.15.96:43088) 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff078cd6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38408/0 (socket says 172.21.15.96:38408) 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 -- 172.21.15.96:0/2465290369 learned_addr learned my addr 172.21.15.96:0/2465290369 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff079cd8640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da714ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 -- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 msgr2=0x5565da714ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da714ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 -- 172.21.15.96:0/2465290369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 msgr2=0x5565da71a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 -- 172.21.15.96:0/2465290369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565da71a9a0 con 0x5565da6f44d0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff078cd6640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff0794d7640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0abc10 tx=0x7ff06c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31754c0951fb5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.547+0000 7ff058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0c5070 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff06c0a77e0 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c0ab790 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff06c020050 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff07db10740 10 client.16071 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.548+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565da78eb80 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06c0ab790 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff06c0cd220 con 0x5565da6f44d0 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff07db10740 10 client.16071 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff07db10740 10 client.16071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff07db10740 20 client.16071 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da78ef10 0x5565da791330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.549+0000 7ff07db10740 4 client.16071 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:50:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.550+0000 7ff078cd6640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da78ef10 0x5565da791330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:33.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.550+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565da6688a0 con 0x5565da78ef10 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.550+0000 7ff078cd6640 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da78ef10 0x5565da791330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45e762f61c87cb69 server_cookie=36c106a81a34a006 in_seq=0 out_seq=0 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.550+0000 7ff058ff9640 10 client.16071 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.550+0000 7ff058ff9640 1 -- 172.21.15.96:0/2465290369 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565da6688a0 con 0x5565da78ef10 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff058ff9640 10 client.16071 handle_command_reply: tid=0 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 client.16071 shutdown 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 msgr2=0x5565da719cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0abc10 tx=0x7ff06c009be0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 shutdown_connections 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5565da78ef10 0x5565da791330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565da6f4df0 0x5565da71a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565da700d80 0x5565da714ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 --2- 172.21.15.96:0/2465290369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565da6f44d0 0x5565da719cf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 >> 172.21.15.96:0/2465290369 conn(0x5565da6f2300 msgr2=0x5565da6fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff058ff9640 0 client.16071 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 shutdown_connections 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 1 -- 172.21.15.96:0/2465290369 wait complete. 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.551+0000 7ff07db10740 20 client.16071 trim_cache size 0 max 16384 2024-09-10T22:50:33.558 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 msgr2=0x7ff05c0b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 0x7ff05c0b7930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff068004650 tx=0x7ff06802a440 comp rx=0 tx=0).stop 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 msgr2=0x7ff074112370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 0x7ff074112370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00d6d0 tx=0x7ff06c00db90 comp rx=0 tx=0).stop 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 shutdown_connections 2024-09-10T22:50:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0741076a0 0x7ff0741128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff05c0b5470 0x7ff05c0b7930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff074108020 0x7ff074117c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 --2- 172.21.15.96:0/1613328510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff074101c90 0x7ff074112370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 >> 172.21.15.96:0/1613328510 conn(0x7ff0740fd700 msgr2=0x7ff0740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.552+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 shutdown_connections 2024-09-10T22:50:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:33.553+0000 7ff07b73b640 1 -- 172.21.15.96:0/1613328510 wait complete. 2024-09-10T22:50:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:33 smithi096 ceph-mon[30869]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 771 op/s 2024-09-10T22:50:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:33 smithi121 ceph-mon[35119]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 771 op/s 2024-09-10T22:50:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:33 smithi117 ceph-mon[35081]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 771 op/s 2024-09-10T22:50:34.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:50:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:35 smithi096 ceph-mon[30869]: pgmap v445: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 866 op/s 2024-09-10T22:50:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:35 smithi121 ceph-mon[35119]: pgmap v445: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 866 op/s 2024-09-10T22:50:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:35 smithi117 ceph-mon[35081]: pgmap v445: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 866 op/s 2024-09-10T22:50:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 180.0 s, 3634.2 tps, lat 27.608 ms stddev 32.103 2024-09-10T22:50:36.513 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 -- 172.21.15.96:0/845479438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 msgr2=0x7f68a410ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 --2- 172.21.15.96:0/845479438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a410ee00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6894009860 tx=0x7f689402a340 comp rx=0 tx=0).stop 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 -- 172.21.15.96:0/845479438 shutdown_connections 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 --2- 172.21.15.96:0/845479438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a410ee00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 --2- 172.21.15.96:0/845479438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a410f3d0 0x7f68a41117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 --2- 172.21.15.96:0/845479438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a4075540 0x7f68a4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.978+0000 7f68a906a640 1 -- 172.21.15.96:0/845479438 >> 172.21.15.96:0/845479438 conn(0x7f68a40fe860 msgr2=0x7f68a4100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.979+0000 7f68a906a640 1 -- 172.21.15.96:0/845479438 shutdown_connections 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.979+0000 7f68a906a640 1 -- 172.21.15.96:0/845479438 wait complete. 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.980+0000 7f68a906a640 1 Processor -- start 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.980+0000 7f68a906a640 1 -- start start 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a4075540 0x7f68a419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a410f3d0 0x7f68a41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4114fc0 con 0x7f68a4075540 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4114e40 con 0x7f68a4075f40 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a906a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4115140 con 0x7f68a410f3d0 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a410f3d0 0x7f68a41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a4075540 0x7f68a419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38424/0 (socket says 172.21.15.96:38424) 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.981+0000 7f68a259c640 1 -- 172.21.15.96:0/2869889492 learned_addr learned my addr 172.21.15.96:0/2869889492 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 -- 172.21.15.96:0/2869889492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a410f3d0 msgr2=0x7f68a41a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a410f3d0 0x7f68a41a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 -- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a4075540 msgr2=0x7f68a419b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a4075540 0x7f68a419b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 -- 172.21.15.96:0/2869889492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a41a1580 con 0x7f68a4075f40 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a259c640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f689402a8b0 tx=0x7f6894032d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f688bfff640 1 -- 172.21.15.96:0/2869889492 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894042070 con 0x7f68a4075f40 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a41a1810 con 0x7f68a4075f40 2024-09-10T22:50:36.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f688bfff640 1 -- 172.21.15.96:0/2869889492 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6894030d50 con 0x7f68a4075f40 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f688bfff640 1 -- 172.21.15.96:0/2869889492 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894037040 con 0x7f68a4075f40 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.982+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a4102990 con 0x7f68a4075f40 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 Processor -- start 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 -- start start 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb7f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb8b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb8b9a0 0x556d4bb8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d4baf5ea0 con 0x556d4bb7f0f0 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d4bae52d0 con 0x556d4bb8b9a0 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68ab43f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d4bae90e0 con 0x556d4bb7fa10 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68a359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb7f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68a2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb8b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68a2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb8b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52928/0 (socket says 172.21.15.96:52928) 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.986+0000 7f68a2d9d640 1 -- 172.21.15.96:0/390467992 learned_addr learned my addr 172.21.15.96:0/390467992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 -- 172.21.15.96:0/390467992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 msgr2=0x556d4bb8b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb8b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 -- 172.21.15.96:0/390467992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb8b9a0 msgr2=0x556d4bb8dd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb8b9a0 0x556d4bb8dd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 -- 172.21.15.96:0/390467992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4b843650 con 0x556d4bb7f0f0 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68a359e640 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb7f4d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a756e5b1d6ac3af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68897fa640 1 -- 172.21.15.96:0/390467992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898004030 con 0x556d4bb7f0f0 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68897fa640 1 -- 172.21.15.96:0/390467992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68980035f0 con 0x556d4bb7f0f0 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f68897fa640 1 -- 172.21.15.96:0/390467992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898019620 con 0x556d4bb7f0f0 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.987+0000 7f688bfff640 1 -- 172.21.15.96:0/2869889492 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f68940052d0 con 0x7f68a4075f40 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.988+0000 7f68ab43f740 1 -- 172.21.15.96:0/390467992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 msgr2=0x556d4bb7f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.988+0000 7f68ab43f740 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb7f4d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898011810 comp rx=0 tx=0).stop 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f688bfff640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 0x7f688c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 -- 172.21.15.96:0/390467992 shutdown_connections 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb8b9a0 0x556d4bb8dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb8b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 --2- 172.21.15.96:0/390467992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb7f4d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 -- 172.21.15.96:0/390467992 >> 172.21.15.96:0/390467992 conn(0x556d4bb7cf20 msgr2=0x556d4bb7e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68a359e640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 0x7f688c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f688bfff640 1 -- 172.21.15.96:0/2869889492 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f68941030e0 con 0x7f68a4075f40 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68a359e640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 0x7f688c0b77e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68980099d0 tx=0x7f6898003ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 -- 172.21.15.96:0/390467992 shutdown_connections 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.989+0000 7f68ab43f740 1 -- 172.21.15.96:0/390467992 wait complete. 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 -- start start 2024-09-10T22:50:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 Processor -- start 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb88da0 0x556d4bb89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d4b946700 con 0x556d4bb7f0f0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d4ba80160 con 0x556d4bb7fa10 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.990+0000 7f68ab43f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d4bb8e620 con 0x556d4bb88da0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38438/0 (socket says 172.21.15.96:38438) 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 -- 172.21.15.96:0/3437717293 learned_addr learned my addr 172.21.15.96:0/3437717293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68ab43f740 10 client.0 _ll_register_callbacks cb 0x556d4ba999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a359e640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb88da0 0x556d4bb89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 -- 172.21.15.96:0/3437717293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb88da0 msgr2=0x556d4bb89220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb88da0 0x556d4bb89220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 -- 172.21.15.96:0/3437717293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 msgr2=0x556d4bb8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 -- 172.21.15.96:0/3437717293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4baadc90 con 0x556d4bb7fa10 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f68a259c640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f68940c8f00 tx=0x7f68940c6320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee76c378345eaf6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.991+0000 7f6886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.992+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940c8480 con 0x556d4bb7fa10 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f68940d51c0 con 0x556d4bb7fa10 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68980035f0 con 0x556d4bb7fa10 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68940c8620 con 0x556d4bb7fa10 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 10 client.25616 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:36.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d4bb9a430 con 0x556d4bb7fa10 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68980035f0 con 0x556d4bb7fa10 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f68940052d0 con 0x556d4bb7fa10 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 10 client.25616 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 10 client.25616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 20 client.25616 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bb9b780 0x556d4bb9dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 4 client.25616 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.993+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4baf7df0 con 0x556d4bb9b780 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.994+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bb9b780 0x556d4bb9dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.995+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bb9b780 0x556d4bb9dbb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=403f591c7cea5e97 server_cookie=f30f0dd876739589 in_seq=0 out_seq=0 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.995+0000 7f6886ff5640 10 client.25616 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.995+0000 7f6886ff5640 1 -- 172.21.15.96:0/3437717293 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d4baf7df0 con 0x556d4bb9b780 2024-09-10T22:50:36.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.995+0000 7f6886ff5640 10 client.25616 handle_command_reply: tid=0 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 client.25616 shutdown 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 msgr2=0x556d4bb88770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f68940c8f00 tx=0x7f68940c6320 comp rx=0 tx=0).stop 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 shutdown_connections 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bb9b780 0x556d4bb9dbb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb88770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb88da0 0x556d4bb89220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3437717293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 >> 172.21.15.96:0/3437717293 conn(0x556d4bb7cf20 msgr2=0x556d4bb82a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f6886ff5640 0 client.25616 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 shutdown_connections 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 1 -- 172.21.15.96:0/3437717293 wait complete. 2024-09-10T22:50:36.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:35.996+0000 7f68ab43f740 20 client.25616 trim_cache size 0 max 16384 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 Processor -- start 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 -- start start 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7f0f0 0x556d4bb87c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb83950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b4870 con 0x556d4bb88da0 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b63b0 con 0x556d4bb7fa10 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68ab43f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b4f40 con 0x556d4bb7f0f0 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68a359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68a359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43124/0 (socket says 172.21.15.96:43124) 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68a359e640 1 -- 172.21.15.96:0/3870718698 learned_addr learned my addr 172.21.15.96:0/3870718698 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:36.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7f0f0 0x556d4bb87c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.018+0000 7f68a259c640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb83950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 -- 172.21.15.96:0/3870718698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7f0f0 msgr2=0x556d4bb87c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7f0f0 0x556d4bb87c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 -- 172.21.15.96:0/3870718698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 msgr2=0x556d4bb83950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb83950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 -- 172.21.15.96:0/3870718698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4bb88ba0 con 0x556d4bb88da0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a259c640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb83950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f68a359e640 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6898003f10 tx=0x7f6898023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d57d3552c889970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f6888ff9640 1 -- 172.21.15.96:0/3870718698 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689802c050 con 0x556d4bb88da0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f6888ff9640 1 -- 172.21.15.96:0/3870718698 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6898036070 con 0x556d4bb88da0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.019+0000 7f6888ff9640 1 -- 172.21.15.96:0/3870718698 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689801f040 con 0x556d4bb88da0 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- 172.21.15.96:0/3870718698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 msgr2=0x556d4bb83e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6898003f10 tx=0x7f6898023000 comp rx=0 tx=0).stop 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- 172.21.15.96:0/3870718698 shutdown_connections 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb7fa10 0x556d4bb83950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb88da0 0x556d4bb83e90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- 172.21.15.96:0/3870718698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7f0f0 0x556d4bb87c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- 172.21.15.96:0/3870718698 >> 172.21.15.96:0/3870718698 conn(0x556d4bb7cf20 msgr2=0x556d4bb7e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- 172.21.15.96:0/3870718698 shutdown_connections 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- 172.21.15.96:0/3870718698 wait complete. 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 -- start start 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 Processor -- start 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.020+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb93e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 0x556d4bb8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b5ce0 con 0x556d4bb7f0f0 2024-09-10T22:50:36.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b5610 con 0x556d4bb88da0 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b2870 con 0x556d4bb7fa10 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 10 client.0 _ll_register_callbacks cb 0x7f6894040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43138/0 (socket says 172.21.15.96:43138) 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 -- 172.21.15.96:0/1725971004 learned_addr learned my addr 172.21.15.96:0/1725971004 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a359e640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 0x556d4bb8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 -- 172.21.15.96:0/1725971004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 msgr2=0x556d4bb93e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb93e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 -- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 msgr2=0x556d4bb8e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 0x556d4bb8e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 -- 172.21.15.96:0/1725971004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4bb8ee10 con 0x556d4bb7f0f0 2024-09-10T22:50:36.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a359e640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 0x556d4bb8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68a2d9d640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f689c000f80 tx=0x7f689c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706ffffa505ee97d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f6886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00fa70 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f689c00fc10 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6898036070 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f689c00fe10 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 10 client.16086 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.021+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d4bc08f00 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6898036070 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f689c0233f0 con 0x556d4bb7f0f0 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 10 client.16086 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 10 client.16086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:50:36.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 20 client.16086 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bc096b0 0x556d4bc0bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 4 client.16086 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d4baf38a0 con 0x556d4bc096b0 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68a259c640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bc096b0 0x556d4bc0bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f6886ff5640 10 client.16086 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.022+0000 7f68a259c640 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bc096b0 0x556d4bc0bad0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30e9c9cec5d9f84f server_cookie=466dfb51a366a8e6 in_seq=0 out_seq=0 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f6886ff5640 1 -- 172.21.15.96:0/1725971004 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d4baf38a0 con 0x556d4bc096b0 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f6886ff5640 10 client.16086 handle_command_reply: tid=0 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 client.16086 shutdown 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 msgr2=0x556d4bb93920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f689c000f80 tx=0x7f689c011b50 comp rx=0 tx=0).stop 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 shutdown_connections 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556d4bc096b0 0x556d4bc0bad0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d4bb88da0 0x556d4bb8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f6886ff5640 0 client.16086 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d4bb7fa10 0x556d4bb93e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 --2- 172.21.15.96:0/1725971004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d4bb7f0f0 0x556d4bb93920 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 >> 172.21.15.96:0/1725971004 conn(0x556d4bb7cf20 msgr2=0x556d4bb83950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 shutdown_connections 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 1 -- 172.21.15.96:0/1725971004 wait complete. 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.023+0000 7f68ab43f740 20 client.16086 trim_cache size 0 max 16384 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 msgr2=0x7f688c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 0x7f688c0b77e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68980099d0 tx=0x7f6898003ea0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 msgr2=0x7f68a419ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f689402a8b0 tx=0x7f6894032d20 comp rx=0 tx=0).stop 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 shutdown_connections 2024-09-10T22:50:36.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68a4075f40 0x7f68a419ba10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f688c0b5320 0x7f688c0b77e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68a410f3d0 0x7f68a41a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 --2- 172.21.15.96:0/2869889492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68a4075540 0x7f68a419b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 >> 172.21.15.96:0/2869889492 conn(0x7f68a40fe860 msgr2=0x7f68a41115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 shutdown_connections 2024-09-10T22:50:36.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:36.024+0000 7f68a906a640 1 -- 172.21.15.96:0/2869889492 wait complete. 2024-09-10T22:50:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:37 smithi096 ceph-mon[30869]: pgmap v446: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 960 op/s 2024-09-10T22:50:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:37 smithi121 ceph-mon[35119]: pgmap v446: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 960 op/s 2024-09-10T22:50:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:37 smithi117 ceph-mon[35081]: pgmap v446: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 36 MiB/s wr, 960 op/s 2024-09-10T22:50:37.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:50:38.190 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 -- 172.21.15.96:0/3129299412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2954102dc0 msgr2=0x7f29541031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 --2- 172.21.15.96:0/3129299412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2954102dc0 0x7f29541031a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2944009860 tx=0x7f294402a340 comp rx=0 tx=0).stop 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 -- 172.21.15.96:0/3129299412 shutdown_connections 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 --2- 172.21.15.96:0/3129299412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29541087d0 0x7f2954108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 --2- 172.21.15.96:0/3129299412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954109150 0x7f295410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 --2- 172.21.15.96:0/3129299412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2954102dc0 0x7f29541031a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 -- 172.21.15.96:0/3129299412 >> 172.21.15.96:0/3129299412 conn(0x7f29540fe870 msgr2=0x7f2954100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 -- 172.21.15.96:0/3129299412 shutdown_connections 2024-09-10T22:50:38.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.749+0000 7f295a3f8640 1 -- 172.21.15.96:0/3129299412 wait complete. 2024-09-10T22:50:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.750+0000 7f295a3f8640 1 Processor -- start 2024-09-10T22:50:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.750+0000 7f295a3f8640 1 -- start start 2024-09-10T22:50:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954102dc0 0x7f295419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 0x7f29541a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2954112e10 con 0x7f2954102dc0 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2954112c90 con 0x7f2954109150 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f295a3f8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2954112f90 con 0x7f29541087d0 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f2958995640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f2958995640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45590/0 (socket says 172.21.15.96:45590) 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f2958995640 1 -- 172.21.15.96:0/1917562895 learned_addr learned my addr 172.21.15.96:0/1917562895 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.751+0000 7f2953fff640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954102dc0 0x7f295419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f29537fe640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 0x7f29541a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f2958995640 1 -- 172.21.15.96:0/1917562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 msgr2=0x7f29541a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f2958995640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 0x7f29541a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f2958995640 1 -- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954102dc0 msgr2=0x7f295419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f2958995640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954102dc0 0x7f295419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f2958995640 1 -- 172.21.15.96:0/1917562895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295410f1f0 con 0x7f2954109150 2024-09-10T22:50:38.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.752+0000 7f29537fe640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 0x7f29541a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f2958995640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f294c00eda0 tx=0x7f294c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f29517fa640 1 -- 172.21.15.96:0/1917562895 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c003b70 con 0x7f2954109150 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f29517fa640 1 -- 172.21.15.96:0/1917562895 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c003d10 con 0x7f2954109150 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f29517fa640 1 -- 172.21.15.96:0/1917562895 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c0178e0 con 0x7f2954109150 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295410f4e0 con 0x7f2954109150 2024-09-10T22:50:38.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.753+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295419a9c0 con 0x7f2954109150 2024-09-10T22:50:38.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.763+0000 7f295c7cd740 1 Processor -- start 2024-09-10T22:50:38.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 -- start start 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b6074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6139a0 0x556a5b615d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556a5b57dea0 con 0x556a5b607a10 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a5b56d2d0 con 0x556a5b6139a0 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.765+0000 7f295c7cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a5b5710e0 con 0x556a5b6070f0 2024-09-10T22:50:38.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.766+0000 7f29517fa640 1 -- 172.21.15.96:0/1917562895 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f294c00b030 con 0x7f2954109150 2024-09-10T22:50:38.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.767+0000 7f29517fa640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 0x7f293c0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.767+0000 7f2953fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.767+0000 7f2953fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38466/0 (socket says 172.21.15.96:38466) 2024-09-10T22:50:38.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.767+0000 7f2953fff640 1 -- 172.21.15.96:0/494020558 learned_addr learned my addr 172.21.15.96:0/494020558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:38.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.767+0000 7f29517fa640 1 -- 172.21.15.96:0/1917562895 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f294c0e5950 con 0x7f2954109150 2024-09-10T22:50:38.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.769+0000 7f29537fe640 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b6074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.769+0000 7f29537fe640 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6139a0 0x556a5b615d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.769+0000 7f2958995640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 0x7f293c0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 -- 172.21.15.96:0/494020558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 msgr2=0x556a5b6074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b6074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 -- 172.21.15.96:0/494020558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6139a0 msgr2=0x556a5b615d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6139a0 0x556a5b615d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 -- 172.21.15.96:0/494020558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b2cb650 con 0x556a5b607a10 2024-09-10T22:50:38.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.772+0000 7f2953fff640 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f294402a930 tx=0x7f2944030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a987958d62bb5e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.773+0000 7f293a7fc640 1 -- 172.21.15.96:0/494020558 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944043070 con 0x556a5b607a10 2024-09-10T22:50:38.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.773+0000 7f293a7fc640 1 -- 172.21.15.96:0/494020558 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2944030450 con 0x556a5b607a10 2024-09-10T22:50:38.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.773+0000 7f293a7fc640 1 -- 172.21.15.96:0/494020558 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944037040 con 0x556a5b607a10 2024-09-10T22:50:38.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.774+0000 7f295c7cd740 1 -- 172.21.15.96:0/494020558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 msgr2=0x556a5b613460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.774+0000 7f295c7cd740 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f294402a930 tx=0x7f2944030b50 comp rx=0 tx=0).stop 2024-09-10T22:50:38.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f2958995640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 0x7f293c0b7970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f294c0abe10 tx=0x7f294c011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f295c7cd740 1 -- 172.21.15.96:0/494020558 shutdown_connections 2024-09-10T22:50:38.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f295c7cd740 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6139a0 0x556a5b615d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f295c7cd740 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b613460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f295c7cd740 1 --2- 172.21.15.96:0/494020558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b6074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.776+0000 7f295c7cd740 1 -- 172.21.15.96:0/494020558 >> 172.21.15.96:0/494020558 conn(0x556a5b604f20 msgr2=0x556a5b6060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.779+0000 7f295c7cd740 1 -- 172.21.15.96:0/494020558 shutdown_connections 2024-09-10T22:50:38.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.779+0000 7f295c7cd740 1 -- 172.21.15.96:0/494020558 wait complete. 2024-09-10T22:50:38.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.779+0000 7f295c7cd740 1 -- start start 2024-09-10T22:50:38.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.779+0000 7f295c7cd740 1 Processor -- start 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b613290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b6106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556a5b3ce700 con 0x556a5b607a10 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a5b508160 con 0x556a5b6070f0 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a5b616610 con 0x556a5b6139a0 2024-09-10T22:50:38.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.780+0000 7f295c7cd740 10 client.0 _ll_register_callbacks cb 0x556a5b5219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:38.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.781+0000 7f29537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.781+0000 7f29537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43152/0 (socket says 172.21.15.96:43152) 2024-09-10T22:50:38.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.781+0000 7f29537fe640 1 -- 172.21.15.96:0/2979310271 learned_addr learned my addr 172.21.15.96:0/2979310271 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.781+0000 7f29537fe640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b613290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 -- 172.21.15.96:0/2979310271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 msgr2=0x556a5b613290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b613290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 -- 172.21.15.96:0/2979310271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 msgr2=0x556a5b6106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b6106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 -- 172.21.15.96:0/2979310271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b535c90 con 0x556a5b6139a0 2024-09-10T22:50:38.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29537fe640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2948018610 tx=0x7f29480047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0447d31fff7202 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.784+0000 7f29337fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:38.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948020070 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f294801a030 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2944030450 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f294801a230 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f295c7cd740 10 client.25693 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.785+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a5b621f90 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.786+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2944030450 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2948023020 con 0x556a5b6139a0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 10 client.25693 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 10 client.25693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 20 client.25693 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b6238e0 0x556a5b625d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 4 client.25693 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.787+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a5b57fdf0 con 0x556a5b6238e0 2024-09-10T22:50:38.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.788+0000 7f2953fff640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b6238e0 0x556a5b625d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.793+0000 7f2953fff640 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b6238e0 0x556a5b625d60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=786ff48c1a82dd4 server_cookie=ed7b343631587f57 in_seq=0 out_seq=0 2024-09-10T22:50:38.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.793+0000 7f29337fe640 10 client.25693 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:50:38.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.796+0000 7f29337fe640 1 -- 172.21.15.96:0/2979310271 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a5b57fdf0 con 0x556a5b6238e0 2024-09-10T22:50:38.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.796+0000 7f29337fe640 10 client.25693 handle_command_reply: tid=0 2024-09-10T22:50:38.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.796+0000 7f295c7cd740 1 client.25693 shutdown 2024-09-10T22:50:38.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.798+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 msgr2=0x556a5b610d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.798+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2948018610 tx=0x7f29480047d0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 shutdown_connections 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b610d10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b6106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b6238e0 0x556a5b625d60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2979310271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b613290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 >> 172.21.15.96:0/2979310271 conn(0x556a5b604f20 msgr2=0x556a5b60aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f29337fe640 0 client.25693 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 shutdown_connections 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 1 -- 172.21.15.96:0/2979310271 wait complete. 2024-09-10T22:50:38.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.799+0000 7f295c7cd740 20 client.25693 trim_cache size 0 max 16384 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 Processor -- start 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 -- start start 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b60ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b60f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29541bced0 con 0x556a5b607a10 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29541bea10 con 0x556a5b6070f0 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.821+0000 7f295c7cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29541bd5a0 con 0x556a5b6139a0 2024-09-10T22:50:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45626/0 (socket says 172.21.15.96:45626) 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 -- 172.21.15.96:0/2085444557 learned_addr learned my addr 172.21.15.96:0/2085444557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f2953fff640 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b60ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b60f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 -- 172.21.15.96:0/2085444557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 msgr2=0x556a5b60f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b60f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 -- 172.21.15.96:0/2085444557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 msgr2=0x556a5b60ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b60ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 -- 172.21.15.96:0/2085444557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b605c00 con 0x556a5b6070f0 2024-09-10T22:50:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.822+0000 7f29537fe640 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f294801a770 tx=0x7f294801a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f3989dad8461b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.823+0000 7f2939ffb640 1 -- 172.21.15.96:0/2085444557 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948033070 con 0x556a5b6070f0 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.823+0000 7f2939ffb640 1 -- 172.21.15.96:0/2085444557 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2948010040 con 0x556a5b6070f0 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.823+0000 7f2939ffb640 1 -- 172.21.15.96:0/2085444557 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948020070 con 0x556a5b6070f0 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- 172.21.15.96:0/2085444557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 msgr2=0x556a5b604760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f294801a770 tx=0x7f294801a6c0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- 172.21.15.96:0/2085444557 shutdown_connections 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6139a0 0x556a5b60f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b607a10 0x556a5b60ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2085444557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b6070f0 0x556a5b604760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- 172.21.15.96:0/2085444557 >> 172.21.15.96:0/2085444557 conn(0x556a5b604f20 msgr2=0x556a5b5fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- 172.21.15.96:0/2085444557 shutdown_connections 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- 172.21.15.96:0/2085444557 wait complete. 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.824+0000 7f295c7cd740 1 -- start start 2024-09-10T22:50:38.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.825+0000 7f295c7cd740 1 Processor -- start 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.825+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.825+0000 7f295c7cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 0x556a5b62afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f29537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f29537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43178/0 (socket says 172.21.15.96:43178) 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f29537fe640 1 -- 172.21.15.96:0/2362671813 learned_addr learned my addr 172.21.15.96:0/2362671813 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b6139a0 0x556a5b62b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f2953fff640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 0x556a5b62afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29541be340 con 0x556a5b6139a0 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f29541bdc70 con 0x556a5b607a10 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29541baf60 con 0x556a5b6070f0 2024-09-10T22:50:38.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.826+0000 7f29537fe640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b6139a0 0x556a5b62b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f295c7cd740 10 client.0 _ll_register_callbacks cb 0x7f294800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 -- 172.21.15.96:0/2362671813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 msgr2=0x556a5b62afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 0x556a5b62afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 -- 172.21.15.96:0/2362671813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b6139a0 msgr2=0x556a5b62b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b6139a0 0x556a5b62b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b62bad0 con 0x556a5b6070f0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f2953fff640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 0x556a5b62afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29537fe640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f294801a200 tx=0x7f294801a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1741f783faba71aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29337fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294803d060 con 0x556a5b6070f0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2948014620 con 0x556a5b6070f0 2024-09-10T22:50:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.827+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2948010040 con 0x556a5b6070f0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.828+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2948020070 con 0x556a5b6070f0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.828+0000 7f295c7cd740 10 client.25699 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.828+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a5b619610 con 0x556a5b6070f0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.829+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2948010040 con 0x556a5b6070f0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2948023070 con 0x556a5b6070f0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 10 client.25699 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 10 client.25699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 20 client.25699 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b690be0 0x556a5b693000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 4 client.25699 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a5b57b8a0 con 0x556a5b690be0 2024-09-10T22:50:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.830+0000 7f2953fff640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b690be0 0x556a5b693000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.831+0000 7f2953fff640 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b690be0 0x556a5b693000 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e9b37ab465d484f server_cookie=76d0401568ff48d5 in_seq=0 out_seq=0 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.831+0000 7f29337fe640 10 client.25699 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.831+0000 7f29337fe640 1 -- 172.21.15.96:0/2362671813 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a5b57b8a0 con 0x556a5b690be0 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.831+0000 7f29337fe640 10 client.25699 handle_command_reply: tid=0 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.831+0000 7f295c7cd740 1 client.25699 shutdown 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 msgr2=0x556a5b61bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f294801a200 tx=0x7f294801a060 comp rx=0 tx=0).stop 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 shutdown_connections 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556a5b690be0 0x556a5b693000 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a5b607a10 0x556a5b62afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556a5b6139a0 0x556a5b62b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 --2- 172.21.15.96:0/2362671813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b6070f0 0x556a5b61bde0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 >> 172.21.15.96:0/2362671813 conn(0x556a5b604f20 msgr2=0x556a5b605f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f29337fe640 0 client.25699 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 shutdown_connections 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.832+0000 7f295c7cd740 1 -- 172.21.15.96:0/2362671813 wait complete. 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.833+0000 7f295c7cd740 20 client.25699 trim_cache size 0 max 16384 2024-09-10T22:50:38.835 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 msgr2=0x7f293c0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 0x7f293c0b7970 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f294c0abe10 tx=0x7f294c011040 comp rx=0 tx=0).stop 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 msgr2=0x7f295410e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f294c00eda0 tx=0x7f294c00c6e0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 shutdown_connections 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954109150 0x7f295410e930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f293c0b54b0 0x7f293c0b7970 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29541087d0 0x7f29541a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 --2- 172.21.15.96:0/1917562895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2954102dc0 0x7f295419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 >> 172.21.15.96:0/1917562895 conn(0x7f29540fe870 msgr2=0x7f2954104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 shutdown_connections 2024-09-10T22:50:38.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:38.834+0000 7f295a3f8640 1 -- 172.21.15.96:0/1917562895 wait complete. 2024-09-10T22:50:39.619 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:39 smithi096 ceph-mon[30869]: pgmap v447: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 823 op/s 2024-09-10T22:50:39.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:50:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:39 smithi121 ceph-mon[35119]: pgmap v447: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 823 op/s 2024-09-10T22:50:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:39 smithi117 ceph-mon[35081]: pgmap v447: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 823 op/s 2024-09-10T22:50:40.014 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.454+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/3311263396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b8106460 msgr2=0x7fb4b8106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.454+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/3311263396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b8106460 0x7fb4b8106910 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009ca0 tx=0x7fb4b002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.455+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/3311263396 shutdown_connections 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.455+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/3311263396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b8106460 0x7fb4b8106910 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.455+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/3311263396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b81038b0 0x7fb4b8105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.455+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/3311263396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b806b470 0x7fb4b8103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.455+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/3311263396 >> 172.21.15.96:0/3311263396 conn(0x7fb4b80fd140 msgr2=0x7fb4b80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.457+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/3311263396 shutdown_connections 2024-09-10T22:50:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.457+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/3311263396 wait complete. 2024-09-10T22:50:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.457+0000 7fb4bdd6b640 1 Processor -- start 2024-09-10T22:50:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 -- start start 2024-09-10T22:50:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 0x7fb4b8110a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8106460 0x7fb4b8111480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8107e00 con 0x7fb4b81038b0 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8107c80 con 0x7fb4b8106460 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4bdd6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8107f80 con 0x7fb4b806b470 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 0x7fb4b8110a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 0x7fb4b8110a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43190/0 (socket says 172.21.15.96:43190) 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b77fe640 1 -- 172.21.15.96:0/1923160424 learned_addr learned my addr 172.21.15.96:0/1923160424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38504/0 (socket says 172.21.15.96:38504) 2024-09-10T22:50:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.458+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8106460 0x7fb4b8111480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.459+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1923160424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 msgr2=0x7fb4b8110a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.459+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 0x7fb4b8110a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.459+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1923160424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8106460 msgr2=0x7fb4b8111480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.459+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8106460 0x7fb4b8111480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.459+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1923160424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b810f710 con 0x7fb4b81038b0 2024-09-10T22:50:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b3f0 tx=0x7fb4ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/1923160424 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac015020 con 0x7fb4b81038b0 2024-09-10T22:50:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b810fa00 con 0x7fb4b81038b0 2024-09-10T22:50:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/1923160424 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4ac011ce0 con 0x7fb4b81038b0 2024-09-10T22:50:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/1923160424 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac010d80 con 0x7fb4b81038b0 2024-09-10T22:50:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.460+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b8110020 con 0x7fb4b81038b0 2024-09-10T22:50:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.463+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/1923160424 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb4ac011830 con 0x7fb4b81038b0 2024-09-10T22:50:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.464+0000 7fb4c0c18740 1 Processor -- start 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.464+0000 7fb4c0c18740 1 -- start start 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.464+0000 7fb4b4ff9640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 0x7fb4a00b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560087a9d6f0 con 0x560087b2e190 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560087a8cb20 con 0x560087b30bd0 2024-09-10T22:50:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560087a90930 con 0x560087b2b860 2024-09-10T22:50:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 0x7fb4a00b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/1923160424 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4ac0e5530 con 0x7fb4b81038b0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45664/0 (socket says 172.21.15.96:45664) 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 -- 172.21.15.96:0/1121707909 learned_addr learned my addr 172.21.15.96:0/1121707909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 -- 172.21.15.96:0/1121707909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 msgr2=0x560087b2dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b2dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 -- 172.21.15.96:0/1121707909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 msgr2=0x560087b30580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b30580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b7fff640 1 -- 172.21.15.96:0/1121707909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560087a52530 con 0x560087b30bd0 2024-09-10T22:50:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.465+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b2dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.466+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002aed0 tx=0x7fb4b0031dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6c8ac6f6642a598 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.466+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 0x7fb4a00b77d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009ca0 tx=0x7fb4b0042000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.466+0000 7fb49dffb640 1 -- 172.21.15.96:0/1121707909 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b004d070 con 0x560087b30bd0 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.467+0000 7fb49dffb640 1 -- 172.21.15.96:0/1121707909 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b0036050 con 0x560087b30bd0 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.467+0000 7fb49dffb640 1 -- 172.21.15.96:0/1121707909 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0049680 con 0x560087b30bd0 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.467+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1121707909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 msgr2=0x560087b33090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002aed0 tx=0x7fb4b0031dc0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1121707909 shutdown_connections 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b33090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b30580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1121707909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b2dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1121707909 >> 172.21.15.96:0/1121707909 conn(0x560087b269b0 msgr2=0x560087b27740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1121707909 shutdown_connections 2024-09-10T22:50:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1121707909 wait complete. 2024-09-10T22:50:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 -- start start 2024-09-10T22:50:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.468+0000 7fb4c0c18740 1 Processor -- start 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600879c4ab0 con 0x560087b2e190 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560087a32730 con 0x560087b30bd0 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560087b33b50 con 0x560087b2b860 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.470+0000 7fb4c0c18740 10 client.0 _ll_register_callbacks cb 0x560087a866b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43204/0 (socket says 172.21.15.96:43204) 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 -- 172.21.15.96:0/880870952 learned_addr learned my addr 172.21.15.96:0/880870952 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 -- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 msgr2=0x560087b36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 -- 172.21.15.96:0/880870952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 msgr2=0x560087b355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 -- 172.21.15.96:0/880870952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600877ee790 con 0x560087b2b860 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.471+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e4a0 tx=0x7fb4a800e960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d15089acb951da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8017020 con 0x560087b2b860 2024-09-10T22:50:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4a8013610 con 0x560087b2b860 2024-09-10T22:50:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b0036050 con 0x560087b2b860 2024-09-10T22:50:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4a800f040 con 0x560087b2b860 2024-09-10T22:50:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb4c0c18740 10 client.25705 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.472+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560087867620 con 0x560087b2b860 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b0036050 con 0x560087b2b860 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a8024020 con 0x560087b2b860 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 10 client.25705 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 20 client.25705 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087b40c60 0x560087b43100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 4 client.25705 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560087a9f640 con 0x560087b40c60 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.473+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087b40c60 0x560087b43100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.474+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087b40c60 0x560087b43100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b807d41533e6ff3 server_cookie=51693848e038fa35 in_seq=0 out_seq=0 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.474+0000 7fb496ffd640 10 client.25705 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.474+0000 7fb496ffd640 1 -- 172.21.15.96:0/880870952 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560087a9f640 con 0x560087b40c60 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.474+0000 7fb496ffd640 10 client.25705 handle_command_reply: tid=0 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 client.25705 shutdown 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 msgr2=0x560087b34f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e4a0 tx=0x7fb4a800e960 comp rx=0 tx=0).stop 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 shutdown_connections 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087b40c60 0x560087b43100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b2e190 0x560087b355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/880870952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b34f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb496ffd640 0 client.25705 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 >> 172.21.15.96:0/880870952 conn(0x560087b269b0 msgr2=0x560087b25bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.475+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 shutdown_connections 2024-09-10T22:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.476+0000 7fb4c0c18740 1 -- 172.21.15.96:0/880870952 wait complete. 2024-09-10T22:50:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.476+0000 7fb4c0c18740 20 client.25705 trim_cache size 0 max 16384 2024-09-10T22:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.507+0000 7fb4c0c18740 1 Processor -- start 2024-09-10T22:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.507+0000 7fb4c0c18740 1 -- start start 2024-09-10T22:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b30bd0 0x560087b349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b8bb0 con 0x560087b30bd0 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81ba6f0 con 0x560087b2e190 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b9280 con 0x560087b2b860 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45680/0 (socket says 172.21.15.96:45680) 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/2119555633 learned_addr learned my addr 172.21.15.96:0/2119555633 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b30bd0 0x560087b349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/2119555633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 msgr2=0x560087b33f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b33f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/2119555633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b30bd0 msgr2=0x560087b349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b30bd0 0x560087b349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.508+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/2119555633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560087b2dfc0 con 0x560087b2e190 2024-09-10T22:50:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.509+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0e30d0 tx=0x7fb4ac0e2fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb25de5919c3593 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.509+0000 7fb49d7fa640 1 -- 172.21.15.96:0/2119555633 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0b0870 con 0x560087b2e190 2024-09-10T22:50:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.509+0000 7fb49d7fa640 1 -- 172.21.15.96:0/2119555633 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4ac0b5320 con 0x560087b2e190 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb49d7fa640 1 -- 172.21.15.96:0/2119555633 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac098b90 con 0x560087b2e190 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 -- 172.21.15.96:0/2119555633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 msgr2=0x560087b34460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0e30d0 tx=0x7fb4ac0e2fa0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 -- 172.21.15.96:0/2119555633 shutdown_connections 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b2e190 0x560087b34460 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0e30d0 tx=0x7fb4ac0e2fa0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b30bd0 0x560087b349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/2119555633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b33f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.510+0000 7fb4c0c18740 1 -- 172.21.15.96:0/2119555633 >> 172.21.15.96:0/2119555633 conn(0x560087b269b0 msgr2=0x560087b30fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.511+0000 7fb4c0c18740 1 -- 172.21.15.96:0/2119555633 shutdown_connections 2024-09-10T22:50:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.511+0000 7fb4c0c18740 1 -- 172.21.15.96:0/2119555633 wait complete. 2024-09-10T22:50:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.511+0000 7fb4c0c18740 1 -- start start 2024-09-10T22:50:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.511+0000 7fb4c0c18740 1 Processor -- start 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b3c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b3ba20 0x560087b3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81ba020 con 0x560087b3ba20 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b9950 con 0x560087b30bd0 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b6b90 con 0x560087b2b860 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.512+0000 7fb4c0c18740 10 client.0 _ll_register_callbacks cb 0x7fb4a80091c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45690/0 (socket says 172.21.15.96:45690) 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1013723863 learned_addr learned my addr 172.21.15.96:0/1013723863 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b3ba20 0x560087b3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b3c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1013723863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 msgr2=0x560087b3c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b3c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1013723863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b3ba20 msgr2=0x560087b3bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b3ba20 0x560087b3bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.513+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1013723863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560087b3a000 con 0x560087b30bd0 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.514+0000 7fb4b6ffd640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b3c0 tx=0x7fb4ac000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eff3303cce195ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.514+0000 7fb496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.514+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac093020 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4ac0931c0 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ac0b5320 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4ac091bb0 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb4c0c18740 10 client.25649 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560087bbedf0 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.515+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4ac0b5320 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb4ac0a7770 con 0x560087b30bd0 2024-09-10T22:50:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 10 client.25649 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 20 client.25649 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087bbf6d0 0x560087bc1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 4 client.25649 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560087a9b0f0 con 0x560087bbf6d0 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.516+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087bbf6d0 0x560087bc1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.517+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087bbf6d0 0x560087bc1af0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0939afc93289eb server_cookie=f23d82937f9c808 in_seq=0 out_seq=0 2024-09-10T22:50:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.517+0000 7fb496ffd640 10 client.25649 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb496ffd640 1 -- 172.21.15.96:0/1013723863 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560087a9b0f0 con 0x560087bbf6d0 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb496ffd640 10 client.25649 handle_command_reply: tid=0 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb4c0c18740 1 client.25649 shutdown 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 msgr2=0x560087b3b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b3c0 tx=0x7fb4ac000bc0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 shutdown_connections 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.543+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560087bbf6d0 0x560087bc1af0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560087b30bd0 0x560087b3b4e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560087b3ba20 0x560087b3bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 --2- 172.21.15.96:0/1013723863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560087b2b860 0x560087b3c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb496ffd640 0 client.25649 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 >> 172.21.15.96:0/1013723863 conn(0x560087b269b0 msgr2=0x560087b253d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 shutdown_connections 2024-09-10T22:50:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 1 -- 172.21.15.96:0/1013723863 wait complete. 2024-09-10T22:50:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.544+0000 7fb4c0c18740 20 client.25649 trim_cache size 0 max 16384 2024-09-10T22:50:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:50:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:50:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:50:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 msgr2=0x7fb4a00b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 0x7fb4a00b77d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009ca0 tx=0x7fb4b0042000 comp rx=0 tx=0).stop 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 msgr2=0x7fb4b8110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b3f0 tx=0x7fb4ac00b8b0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 shutdown_connections 2024-09-10T22:50:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8106460 0x7fb4b8111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4a00b5310 0x7fb4a00b77d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81038b0 0x7fb4b8110f40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 --2- 172.21.15.96:0/1923160424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4b806b470 0x7fb4b8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.545+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 >> 172.21.15.96:0/1923160424 conn(0x7fb4b80fd140 msgr2=0x7fb4b80fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.546+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 shutdown_connections 2024-09-10T22:50:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:40.546+0000 7fb4bdd6b640 1 -- 172.21.15.96:0/1923160424 wait complete. 2024-09-10T22:50:40.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 -- 172.21.15.96:0/3042343939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810a150 msgr2=0x7f44f810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 --2- 172.21.15.96:0/3042343939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810a150 0x7f44f810a530 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009900 tx=0x7f44e802a400 comp rx=0 tx=0).stop 2024-09-10T22:50:40.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 -- 172.21.15.96:0/3042343939 shutdown_connections 2024-09-10T22:50:40.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 --2- 172.21.15.96:0/3042343939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f810aa70 0x7f44f81154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 --2- 172.21.15.96:0/3042343939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8115a10 0x7f44f8117e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 --2- 172.21.15.96:0/3042343939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810a150 0x7f44f810a530 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.621+0000 7f44fd686640 1 -- 172.21.15.96:0/3042343939 >> 172.21.15.96:0/3042343939 conn(0x7f44f806e2e0 msgr2=0x7f44f806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.622+0000 7f44fd686640 1 -- 172.21.15.96:0/3042343939 shutdown_connections 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.622+0000 7f44fd686640 1 -- 172.21.15.96:0/3042343939 wait complete. 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 Processor -- start 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 -- start start 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810a150 0x7f44f81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810aa70 0x7f44f81a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44f811b5c0 con 0x7f44f810aa70 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44f811b440 con 0x7f44f8115a10 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.623+0000 7f44fd686640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44f811b740 con 0x7f44f810a150 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45710/0 (socket says 172.21.15.96:45710) 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 -- 172.21.15.96:0/1503557011 learned_addr learned my addr 172.21.15.96:0/1503557011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810a150 0x7f44f81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 -- 172.21.15.96:0/1503557011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810a150 msgr2=0x7f44f81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810a150 0x7f44f81a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 -- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810aa70 msgr2=0x7f44f81a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810aa70 0x7f44f81a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 -- 172.21.15.96:0/1503557011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f81a7b60 con 0x7f44f8115a10 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.624+0000 7f44f77fe640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44f0012de0 tx=0x7f44f000c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.625+0000 7f44effff640 1 -- 172.21.15.96:0/1503557011 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f001f030 con 0x7f44f8115a10 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.625+0000 7f44effff640 1 -- 172.21.15.96:0/1503557011 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44f000ce10 con 0x7f44f8115a10 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.625+0000 7f44effff640 1 -- 172.21.15.96:0/1503557011 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0005020 con 0x7f44f8115a10 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.625+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f81a7e50 con 0x7f44f8115a10 2024-09-10T22:50:40.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.625+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f810b260 con 0x7f44f8115a10 2024-09-10T22:50:40.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.629+0000 7f44ffa5b740 1 Processor -- start 2024-09-10T22:50:40.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.630+0000 7f44effff640 1 -- 172.21.15.96:0/1503557011 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f44f002c060 con 0x7f44f8115a10 2024-09-10T22:50:40.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.630+0000 7f44ffa5b740 1 -- start start 2024-09-10T22:50:40.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a9473c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a946b2ea0 con 0x559a9473c0f0 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a946a22d0 con 0x559a947489a0 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a946a60e0 con 0x559a9473ca10 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43250/0 (socket says 172.21.15.96:43250) 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.631+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1199639520 learned_addr learned my addr 172.21.15.96:0/1199639520 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f67fc640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a9473c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f67fc640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1199639520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 msgr2=0x559a9474ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44effff640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 0x7f44dc0b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1199639520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 msgr2=0x559a9473c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a9473c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1199639520 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a94400650 con 0x559a9473ca10 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44f67fc640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.632+0000 7f44effff640 1 -- 172.21.15.96:0/1503557011 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44f00edea0 con 0x7f44f8115a10 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.633+0000 7f44f77fe640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 0x7f44dc0b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.633+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44e802a970 tx=0x7f44e80040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cf21d9ea341b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.633+0000 7f44ed7fa640 1 -- 172.21.15.96:0/1199639520 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e803f070 con 0x559a9473ca10 2024-09-10T22:50:40.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.633+0000 7f44ed7fa640 1 -- 172.21.15.96:0/1199639520 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e80044c0 con 0x559a9473ca10 2024-09-10T22:50:40.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.633+0000 7f44ed7fa640 1 -- 172.21.15.96:0/1199639520 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8035030 con 0x559a9473ca10 2024-09-10T22:50:40.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.634+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1199639520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 msgr2=0x559a94748460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.634+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44e802a970 tx=0x7f44e80040d0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.634+0000 7f44f77fe640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 0x7f44dc0b7930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f44f00b45f0 tx=0x7f44f002bf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1199639520 shutdown_connections 2024-09-10T22:50:40.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94748460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1199639520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a9473c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1199639520 >> 172.21.15.96:0/1199639520 conn(0x559a94739f20 msgr2=0x559a9473b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1199639520 shutdown_connections 2024-09-10T22:50:40.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.635+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1199639520 wait complete. 2024-09-10T22:50:40.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 -- start start 2024-09-10T22:50:40.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 Processor -- start 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94748300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94745750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94745dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a94503700 con 0x559a9473ca10 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a9463d160 con 0x559a947489a0 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a9474b5c0 con 0x559a9473c0f0 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.636+0000 7f44ffa5b740 10 client.0 _ll_register_callbacks cb 0x559a946569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94745750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94748300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94745750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38582/0 (socket says 172.21.15.96:38582) 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f6ffd640 1 -- 172.21.15.96:0/3636581402 learned_addr learned my addr 172.21.15.96:0/3636581402 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94745dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 -- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 msgr2=0x559a94745dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94745dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 -- 172.21.15.96:0/3636581402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 msgr2=0x559a94745750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94745750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 -- 172.21.15.96:0/3636581402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9466ac90 con 0x559a9473c0f0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44f67fc640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94748300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44f8074550 tx=0x7f44e40095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57f5f463c92931bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.637+0000 7f44da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4010b60 con 0x559a9473c0f0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a946b4df0 con 0x559a9473c0f0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44e4010d00 con 0x559a9473c0f0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44e4004030 con 0x559a9473c0f0 2024-09-10T22:50:40.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44ffa5b740 10 client.25720 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.638+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a94757050 con 0x559a9473c0f0 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559a946b4df0 con 0x559a9473c0f0 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f44e401f020 con 0x559a9473c0f0 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 10 client.25720 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 20 client.25720 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a94758460 0x559a9475a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 4 client.25720 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a946b08a0 con 0x559a94758460 2024-09-10T22:50:40.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.639+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a94758460 0x559a9475a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.640+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a94758460 0x559a9475a850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f6300209be5183 server_cookie=cbb938f72bb43ddb in_seq=0 out_seq=0 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.640+0000 7f44da7fc640 10 client.25720 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44da7fc640 1 -- 172.21.15.96:0/3636581402 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a946b08a0 con 0x559a94758460 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44da7fc640 10 client.25720 handle_command_reply: tid=0 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 client.25720 shutdown 2024-09-10T22:50:40.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 msgr2=0x559a94748300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94748300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44f8074550 tx=0x7f44e40095d0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 shutdown_connections 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a94758460 0x559a9475a850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94745dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94745750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/3636581402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94748300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 >> 172.21.15.96:0/3636581402 conn(0x559a94739f20 msgr2=0x559a9473fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44da7fc640 0 client.25720 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 shutdown_connections 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 1 -- 172.21.15.96:0/3636581402 wait complete. 2024-09-10T22:50:40.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.641+0000 7f44ffa5b740 20 client.25720 trim_cache size 0 max 16384 2024-09-10T22:50:40.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 Processor -- start 2024-09-10T22:50:40.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 -- start start 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a94754820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94749d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b48f0 con 0x559a9473c0f0 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b6430 con 0x559a947489a0 2024-09-10T22:50:40.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.666+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b4fc0 con 0x559a9473ca10 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.667+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.667+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43266/0 (socket says 172.21.15.96:43266) 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.667+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1936671791 learned_addr learned my addr 172.21.15.96:0/1936671791 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.667+0000 7f44f67fc640 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94749d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.668+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1936671791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 msgr2=0x559a94749d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.668+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94749d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.668+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1936671791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 msgr2=0x559a94754820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.668+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a94754820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.668+0000 7f44f6ffd640 1 -- 172.21.15.96:0/1936671791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9474b500 con 0x559a9473ca10 2024-09-10T22:50:40.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.670+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009a20 tx=0x7f44e80043e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c95f450df83d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.671+0000 7f44ecff9640 1 -- 172.21.15.96:0/1936671791 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e803f040 con 0x559a9473ca10 2024-09-10T22:50:40.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.671+0000 7f44ecff9640 1 -- 172.21.15.96:0/1936671791 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e80035f0 con 0x559a9473ca10 2024-09-10T22:50:40.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1936671791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 msgr2=0x559a94739760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009a20 tx=0x7f44e80043e0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1936671791 shutdown_connections 2024-09-10T22:50:40.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a94749d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473ca10 0x559a94739760 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/1936671791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473c0f0 0x559a94754820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.672+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1936671791 >> 172.21.15.96:0/1936671791 conn(0x559a94739f20 msgr2=0x559a9473b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.676+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1936671791 shutdown_connections 2024-09-10T22:50:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.677+0000 7f44ffa5b740 1 -- 172.21.15.96:0/1936671791 wait complete. 2024-09-10T22:50:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.678+0000 7f44ffa5b740 1 -- start start 2024-09-10T22:50:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.678+0000 7f44ffa5b740 1 Processor -- start 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.678+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94750dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b5d60 con 0x559a9473ca10 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b5690 con 0x559a947489a0 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81b28f0 con 0x559a9473c0f0 2024-09-10T22:50:40.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.679+0000 7f44ffa5b740 10 client.0 _ll_register_callbacks cb 0x7f44e4009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43278/0 (socket says 172.21.15.96:43278) 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 -- 172.21.15.96:0/99025938 learned_addr learned my addr 172.21.15.96:0/99025938 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 -- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 msgr2=0x559a9474b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 -- 172.21.15.96:0/99025938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 msgr2=0x559a94750dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94750dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.683+0000 7f44f67fc640 1 -- 172.21.15.96:0/99025938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9474bd80 con 0x559a9473c0f0 2024-09-10T22:50:40.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.684+0000 7f44f67fc640 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009610 tx=0x7f44e4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d00e4e0d76babc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:40.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.684+0000 7f44da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:40.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.684+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40043e0 con 0x559a9473c0f0 2024-09-10T22:50:40.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.684+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44e40035f0 con 0x559a9473c0f0 2024-09-10T22:50:40.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.684+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e80035f0 con 0x559a9473c0f0 2024-09-10T22:50:40.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.688+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44e4004580 con 0x559a9473c0f0 2024-09-10T22:50:40.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.689+0000 7f44ffa5b740 10 client.25726 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:40.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.689+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a947c5b20 con 0x559a9473c0f0 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.690+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e80035f0 con 0x559a9473c0f0 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.690+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f44e401f070 con 0x559a9473c0f0 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 10 client.25726 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 20 client.25726 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a947c6130 0x559a947c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 4 client.25726 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:50:40.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.691+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a946adcd0 con 0x559a947c6130 2024-09-10T22:50:40.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.697+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a947c6130 0x559a947c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:40.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.700+0000 7f44f6ffd640 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a947c6130 0x559a947c8520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf847b4ec7687730 server_cookie=a0909b33693aa7a1 in_seq=0 out_seq=0 2024-09-10T22:50:40.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44da7fc640 10 client.25726 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44da7fc640 1 -- 172.21.15.96:0/99025938 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a946adcd0 con 0x559a947c6130 2024-09-10T22:50:40.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44da7fc640 10 client.25726 handle_command_reply: tid=0 2024-09-10T22:50:40.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44ffa5b740 1 client.25726 shutdown 2024-09-10T22:50:40.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 msgr2=0x559a94750890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.701+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009610 tx=0x7f44e4005f90 comp rx=0 tx=0).stop 2024-09-10T22:50:40.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 shutdown_connections 2024-09-10T22:50:40.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a947c6130 0x559a947c8520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a947489a0 0x559a9474b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a9473ca10 0x559a94750dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 --2- 172.21.15.96:0/99025938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a9473c0f0 0x559a94750890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 >> 172.21.15.96:0/99025938 conn(0x559a94739f20 msgr2=0x559a94746d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44da7fc640 0 client.25726 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:40.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.705+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 shutdown_connections 2024-09-10T22:50:40.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.707+0000 7f44ffa5b740 1 -- 172.21.15.96:0/99025938 wait complete. 2024-09-10T22:50:40.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.708+0000 7f44ffa5b740 20 client.25726 trim_cache size 0 max 16384 2024-09-10T22:50:40.710 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:50:40.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.710+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 msgr2=0x7f44dc0b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.710+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 0x7f44dc0b7930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f44f00b45f0 tx=0x7f44f002bf20 comp rx=0 tx=0).stop 2024-09-10T22:50:40.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.710+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 msgr2=0x7f44f81a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.710+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44f0012de0 tx=0x7f44f000c640 comp rx=0 tx=0).stop 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 shutdown_connections 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f8115a10 0x7f44f81a7390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44dc0b5470 0x7f44dc0b7930 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44f810aa70 0x7f44f81a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 --2- 172.21.15.96:0/1503557011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810a150 0x7f44f81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.711+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 >> 172.21.15.96:0/1503557011 conn(0x7f44f806e2e0 msgr2=0x7f44f8071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.712+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 shutdown_connections 2024-09-10T22:50:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:40.712+0000 7f44fd686640 1 -- 172.21.15.96:0/1503557011 wait complete. 2024-09-10T22:50:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 185.0 s, 4210.8 tps, lat 23.726 ms stddev 27.663 2024-09-10T22:50:41.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:41 smithi096 ceph-mon[30869]: pgmap v448: 97 pgs: 97 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:41 smithi121 ceph-mon[35119]: pgmap v448: 97 pgs: 97 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:41 smithi117 ceph-mon[35081]: pgmap v448: 97 pgs: 97 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:42.669 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:50:42.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:50:43.097 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:43.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:43 smithi096 ceph-mon[30869]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:43.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.666+0000 7fb4d21ff640 1 -- 172.21.15.96:0/803897995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0776d0 msgr2=0x7fb4cc077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.666+0000 7fb4d21ff640 1 --2- 172.21.15.96:0/803897995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc077ab0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009860 tx=0x7fb4bc02a380 comp rx=0 tx=0).stop 2024-09-10T22:50:43.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.669+0000 7fb4d21ff640 1 -- 172.21.15.96:0/803897995 shutdown_connections 2024-09-10T22:50:43.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.669+0000 7fb4d21ff640 1 --2- 172.21.15.96:0/803897995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc077ff0 0x7fb4cc1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.669+0000 7fb4d21ff640 1 --2- 172.21.15.96:0/803897995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4cc113800 0x7fb4cc115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.669+0000 7fb4d21ff640 1 --2- 172.21.15.96:0/803897995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc077ab0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.669+0000 7fb4d21ff640 1 -- 172.21.15.96:0/803897995 >> 172.21.15.96:0/803897995 conn(0x7fb4cc100840 msgr2=0x7fb4cc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.674+0000 7fb4d21ff640 1 -- 172.21.15.96:0/803897995 shutdown_connections 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.674+0000 7fb4d21ff640 1 -- 172.21.15.96:0/803897995 wait complete. 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 Processor -- start 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 -- start start 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 0x7fb4cc112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4cc10bcd0 0x7fb4cc10c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.675+0000 7fb4d21ff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc119340 con 0x7fb4cc10bcd0 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.676+0000 7fb4d21ff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1191c0 con 0x7fb4cc0776d0 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.676+0000 7fb4d21ff640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1194c0 con 0x7fb4cc113800 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.676+0000 7fb4d09fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 0x7fb4cc112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.676+0000 7fb4d09fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 0x7fb4cc112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43288/0 (socket says 172.21.15.96:43288) 2024-09-10T22:50:43.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.676+0000 7fb4d09fc640 1 -- 172.21.15.96:0/2651452872 learned_addr learned my addr 172.21.15.96:0/2651452872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 -- 172.21.15.96:0/2651452872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 msgr2=0x7fb4cc112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 0x7fb4cc112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 -- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4cc10bcd0 msgr2=0x7fb4cc10c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4cc10bcd0 0x7fb4cc10c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.677+0000 7fb4d11fd640 1 -- 172.21.15.96:0/2651452872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc10c950 con 0x7fb4cc0776d0 2024-09-10T22:50:43.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.678+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc112aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc02a970 tx=0x7fb4bc0306f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.681+0000 7fb4c27fc640 1 -- 172.21.15.96:0/2651452872 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc037050 con 0x7fb4cc0776d0 2024-09-10T22:50:43.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.681+0000 7fb4c27fc640 1 -- 172.21.15.96:0/2651452872 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4bc038040 con 0x7fb4cc0776d0 2024-09-10T22:50:43.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.681+0000 7fb4c27fc640 1 -- 172.21.15.96:0/2651452872 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc003d00 con 0x7fb4cc0776d0 2024-09-10T22:50:43.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.681+0000 7fb4d21ff640 1 -- 172.21.15.96:0/2651452872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc111710 con 0x7fb4cc0776d0 2024-09-10T22:50:43.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.681+0000 7fb4d21ff640 1 -- 172.21.15.96:0/2651452872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc111ae0 con 0x7fb4cc0776d0 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.685+0000 7fb4d5836740 1 Processor -- start 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 -- start start 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283c9b50 0x55d7283cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d728333e60 con 0x55d7283c9b50 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d728323290 con 0x55d7283bdbc0 2024-09-10T22:50:43.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.686+0000 7fb4d5836740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d7283270a0 con 0x55d7283bd2a0 2024-09-10T22:50:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43298/0 (socket says 172.21.15.96:43298) 2024-09-10T22:50:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 -- 172.21.15.96:0/263437541 learned_addr learned my addr 172.21.15.96:0/263437541 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4c27fc640 1 -- 172.21.15.96:0/2651452872 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb4bc0035f0 con 0x7fb4cc0776d0 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 -- 172.21.15.96:0/263437541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 msgr2=0x55d7283c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 -- 172.21.15.96:0/263437541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283c9b50 msgr2=0x55d7283cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283c9b50 0x55d7283cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.688+0000 7fb4d19fe640 1 -- 172.21.15.96:0/263437541 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d728081930 con 0x55d7283bd2a0 2024-09-10T22:50:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.689+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800ba30 tx=0x7fb4c800bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d924d27a9e0b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.689+0000 7fb4d21ff640 1 -- 172.21.15.96:0/263437541 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8016070 con 0x55d7283bd2a0 2024-09-10T22:50:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.689+0000 7fb4c27fc640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 0x7fb4b80b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.689+0000 7fb4d21ff640 1 -- 172.21.15.96:0/263437541 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c80054d0 con 0x55d7283bd2a0 2024-09-10T22:50:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.689+0000 7fb4d21ff640 1 -- 172.21.15.96:0/263437541 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8004d70 con 0x55d7283bd2a0 2024-09-10T22:50:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.690+0000 7fb4c27fc640 1 -- 172.21.15.96:0/2651452872 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4bc105f60 con 0x7fb4cc0776d0 2024-09-10T22:50:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.691+0000 7fb4d5836740 1 -- 172.21.15.96:0/263437541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 msgr2=0x55d7283bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.691+0000 7fb4d5836740 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800ba30 tx=0x7fb4c800bef0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.692+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 0x7fb4b80b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.695+0000 7fb4d5836740 1 -- 172.21.15.96:0/263437541 shutdown_connections 2024-09-10T22:50:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.695+0000 7fb4d5836740 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.695+0000 7fb4d5836740 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283c9b50 0x55d7283cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.695+0000 7fb4d5836740 1 --2- 172.21.15.96:0/263437541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283bd680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.695+0000 7fb4d5836740 1 -- 172.21.15.96:0/263437541 >> 172.21.15.96:0/263437541 conn(0x55d7283bb0d0 msgr2=0x55d7283bc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.696+0000 7fb4d11fd640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 0x7fb4b80b7a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0c8cf0 tx=0x7fb4bc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.698+0000 7fb4d5836740 1 -- 172.21.15.96:0/263437541 shutdown_connections 2024-09-10T22:50:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.698+0000 7fb4d5836740 1 -- 172.21.15.96:0/263437541 wait complete. 2024-09-10T22:50:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.699+0000 7fb4d5836740 1 -- start start 2024-09-10T22:50:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.699+0000 7fb4d5836740 1 Processor -- start 2024-09-10T22:50:43.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283c9b50 0x55d7283c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d728184890 con 0x55d7283bdbc0 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7282bdd40 con 0x55d7283c9b50 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d7283cc6c0 con 0x55d7283bd2a0 2024-09-10T22:50:43.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d5836740 10 client.0 _ll_register_callbacks cb 0x55d728321c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43302/0 (socket says 172.21.15.96:43302) 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d19fe640 1 -- 172.21.15.96:0/3150568466 learned_addr learned my addr 172.21.15.96:0/3150568466 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.700+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283c9b50 0x55d7283c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.701+0000 7fb4d19fe640 1 -- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283c9b50 msgr2=0x55d7283c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.701+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283c9b50 0x55d7283c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.701+0000 7fb4d19fe640 1 -- 172.21.15.96:0/3150568466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 msgr2=0x55d7283c6fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:43.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.701+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.701+0000 7fb4d19fe640 1 -- 172.21.15.96:0/3150568466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7283d5a10 con 0x55d7283bd2a0 2024-09-10T22:50:43.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.702+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800b7c0 tx=0x7fb4c8004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c6516e7cff3b29e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.702+0000 7fb4b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:43.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.703+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8016040 con 0x55d7283bd2a0 2024-09-10T22:50:43.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.703+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c80054d0 con 0x55d7283bd2a0 2024-09-10T22:50:43.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.703+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4c8005210 con 0x55d7283bd2a0 2024-09-10T22:50:43.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.705+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4c8004df0 con 0x55d7283bd2a0 2024-09-10T22:50:43.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.705+0000 7fb4d5836740 10 client.25738 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:43.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.705+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d7283d83a0 con 0x55d7283bd2a0 2024-09-10T22:50:43.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c80054d0 con 0x55d7283bd2a0 2024-09-10T22:50:43.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb4c80035f0 con 0x55d7283bd2a0 2024-09-10T22:50:43.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 10 client.25738 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:43.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 10 client.25738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 20 client.25738 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7283d9b80 0x55d7283dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 4 client.25738 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.707+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d728335db0 con 0x55d7283d9b80 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.708+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7283d9b80 0x55d7283dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.708+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7283d9b80 0x55d7283dc000 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b30a0d5be5e6a97 server_cookie=d0806192f70dd3da in_seq=0 out_seq=0 2024-09-10T22:50:43.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.708+0000 7fb4b4ff9640 10 client.25738 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:43.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.709+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/3150568466 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d728335db0 con 0x55d7283d9b80 2024-09-10T22:50:43.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.709+0000 7fb4b4ff9640 10 client.25738 handle_command_reply: tid=0 2024-09-10T22:50:43.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.709+0000 7fb4d5836740 1 client.25738 shutdown 2024-09-10T22:50:43.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 msgr2=0x55d7283c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800b7c0 tx=0x7fb4c8004950 comp rx=0 tx=0).stop 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 shutdown_connections 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7283d9b80 0x55d7283dc000 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283c9b50 0x55d7283c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 --2- 172.21.15.96:0/3150568466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283bd2a0 0x55d7283c6a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 >> 172.21.15.96:0/3150568466 conn(0x55d7283bb0d0 msgr2=0x55d7283c0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4b4ff9640 0 client.25738 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 shutdown_connections 2024-09-10T22:50:43.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 1 -- 172.21.15.96:0/3150568466 wait complete. 2024-09-10T22:50:43.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.710+0000 7fb4d5836740 20 client.25738 trim_cache size 0 max 16384 2024-09-10T22:50:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.748+0000 7fb4d5836740 1 Processor -- start 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.748+0000 7fb4d5836740 1 -- start start 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.748+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bd2a0 0x55d7283c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.748+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d5836740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1bd3e0 con 0x55d7283bd2a0 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d5836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1bef20 con 0x55d7283bdbc0 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d5836740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1bdab0 con 0x55d7283c9b50 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43318/0 (socket says 172.21.15.96:43318) 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 -- 172.21.15.96:0/1203029652 learned_addr learned my addr 172.21.15.96:0/1203029652 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d09fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bd2a0 0x55d7283c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 -- 172.21.15.96:0/1203029652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 msgr2=0x55d7283bc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283bc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 -- 172.21.15.96:0/1203029652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bd2a0 msgr2=0x55d7283c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bd2a0 0x55d7283c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.749+0000 7fb4d19fe640 1 -- 172.21.15.96:0/1203029652 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7283dc680 con 0x55d7283c9b50 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.750+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8003e70 tx=0x7fb4c8005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec114b2d1684800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.750+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1203029652 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8014070 con 0x55d7283c9b50 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.750+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1203029652 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c8003b40 con 0x55d7283c9b50 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.750+0000 7fb4b6ffd640 1 -- 172.21.15.96:0/1203029652 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8003ef0 con 0x55d7283c9b50 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.751+0000 7fb4d5836740 1 -- 172.21.15.96:0/1203029652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 msgr2=0x55d7283e1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.751+0000 7fb4d5836740 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8003e70 tx=0x7fb4c8005f10 comp rx=0 tx=0).stop 2024-09-10T22:50:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.752+0000 7fb4d5836740 1 -- 172.21.15.96:0/1203029652 shutdown_connections 2024-09-10T22:50:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.752+0000 7fb4d5836740 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bdbc0 0x55d7283bc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.753+0000 7fb4d5836740 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283e1910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.753+0000 7fb4d5836740 1 --2- 172.21.15.96:0/1203029652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bd2a0 0x55d7283c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.753+0000 7fb4d5836740 1 -- 172.21.15.96:0/1203029652 >> 172.21.15.96:0/1203029652 conn(0x55d7283bb0d0 msgr2=0x55d7283bbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.753+0000 7fb4d5836740 1 -- 172.21.15.96:0/1203029652 shutdown_connections 2024-09-10T22:50:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.753+0000 7fb4d5836740 1 -- 172.21.15.96:0/1203029652 wait complete. 2024-09-10T22:50:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.754+0000 7fb4d5836740 1 -- start start 2024-09-10T22:50:43.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.754+0000 7fb4d5836740 1 Processor -- start 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d5836740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d19fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45768/0 (socket says 172.21.15.96:45768) 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d19fe640 1 -- 172.21.15.96:0/544570668 learned_addr learned my addr 172.21.15.96:0/544570668 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1be850 con 0x55d7283bdbc0 2024-09-10T22:50:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.755+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1be180 con 0x55d7283bd2a0 2024-09-10T22:50:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.756+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc1bb3c0 con 0x55d7283c9b50 2024-09-10T22:50:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.756+0000 7fb4d5836740 10 client.0 _ll_register_callbacks cb 0x7fb4bc1098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.757+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d09fc640 1 -- 172.21.15.96:0/544570668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 msgr2=0x55d7283cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d09fc640 1 -- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 msgr2=0x55d7283d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d09fc640 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7283d2580 con 0x55d7283bdbc0 2024-09-10T22:50:43.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.758+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.759+0000 7fb4d09fc640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283d1e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55d7283d9f60 tx=0x7fb4c4017930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd11a76b4ffac55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.759+0000 7fb4b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.761+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4023020 con 0x55d7283bdbc0 2024-09-10T22:50:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.761+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c8003b40 con 0x55d7283bdbc0 2024-09-10T22:50:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.761+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb4c40187a0 con 0x55d7283bdbc0 2024-09-10T22:50:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:43 smithi121 ceph-mon[35119]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.762+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4c4023020 con 0x55d7283bdbc0 2024-09-10T22:50:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.762+0000 7fb4d5836740 10 client.16137 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.762+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d728447080 con 0x55d7283bdbc0 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4c8003b40 con 0x55d7283bdbc0 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb4c401ea60 con 0x55d7283bdbc0 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 10 client.16137 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 20 client.16137 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7284477e0 0x55d728449c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 4 client.16137 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:50:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.763+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d728331860 con 0x55d7284477e0 2024-09-10T22:50:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.764+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7284477e0 0x55d728449c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.764+0000 7fb4d19fe640 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7284477e0 0x55d728449c00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ee3219388000b2 server_cookie=5f9854219517207c in_seq=0 out_seq=0 2024-09-10T22:50:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.764+0000 7fb4b4ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:43.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.766+0000 7fb4b4ff9640 1 -- 172.21.15.96:0/544570668 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d728331860 con 0x55d7284477e0 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.766+0000 7fb4b4ff9640 10 client.16137 handle_command_reply: tid=0 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.766+0000 7fb4d5836740 1 client.16137 shutdown 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.766+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 msgr2=0x55d7283d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.766+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283d1e90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55d7283d9f60 tx=0x7fb4c4017930 comp rx=0 tx=0).stop 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 shutdown_connections 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d7284477e0 0x55d728449c00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d7283c9b50 0x55d7283cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7283bdbc0 0x55d7283d1e90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 --2- 172.21.15.96:0/544570668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7283bd2a0 0x55d7283d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4b4ff9640 0 client.16137 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 >> 172.21.15.96:0/544570668 conn(0x55d7283bb0d0 msgr2=0x55d7283c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 shutdown_connections 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 1 -- 172.21.15.96:0/544570668 wait complete. 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.767+0000 7fb4d5836740 20 client.16137 trim_cache size 0 max 16384 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "203971b1-c5e2-4181-ab92-43d1598bdd3f", 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:50:43.770 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:50:43.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 msgr2=0x7fb4b80b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 0x7fb4b80b7a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0c8cf0 tx=0x7fb4bc00b040 comp rx=0 tx=0).stop 2024-09-10T22:50:43.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 msgr2=0x7fb4cc112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:43.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc112aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc02a970 tx=0x7fb4bc0306f0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 shutdown_connections 2024-09-10T22:50:43.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.768+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb4b80b55b0 0x7fb4b80b7a70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4cc10bcd0 0x7fb4cc10c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4cc113800 0x7fb4cc112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 --2- 172.21.15.96:0/2651452872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4cc0776d0 0x7fb4cc112aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:43.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 >> 172.21.15.96:0/2651452872 conn(0x7fb4cc100840 msgr2=0x7fb4cc102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 shutdown_connections 2024-09-10T22:50:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:43.769+0000 7fb4b7fff640 1 -- 172.21.15.96:0/2651452872 wait complete. 2024-09-10T22:50:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:43 smithi117 ceph-mon[35081]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:44.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:44 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:50:44.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:44 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:44.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:44 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:50:44.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:44 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:50:44.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:44 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:44.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:50:44.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:44 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:50:44.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:44 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:44.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:44 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:50:44.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:44 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:50:44.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:44 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:44.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:44 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:50:44.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:44 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:44.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:44 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:50:44.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:44 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:50:44.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:44 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:45.140 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:50:45.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:45 smithi096 ceph-mon[30869]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:45.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:45 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:45.664 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:45 smithi121 ceph-mon[35119]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:45.664 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:45 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:45 smithi117 ceph-mon[35081]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 MiB/s wr, 861 op/s 2024-09-10T22:50:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:45 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:50:45.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 -- 172.21.15.96:0/3250418806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4108020 msgr2=0x7f9ce410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/3250418806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4108020 0x7f9ce410e820 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009ca0 tx=0x7f9cdc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 -- 172.21.15.96:0/3250418806 shutdown_connections 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/3250418806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce41076a0 0x7f9ce4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/3250418806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4108020 0x7f9ce410e820 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/3250418806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce4101c90 0x7f9ce4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 -- 172.21.15.96:0/3250418806 >> 172.21.15.96:0/3250418806 conn(0x7f9ce40fd700 msgr2=0x7f9ce40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 -- 172.21.15.96:0/3250418806 shutdown_connections 2024-09-10T22:50:45.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.797+0000 7f9ce9e52640 1 -- 172.21.15.96:0/3250418806 wait complete. 2024-09-10T22:50:45.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.798+0000 7f9ce9e52640 1 Processor -- start 2024-09-10T22:50:45.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.798+0000 7f9ce9e52640 1 -- start start 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 0x7f9ce419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 0x7f9ce41a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111cf0 con 0x7f9ce4101c90 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111b70 con 0x7f9ce4108020 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce9e52640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111e70 con 0x7f9ce41076a0 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 0x7f9ce41a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 0x7f9ce41a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45788/0 (socket says 172.21.15.96:45788) 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43350/0 (socket says 172.21.15.96:43350) 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 0x7f9ce419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce3fff640 1 -- 172.21.15.96:0/4052705257 learned_addr learned my addr 172.21.15.96:0/4052705257 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 -- 172.21.15.96:0/4052705257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 msgr2=0x7f9ce41a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 0x7f9ce41a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 -- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 msgr2=0x7f9ce419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 0x7f9ce419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce2ffd640 1 -- 172.21.15.96:0/4052705257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce41a1540 con 0x7f9ce41076a0 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.799+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 0x7f9ce419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:50:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.800+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e9f0 tx=0x7f9cd800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.800+0000 7f9ce0ff9640 1 -- 172.21.15.96:0/4052705257 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800ccc0 con 0x7f9ce41076a0 2024-09-10T22:50:45.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.800+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41a1830 con 0x7f9ce41076a0 2024-09-10T22:50:45.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.800+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a7ef0 con 0x7f9ce41076a0 2024-09-10T22:50:45.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.800+0000 7f9ce0ff9640 1 -- 172.21.15.96:0/4052705257 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd80108e0 con 0x7f9ce41076a0 2024-09-10T22:50:45.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.801+0000 7f9ce0ff9640 1 -- 172.21.15.96:0/4052705257 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8018880 con 0x7f9ce41076a0 2024-09-10T22:50:45.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 Processor -- start 2024-09-10T22:50:45.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 -- start start 2024-09-10T22:50:45.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d933860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d93f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d942120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e0d8a9e50 con 0x559e0d93fd30 2024-09-10T22:50:45.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e0d899280 con 0x559e0d933da0 2024-09-10T22:50:45.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9cec227740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559e0d89d090 con 0x559e0d933480 2024-09-10T22:50:45.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d93f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d93f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45802/0 (socket says 172.21.15.96:45802) 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9ce37fe640 1 -- 172.21.15.96:0/1924277808 learned_addr learned my addr 172.21.15.96:0/1924277808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.803+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d933860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce0ff9640 1 -- 172.21.15.96:0/4052705257 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9cd8018a20 con 0x7f9ce41076a0 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 -- 172.21.15.96:0/1924277808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 msgr2=0x559e0d93f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d93f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 -- 172.21.15.96:0/1924277808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 msgr2=0x559e0d942120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d942120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 -- 172.21.15.96:0/1924277808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0d5f7650 con 0x559e0d933480 2024-09-10T22:50:45.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d933860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006010 tx=0x7f9cdc009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92e9266fc6151d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.804+0000 7f9cc9ffb640 1 -- 172.21.15.96:0/1924277808 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc031cf0 con 0x559e0d933480 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cc9ffb640 1 -- 172.21.15.96:0/1924277808 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc036030 con 0x559e0d933480 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cc9ffb640 1 -- 172.21.15.96:0/1924277808 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0042a0 con 0x559e0d933480 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cec227740 1 -- 172.21.15.96:0/1924277808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 msgr2=0x559e0d933860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cec227740 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d933860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006010 tx=0x7f9cdc009a10 comp rx=0 tx=0).stop 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cec227740 1 -- 172.21.15.96:0/1924277808 shutdown_connections 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cec227740 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d93f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.805+0000 7f9cec227740 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d942120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9cec227740 1 --2- 172.21.15.96:0/1924277808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d933860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9cec227740 1 -- 172.21.15.96:0/1924277808 >> 172.21.15.96:0/1924277808 conn(0x559e0d9312b0 msgr2=0x559e0d932450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9cec227740 1 -- 172.21.15.96:0/1924277808 shutdown_connections 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9ce0ff9640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 0x7f9ccc0b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9cec227740 1 -- 172.21.15.96:0/1924277808 wait complete. 2024-09-10T22:50:45.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9cec227740 1 -- start start 2024-09-10T22:50:45.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9ce0ff9640 1 -- 172.21.15.96:0/4052705257 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9cd80e5e10 con 0x7f9ce41076a0 2024-09-10T22:50:45.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.806+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 0x7f9ccc0b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 Processor -- start 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933480 0x559e0d93f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 0x559e0d93ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d93cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e0d6fa700 con 0x559e0d93fd30 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e0d834110 con 0x559e0d933480 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559e0d9537e0 con 0x559e0d933da0 2024-09-10T22:50:45.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9cec227740 10 client.0 _ll_register_callbacks cb 0x559e0d84d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 0x7f9ccc0b77d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009430 tx=0x7f9cd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 0x559e0d93ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d93cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 0x559e0d93ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43364/0 (socket says 172.21.15.96:43364) 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce2ffd640 1 -- 172.21.15.96:0/3329757296 learned_addr learned my addr 172.21.15.96:0/3329757296 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.807+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933480 0x559e0d93f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 -- 172.21.15.96:0/3329757296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 msgr2=0x559e0d93ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 0x559e0d93ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 -- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933480 msgr2=0x559e0d93f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933480 0x559e0d93f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 -- 172.21.15.96:0/3329757296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0d861c40 con 0x559e0d93fd30 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d93cfe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006010 tx=0x7f9cdc031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812c1e93ea68a12d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9cc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc04a040 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cdc036030 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9cdc02f810 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.808+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cdc009860 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e0d95f160 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc036030 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9cdc04d030 con 0x559e0d93fd30 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 20 client.16143 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9604b0 0x559e0d9628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 4 client.16143 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0d8abda0 con 0x559e0d9604b0 2024-09-10T22:50:45.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.809+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9604b0 0x559e0d9628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.810+0000 7f9cc2ffd640 10 client.16143 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:45.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.810+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9604b0 0x559e0d9628a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f333a341b98c6de server_cookie=8e9eee7a0dfbbb1a in_seq=0 out_seq=0 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.811+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/3329757296 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e0d8abda0 con 0x559e0d9604b0 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.811+0000 7f9cc2ffd640 10 client.16143 handle_command_reply: tid=0 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.811+0000 7f9cec227740 1 client.16143 shutdown 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 msgr2=0x559e0d93cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d93cfe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006010 tx=0x7f9cdc031700 comp rx=0 tx=0).stop 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 shutdown_connections 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9604b0 0x559e0d9628a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d93cfe0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933da0 0x559e0d93ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 --2- 172.21.15.96:0/3329757296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933480 0x559e0d93f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 >> 172.21.15.96:0/3329757296 conn(0x559e0d9312b0 msgr2=0x559e0d936e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cc2ffd640 0 client.16143 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 shutdown_connections 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.812+0000 7f9cec227740 1 -- 172.21.15.96:0/3329757296 wait complete. 2024-09-10T22:50:45.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.813+0000 7f9cec227740 20 client.16143 trim_cache size 0 max 16384 2024-09-10T22:50:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.843+0000 7f9cec227740 1 Processor -- start 2024-09-10T22:50:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.843+0000 7f9cec227740 1 -- start start 2024-09-10T22:50:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.843+0000 7f9cec227740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d930bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9cec227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d941120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9cec227740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d941660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9cec227740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000e320 con 0x559e0d93fd30 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9cec227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000fe60 con 0x559e0d933da0 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9cec227740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000e9f0 con 0x559e0d933480 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d941120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d930bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d941120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:45828/0 (socket says 172.21.15.96:45828) 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce2ffd640 1 -- 172.21.15.96:0/3254000908 learned_addr learned my addr 172.21.15.96:0/3254000908 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d941660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 -- 172.21.15.96:0/3254000908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 msgr2=0x559e0d941120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d941120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 -- 172.21.15.96:0/3254000908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 msgr2=0x559e0d941660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d941660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce37fe640 1 -- 172.21.15.96:0/3254000908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0d93f0a0 con 0x559e0d933480 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.844+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d941660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.845+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d930bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559e0d962f50 tx=0x7f9cd4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf1ca7f4286e972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.845+0000 7f9cc97fa640 1 -- 172.21.15.96:0/3254000908 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4013d70 con 0x559e0d933480 2024-09-10T22:50:45.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.845+0000 7f9cc97fa640 1 -- 172.21.15.96:0/3254000908 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd401eca0 con 0x559e0d933480 2024-09-10T22:50:45.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.845+0000 7f9cc97fa640 1 -- 172.21.15.96:0/3254000908 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd401c6b0 con 0x559e0d933480 2024-09-10T22:50:45.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 -- 172.21.15.96:0/3254000908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 msgr2=0x559e0d930bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d930bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559e0d962f50 tx=0x7f9cd4016000 comp rx=0 tx=0).stop 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 -- 172.21.15.96:0/3254000908 shutdown_connections 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d941120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d941660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 --2- 172.21.15.96:0/3254000908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d930bc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 -- 172.21.15.96:0/3254000908 >> 172.21.15.96:0/3254000908 conn(0x559e0d9312b0 msgr2=0x559e0d93e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 -- 172.21.15.96:0/3254000908 shutdown_connections 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.846+0000 7f9cec227740 1 -- 172.21.15.96:0/3254000908 wait complete. 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 -- start start 2024-09-10T22:50:45.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 Processor -- start 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d953ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d954140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000f790 con 0x559e0d93fd30 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000f0c0 con 0x559e0d933da0 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.847+0000 7f9cec227740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd000c300 con 0x559e0d933480 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9cec227740 10 client.0 _ll_register_callbacks cb 0x7f9cd40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43388/0 (socket says 172.21.15.96:43388) 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 -- 172.21.15.96:0/2578294922 learned_addr learned my addr 172.21.15.96:0/2578294922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d953ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce3fff640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d954140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 -- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 msgr2=0x559e0d953ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d953ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 -- 172.21.15.96:0/2578294922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 msgr2=0x559e0d954140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d954140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 -- 172.21.15.96:0/2578294922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0d9cd210 con 0x559e0d933480 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9ce37fe640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400a010 tx=0x7f9cd4005170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78a8abd530e9145 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.848+0000 7f9cc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4034030 con 0x559e0d933480 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9cd402c050 con 0x559e0d933480 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd401eca0 con 0x559e0d933480 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cd40055a0 con 0x559e0d933480 2024-09-10T22:50:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 10 client.25759 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e0d9ce390 con 0x559e0d933480 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd401eca0 con 0x559e0d933480 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9cd401a040 con 0x559e0d933480 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 10 client.25759 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 20 client.25759 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9ceb60 0x559e0d9d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 4 client.25759 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.849+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e0d8a7850 con 0x559e0d9ceb60 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9ceb60 0x559e0d9d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9ce2ffd640 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9ceb60 0x559e0d9d0f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de02d7ef21556df3 server_cookie=646ed50dbf112daa in_seq=0 out_seq=0 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9cc2ffd640 10 client.25759 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:45.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9cc2ffd640 1 -- 172.21.15.96:0/2578294922 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x559e0d8a7850 con 0x559e0d9ceb60 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9cc2ffd640 10 client.25759 handle_command_reply: tid=0 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.850+0000 7f9cec227740 1 client.25759 shutdown 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 msgr2=0x559e0d9593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400a010 tx=0x7f9cd4005170 comp rx=0 tx=0).stop 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 shutdown_connections 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559e0d9ceb60 0x559e0d9d0f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e0d933da0 0x559e0d953ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0d93fd30 0x559e0d954140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 --2- 172.21.15.96:0/2578294922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559e0d933480 0x559e0d9593c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 >> 172.21.15.96:0/2578294922 conn(0x559e0d9312b0 msgr2=0x559e0d93b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cc2ffd640 0 client.25759 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 shutdown_connections 2024-09-10T22:50:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 1 -- 172.21.15.96:0/2578294922 wait complete. 2024-09-10T22:50:45.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.851+0000 7f9cec227740 20 client.25759 trim_cache size 0 max 16384 2024-09-10T22:50:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:50:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:50:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:50:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "203971b1-c5e2-4181-ab92-43d1598bdd3f", 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:50:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:50:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:50:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 msgr2=0x7f9ccc0b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 0x7f9ccc0b77d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009430 tx=0x7f9cd4005f90 comp rx=0 tx=0).stop 2024-09-10T22:50:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 msgr2=0x7f9ce419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e9f0 tx=0x7f9cd800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 shutdown_connections 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce4108020 0x7f9ce41a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9ccc0b5310 0x7f9ccc0b77d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce41076a0 0x7f9ce419b9d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 --2- 172.21.15.96:0/4052705257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce4101c90 0x7f9ce419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 >> 172.21.15.96:0/4052705257 conn(0x7f9ce40fd700 msgr2=0x7f9ce406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 shutdown_connections 2024-09-10T22:50:45.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:50:45.852+0000 7f9ce9e52640 1 -- 172.21.15.96:0/4052705257 wait complete. 2024-09-10T22:50:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 190.0 s, 3569.0 tps, lat 28.001 ms stddev 49.149 2024-09-10T22:50:46.680 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:46 smithi121 ceph-mon[35119]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:46.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:46 smithi096 ceph-mon[30869]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:46.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:46 smithi117 ceph-mon[35081]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 48 MiB/s wr, 1.01k op/s 2024-09-10T22:50:47.541 INFO:tasks.cephfs.filesystem:scrub status for tag:203971b1-c5e2-4181-ab92-43d1598bdd3f - {'path': '/', 'tag': '203971b1-c5e2-4181-ab92-43d1598bdd3f', 'options': 'recursive,force'} 2024-09-10T22:50:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:48 smithi096 ceph-mon[30869]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 39 MiB/s wr, 885 op/s 2024-09-10T22:50:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:48 smithi121 ceph-mon[35119]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 39 MiB/s wr, 885 op/s 2024-09-10T22:50:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:48 smithi117 ceph-mon[35081]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 39 MiB/s wr, 885 op/s 2024-09-10T22:50:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:49 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:50:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:49 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:50:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:49 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:50:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:50 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:50:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:50 smithi096 ceph-mon[30869]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 63 MiB/s wr, 1.62k op/s 2024-09-10T22:50:50.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:50 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:50:50.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:50 smithi121 ceph-mon[35119]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 63 MiB/s wr, 1.62k op/s 2024-09-10T22:50:50.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:50 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:50:50.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:50 smithi117 ceph-mon[35081]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 63 MiB/s wr, 1.62k op/s 2024-09-10T22:50:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 195.0 s, 4427.6 tps, lat 22.545 ms stddev 26.760 2024-09-10T22:50:52.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:52 smithi096 ceph-mon[30869]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 61 MiB/s wr, 1.70k op/s 2024-09-10T22:50:52.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:52 smithi121 ceph-mon[35119]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 61 MiB/s wr, 1.70k op/s 2024-09-10T22:50:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:52 smithi117 ceph-mon[35081]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 61 MiB/s wr, 1.70k op/s 2024-09-10T22:50:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.688+0000 7f8e13fff640 1 -- 172.21.15.96:0/3909772259 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1802f310 con 0x7f8e1c104490 2024-09-10T22:50:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 -- 172.21.15.96:0/3909772259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 msgr2=0x7f8e1c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 --2- 172.21.15.96:0/3909772259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c10ae10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009ca0 tx=0x7f8e1802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 -- 172.21.15.96:0/3909772259 shutdown_connections 2024-09-10T22:50:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 --2- 172.21.15.96:0/3909772259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c10ae10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 --2- 172.21.15.96:0/3909772259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 0x7f8e1c103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 --2- 172.21.15.96:0/3909772259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.689+0000 7f8e234b1640 1 -- 172.21.15.96:0/3909772259 >> 172.21.15.96:0/3909772259 conn(0x7f8e1c0fe240 msgr2=0x7f8e1c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.690+0000 7f8e234b1640 1 -- 172.21.15.96:0/3909772259 shutdown_connections 2024-09-10T22:50:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.690+0000 7f8e234b1640 1 -- 172.21.15.96:0/3909772259 wait complete. 2024-09-10T22:50:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 Processor -- start 2024-09-10T22:50:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 -- start start 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 0x7f8e1c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c10e620 con 0x7f8e1c102980 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c10e4a0 con 0x7f8e1c104490 2024-09-10T22:50:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e234b1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c10e7a0 con 0x7f8e1c103ad0 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.691+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e20a4c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 0x7f8e1c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55230/0 (socket says 172.21.15.96:55230) 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e21a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 -- 172.21.15.96:0/438262628 learned_addr learned my addr 172.21.15.96:0/438262628 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 -- 172.21.15.96:0/438262628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 msgr2=0x7f8e1c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 0x7f8e1c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 -- 172.21.15.96:0/438262628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 msgr2=0x7f8e1c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 -- 172.21.15.96:0/438262628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c19a7c0 con 0x7f8e1c102980 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e2124d640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c920 tx=0x7f8e0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.692+0000 7f8e127fc640 1 -- 172.21.15.96:0/438262628 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c007c70 con 0x7f8e1c102980 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.693+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c19aab0 con 0x7f8e1c102980 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.693+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c06cd50 con 0x7f8e1c102980 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.693+0000 7f8e127fc640 1 -- 172.21.15.96:0/438262628 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e0c007e10 con 0x7f8e1c102980 2024-09-10T22:50:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.693+0000 7f8e127fc640 1 -- 172.21.15.96:0/438262628 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c018820 con 0x7f8e1c102980 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.696+0000 7f8e26277740 1 Processor -- start 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 -- start start 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c204c70 0x563a4c207060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c2075a0 0x563a4c209990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a4c1726f0 con 0x563a4c204c70 2024-09-10T22:50:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a4c161b20 con 0x563a4c209fe0 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e26277740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563a4c165930 con 0x563a4c2075a0 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:53782/0 (socket says 172.21.15.96:53782) 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2795347614 learned_addr learned my addr 172.21.15.96:0/2795347614 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c204c70 0x563a4c207060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2795347614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c2075a0 msgr2=0x563a4c209990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c2075a0 0x563a4c209990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2795347614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c204c70 msgr2=0x563a4c207060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c204c70 0x563a4c207060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.697+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2795347614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4c127530 con 0x563a4c209fe0 2024-09-10T22:50:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.698+0000 7f8e127fc640 1 -- 172.21.15.96:0/438262628 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f8e0c00fa90 con 0x7f8e1c102980 2024-09-10T22:50:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.698+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bd10 tx=0x7f8e140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96476a59fc147a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.698+0000 7f8e077fe640 1 -- 172.21.15.96:0/2795347614 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14004030 con 0x563a4c209fe0 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.698+0000 7f8e077fe640 1 -- 172.21.15.96:0/2795347614 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e140035f0 con 0x563a4c209fe0 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.698+0000 7f8e077fe640 1 -- 172.21.15.96:0/2795347614 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14011770 con 0x563a4c209fe0 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.699+0000 7f8e127fc640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 0x7f8e080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.699+0000 7f8e127fc640 1 -- 172.21.15.96:0/438262628 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8e0c0e5ba0 con 0x7f8e1c102980 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.699+0000 7f8e26277740 1 -- 172.21.15.96:0/2795347614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 msgr2=0x563a4c20c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.699+0000 7f8e26277740 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bd10 tx=0x7f8e140107a0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 0x7f8e080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 -- 172.21.15.96:0/2795347614 shutdown_connections 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20c4a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c2075a0 0x563a4c209990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 --2- 172.21.15.96:0/2795347614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c204c70 0x563a4c207060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 -- 172.21.15.96:0/2795347614 >> 172.21.15.96:0/2795347614 conn(0x563a4c1fb9b0 msgr2=0x563a4c1fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 -- 172.21.15.96:0/2795347614 shutdown_connections 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e26277740 1 -- 172.21.15.96:0/2795347614 wait complete. 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.700+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 0x7f8e080b7820 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005e30 tx=0x7f8e18031ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.701+0000 7f8e26277740 1 -- start start 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.701+0000 7f8e26277740 1 Processor -- start 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 0x563a4c20e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 0x563a4c20f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a4c099ab0 con 0x563a4c209fe0 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a4c107730 con 0x563a4c2075a0 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563a4c20cfc0 con 0x563a4c204c70 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e26277740 10 client.0 _ll_register_callbacks cb 0x563a4c15b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e21a4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 0x563a4c20f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e20a4c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 0x563a4c20e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e21a4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 0x563a4c20f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55240/0 (socket says 172.21.15.96:55240) 2024-09-10T22:50:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.702+0000 7f8e21a4e640 1 -- 172.21.15.96:0/2650746710 learned_addr learned my addr 172.21.15.96:0/2650746710 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2650746710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 msgr2=0x563a4c20e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2650746710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 msgr2=0x563a4c20f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 0x563a4c20f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 -- 172.21.15.96:0/2650746710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4bec3790 con 0x563a4c2075a0 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 0x563a4c20e9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bd10 tx=0x7f8e14003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5788860e2a47a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14029060 con 0x563a4c2075a0 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8e140041e0 con 0x563a4c2075a0 2024-09-10T22:50:53.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e140035f0 con 0x563a4c2075a0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.703+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e14003b30 con 0x563a4c2075a0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.705+0000 7f8e26277740 10 client.25712 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.705+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a4bf3c620 con 0x563a4c2075a0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.705+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14004400 con 0x563a4c2075a0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.705+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8e1402e070 con 0x563a4c2075a0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.705+0000 7f8e26277740 10 client.25712 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e26277740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e26277740 20 client.25712 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c203420 0x563a4c203860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e26277740 4 client.25712 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8e140035f0 con 0x563a4c203420 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.706+0000 7f8e2124d640 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c203420 0x563a4c203860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e2124d640 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c203420 0x563a4c203860 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c8df42c2519161 server_cookie=736c986cef659526 in_seq=0 out_seq=0 2024-09-10T22:50:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e04ff9640 10 client.25712 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e04ff9640 1 -- 172.21.15.96:0/2650746710 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8e140035f0 con 0x563a4c203420 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e04ff9640 10 client.25712 handle_command_reply: tid=0 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e26277740 1 client.25712 shutdown 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 msgr2=0x563a4c20e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.707+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 0x563a4c20e9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400bd10 tx=0x7f8e14003df0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 shutdown_connections 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c203420 0x563a4c203860 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e04ff9640 0 client.25712 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c2075a0 0x563a4c20e9b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c209fe0 0x563a4c20f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 --2- 172.21.15.96:0/2650746710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 >> 172.21.15.96:0/2650746710 conn(0x563a4c1fb9b0 msgr2=0x563a4c1fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 shutdown_connections 2024-09-10T22:50:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 1 -- 172.21.15.96:0/2650746710 wait complete. 2024-09-10T22:50:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.708+0000 7f8e26277740 20 client.25712 trim_cache size 0 max 16384 2024-09-10T22:50:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.732+0000 7f8e26277740 1 Processor -- start 2024-09-10T22:50:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.732+0000 7f8e26277740 1 -- start start 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e26277740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e26277740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c20e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:33734/0 (socket says 172.21.15.96:33734) 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e2124d640 1 -- 172.21.15.96:0/3291865827 learned_addr learned my addr 172.21.15.96:0/3291865827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e26277740 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.733+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000e330 con 0x563a4c2075a0 2024-09-10T22:50:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000fe70 con 0x563a4c209fe0 2024-09-10T22:50:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c20e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000ea00 con 0x563a4c204c70 2024-09-10T22:50:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3291865827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 msgr2=0x563a4c20e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3291865827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 msgr2=0x563a4c20e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c20e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3291865827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4c1ff970 con 0x563a4c209fe0 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.734+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20d570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e18031a10 tx=0x7f8e18041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0e167a1b96a226 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.735+0000 7f8e06ffd640 1 -- 172.21.15.96:0/3291865827 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18031720 con 0x563a4c209fe0 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.735+0000 7f8e06ffd640 1 -- 172.21.15.96:0/3291865827 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e18036050 con 0x563a4c209fe0 2024-09-10T22:50:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.735+0000 7f8e06ffd640 1 -- 172.21.15.96:0/3291865827 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180046a0 con 0x563a4c209fe0 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 msgr2=0x563a4c20d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20d570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e18031a10 tx=0x7f8e18041000 comp rx=0 tx=0).stop 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 shutdown_connections 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c20d570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c20e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 --2- 172.21.15.96:0/3291865827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c20e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 >> 172.21.15.96:0/3291865827 conn(0x563a4c1fb9b0 msgr2=0x563a4c1ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 shutdown_connections 2024-09-10T22:50:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- 172.21.15.96:0/3291865827 wait complete. 2024-09-10T22:50:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.736+0000 7f8e26277740 1 -- start start 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.737+0000 7f8e26277740 1 Processor -- start 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c2130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c213cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000f7a0 con 0x563a4c2075a0 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000f0d0 con 0x563a4c209fe0 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0000c310 con 0x563a4c204c70 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e2124d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:33738/0 (socket says 172.21.15.96:33738) 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c213cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e2124d640 1 -- 172.21.15.96:0/3219238360 learned_addr learned my addr 172.21.15.96:0/3219238360 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e20a4c640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c2130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e26277740 10 client.0 _ll_register_callbacks cb 0x7f8e18005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3219238360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 msgr2=0x563a4c212a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3219238360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 msgr2=0x563a4c2130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c2130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e21a4e640 1 -- 172.21.15.96:0/3219238360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4c2144e0 con 0x563a4c209fe0 2024-09-10T22:50:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.738+0000 7f8e2124d640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:50:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.739+0000 7f8e21a4e640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c213cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e18043040 tx=0x7f8e180093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de35194f6753461 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:50:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.739+0000 7f8e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:50:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.739+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1802f820 con 0x563a4c209fe0 2024-09-10T22:50:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.739+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8e1802f9c0 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.739+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e18036050 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e1802fbc0 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 10 client.25718 fetch_fsmap learned FSMap version 32 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a4c215880 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1802fd60 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8e18065030 con 0x563a4c209fe0 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 20 client.25718 populate_metadata read hostname 'smithi096' 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c216260 0x563a4c28d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 4 client.25718 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8e18036050 con 0x563a4c216260 2024-09-10T22:50:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.740+0000 7f8e2124d640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c216260 0x563a4c28d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:50:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.741+0000 7f8e2124d640 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c216260 0x563a4c28d7c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452fa1c9df902e4f server_cookie=6f32326ed45670e in_seq=0 out_seq=0 2024-09-10T22:50:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.741+0000 7f8e04ff9640 10 client.25718 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.781+0000 7f8e04ff9640 1 -- 172.21.15.96:0/3219238360 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8e18036050 con 0x563a4c216260 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.781+0000 7f8e04ff9640 10 client.25718 handle_command_reply: tid=0 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.781+0000 7f8e26277740 1 client.25718 shutdown 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 msgr2=0x563a4c213cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c213cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e18043040 tx=0x7f8e180093d0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 shutdown_connections 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a4c216260 0x563a4c28d7c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a4c209fe0 0x563a4c213cb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a4c2075a0 0x563a4c2130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 --2- 172.21.15.96:0/3219238360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a4c204c70 0x563a4c212a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e04ff9640 0 client.25718 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 >> 172.21.15.96:0/3219238360 conn(0x563a4c1fb9b0 msgr2=0x563a4c1fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 shutdown_connections 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 1 -- 172.21.15.96:0/3219238360 wait complete. 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.782+0000 7f8e26277740 20 client.25718 trim_cache size 0 max 16384 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:50:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 msgr2=0x7f8e080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 0x7f8e080b7820 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005e30 tx=0x7f8e18031ae0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 msgr2=0x7f8e1c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c920 tx=0x7f8e0c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 shutdown_connections 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c104490 0x7f8e1c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8e080b5360 0x7f8e080b7820 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e1c103ad0 0x7f8e1c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 --2- 172.21.15.96:0/438262628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c102980 0x7f8e1c198c80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:50:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.783+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 >> 172.21.15.96:0/438262628 conn(0x7f8e1c0fe240 msgr2=0x7f8e1c106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:50:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.784+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 shutdown_connections 2024-09-10T22:50:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:50:53.784+0000 7f8e234b1640 1 -- 172.21.15.96:0/438262628 wait complete. 2024-09-10T22:50:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:54 smithi096 ceph-mon[30869]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 58 MiB/s wr, 1.73k op/s 2024-09-10T22:50:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:54 smithi121 ceph-mon[35119]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 58 MiB/s wr, 1.73k op/s 2024-09-10T22:50:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:54 smithi117 ceph-mon[35081]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 58 MiB/s wr, 1.73k op/s 2024-09-10T22:50:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 200.0 s, 4130.8 tps, lat 24.212 ms stddev 27.207 2024-09-10T22:50:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:56 smithi096 ceph-mon[30869]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2024-09-10T22:50:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:56 smithi121 ceph-mon[35119]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2024-09-10T22:50:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:56 smithi117 ceph-mon[35081]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2024-09-10T22:50:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:50:58 smithi096 ceph-mon[30869]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 45 MiB/s wr, 1.50k op/s 2024-09-10T22:50:58.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:50:58 smithi121 ceph-mon[35119]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 45 MiB/s wr, 1.50k op/s 2024-09-10T22:50:58.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:50:58 smithi117 ceph-mon[35081]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 45 MiB/s wr, 1.50k op/s 2024-09-10T22:51:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:00 smithi096 ceph-mon[30869]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 46 MiB/s wr, 1.66k op/s 2024-09-10T22:51:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:00 smithi121 ceph-mon[35119]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 46 MiB/s wr, 1.66k op/s 2024-09-10T22:51:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:00 smithi117 ceph-mon[35081]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 46 MiB/s wr, 1.66k op/s 2024-09-10T22:51:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 205.0 s, 3888.4 tps, lat 25.727 ms stddev 27.489 2024-09-10T22:51:02.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:02 smithi096 ceph-mon[30869]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-09-10T22:51:02.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:02 smithi121 ceph-mon[35119]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-09-10T22:51:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:02 smithi117 ceph-mon[35081]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-09-10T22:51:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:04 smithi121 ceph-mon[35119]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-09-10T22:51:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:04 smithi117 ceph-mon[35081]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-09-10T22:51:05.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:04 smithi096 ceph-mon[30869]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-09-10T22:51:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 210.0 s, 3452.0 tps, lat 28.716 ms stddev 32.143 2024-09-10T22:51:06.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:06 smithi121 ceph-mon[35119]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-09-10T22:51:06.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:06 smithi117 ceph-mon[35081]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-09-10T22:51:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.903+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/4220235186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 msgr2=0x7fb3f0105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.903+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/4220235186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0105530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009be0 tx=0x7fb3e802a6c0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/4220235186 shutdown_connections 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/4220235186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/4220235186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 0x7fb3f010ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/4220235186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0105530 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/4220235186 >> 172.21.15.96:0/4220235186 conn(0x7fb3f00fd140 msgr2=0x7fb3f00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.904+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/4220235186 shutdown_connections 2024-09-10T22:51:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.905+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/4220235186 wait complete. 2024-09-10T22:51:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 Processor -- start 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 -- start start 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f00722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 0x7fb3f0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f010c410 con 0x7fb3f0108670 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f010c290 con 0x7fb3f0105ac0 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3f4ea9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f010c590 con 0x7fb3f006b470 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44408/0 (socket says 172.21.15.96:44408) 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3ee59c640 1 -- 172.21.15.96:0/2755959328 learned_addr learned my addr 172.21.15.96:0/2755959328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f00722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.906+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 0x7fb3f0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 -- 172.21.15.96:0/2755959328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 msgr2=0x7fb3f00722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f00722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 -- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 msgr2=0x7fb3f0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 0x7fb3f0072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 -- 172.21.15.96:0/2755959328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f00737c0 con 0x7fb3f006b470 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f00722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3e802acb0 tx=0x7fb3e8030de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3df7fe640 1 -- 172.21.15.96:0/2755959328 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8041020 con 0x7fb3f006b470 2024-09-10T22:51:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3df7fe640 1 -- 172.21.15.96:0/2755959328 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80318f0 con 0x7fb3f006b470 2024-09-10T22:51:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f0073a50 con 0x7fb3f006b470 2024-09-10T22:51:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3df7fe640 1 -- 172.21.15.96:0/2755959328 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8036040 con 0x7fb3f006b470 2024-09-10T22:51:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.907+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f0074150 con 0x7fb3f006b470 2024-09-10T22:51:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.910+0000 7fb3f7c72740 1 Processor -- start 2024-09-10T22:51:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 -- start start 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 0x565176d8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565176cf26f0 con 0x565176d8e3e0 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565176ce1b20 con 0x565176d8b9a0 2024-09-10T22:51:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.911+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x565176ce5930 con 0x565176d89070 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3edd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44416/0 (socket says 172.21.15.96:44416) 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3eed9d640 1 -- 172.21.15.96:0/3907311564 learned_addr learned my addr 172.21.15.96:0/3907311564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3df7fe640 1 -- 172.21.15.96:0/2755959328 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb3e8004670 con 0x7fb3f006b470 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.912+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 0x565176d8dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 -- 172.21.15.96:0/3907311564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 msgr2=0x565176d8dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 0x565176d8dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 -- 172.21.15.96:0/3907311564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 msgr2=0x565176d908a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d908a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 -- 172.21.15.96:0/3907311564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565176ca7530 con 0x565176d89070 2024-09-10T22:51:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 0x565176d8dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.913+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009c20 tx=0x7fb3e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a228081bdbbb1df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.914+0000 7fb3dcff9640 1 -- 172.21.15.96:0/3907311564 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4004230 con 0x565176d89070 2024-09-10T22:51:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.914+0000 7fb3dcff9640 1 -- 172.21.15.96:0/3907311564 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e40043d0 con 0x565176d89070 2024-09-10T22:51:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 -- 172.21.15.96:0/3907311564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 msgr2=0x565176d8b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009c20 tx=0x7fb3e4011810 comp rx=0 tx=0).stop 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 -- 172.21.15.96:0/3907311564 shutdown_connections 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8b9a0 0x565176d8dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d908a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/3907311564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d8b460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 -- 172.21.15.96:0/3907311564 >> 172.21.15.96:0/3907311564 conn(0x565176d841c0 msgr2=0x565176d84f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 -- 172.21.15.96:0/3907311564 shutdown_connections 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3f7c72740 1 -- 172.21.15.96:0/3907311564 wait complete. 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3df7fe640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 0x7fb3d80b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.915+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 0x7fb3d80b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.916+0000 7fb3df7fe640 1 -- 172.21.15.96:0/2755959328 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb3e8106b40 con 0x7fb3f006b470 2024-09-10T22:51:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.916+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 0x7fb3d80b7660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00058c0 tx=0x7fb3e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.916+0000 7fb3f7c72740 1 -- start start 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.916+0000 7fb3f7c72740 1 Processor -- start 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d92dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d93990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565176c19ab0 con 0x565176d8e3e0 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565176c87730 con 0x565176d89070 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x565176d91410 con 0x565176d8b9a0 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3f7c72740 10 client.0 _ll_register_callbacks cb 0x565176cdb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56334/0 (socket says 172.21.15.96:56334) 2024-09-10T22:51:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1929555632 learned_addr learned my addr 172.21.15.96:0/1929555632 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d92dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d93990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1929555632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 msgr2=0x565176d92dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d92dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1929555632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 msgr2=0x565176d93990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.917+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d93990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1929555632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565176a43790 con 0x565176d89070 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80ca610 tx=0x7fb3e80c9240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61bae886e53a8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80ccb10 con 0x565176d89070 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb3e80d66a0 con 0x565176d89070 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e40043d0 con 0x565176d89070 2024-09-10T22:51:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3e80cceb0 con 0x565176d89070 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3f7c72740 10 client.25724 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565176abc620 con 0x565176d89070 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e40043d0 con 0x565176d89070 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb3e80f40d0 con 0x565176d89070 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3f7c72740 10 client.25724 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.918+0000 7fb3f7c72740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3f7c72740 20 client.25724 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9de30 0x565176da02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3f7c72740 4 client.25724 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565176cf4640 con 0x565176d9de30 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9de30 0x565176da02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3d5ffb640 10 client.25724 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.919+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9de30 0x565176da02d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95842f86dc8a36ef server_cookie=91dc744179398098 in_seq=0 out_seq=0 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/1929555632 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565176cf4640 con 0x565176d9de30 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3d5ffb640 10 client.25724 handle_command_reply: tid=0 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 client.25724 shutdown 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 msgr2=0x565176d92720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80ca610 tx=0x7fb3e80c9240 comp rx=0 tx=0).stop 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 shutdown_connections 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9de30 0x565176da02d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d93990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3d5ffb640 0 client.25724 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d92dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1929555632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d92720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 >> 172.21.15.96:0/1929555632 conn(0x565176d841c0 msgr2=0x565176d7abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 shutdown_connections 2024-09-10T22:51:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1929555632 wait complete. 2024-09-10T22:51:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.920+0000 7fb3f7c72740 20 client.25724 trim_cache size 0 max 16384 2024-09-10T22:51:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.947+0000 7fb3f7c72740 1 Processor -- start 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 -- start start 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b0310 con 0x565176d8e3e0 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b1e50 con 0x565176d89070 2024-09-10T22:51:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.953+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b09e0 con 0x565176d8b9a0 2024-09-10T22:51:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.959+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.959+0000 7fb3ee59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56350/0 (socket says 172.21.15.96:56350) 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.959+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1072432562 learned_addr learned my addr 172.21.15.96:0/1072432562 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.960+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.960+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1072432562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 msgr2=0x565176d884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1072432562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 msgr2=0x565176d88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 -- 172.21.15.96:0/1072432562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565176da0950 con 0x565176d89070 2024-09-10T22:51:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.962+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8004af0 tx=0x7fb3e8004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f687a336ac36eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.965+0000 7fb3d7fff640 1 -- 172.21.15.96:0/1072432562 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80cd170 con 0x565176d89070 2024-09-10T22:51:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.965+0000 7fb3d7fff640 1 -- 172.21.15.96:0/1072432562 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80cdaa0 con 0x565176d89070 2024-09-10T22:51:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.966+0000 7fb3d7fff640 1 -- 172.21.15.96:0/1072432562 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b4b30 con 0x565176d89070 2024-09-10T22:51:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.967+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1072432562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 msgr2=0x565176d84c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.967+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8004af0 tx=0x7fb3e8004140 comp rx=0 tx=0).stop 2024-09-10T22:51:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.969+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1072432562 shutdown_connections 2024-09-10T22:51:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.969+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8e3e0 0x565176d88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.969+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d8b9a0 0x565176d884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.969+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/1072432562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d89070 0x565176d84c60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.969+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1072432562 >> 172.21.15.96:0/1072432562 conn(0x565176d841c0 msgr2=0x565176d922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.977+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1072432562 shutdown_connections 2024-09-10T22:51:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.977+0000 7fb3f7c72740 1 -- 172.21.15.96:0/1072432562 wait complete. 2024-09-10T22:51:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.977+0000 7fb3f7c72740 1 -- start start 2024-09-10T22:51:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.978+0000 7fb3f7c72740 1 Processor -- start 2024-09-10T22:51:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.978+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.978+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 0x565176d97060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.978+0000 7fb3f7c72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.978+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b1780 con 0x565176d8b9a0 2024-09-10T22:51:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3f7c72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01b10b0 con 0x565176d8e3e0 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56366/0 (socket says 172.21.15.96:56366) 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 -- 172.21.15.96:0/81382056 learned_addr learned my addr 172.21.15.96:0/81382056 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 0x565176d97060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01ae2f0 con 0x565176d89070 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3f7c72740 10 client.0 _ll_register_callbacks cb 0x7fb3e8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 -- 172.21.15.96:0/81382056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 msgr2=0x565176d969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 -- 172.21.15.96:0/81382056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 msgr2=0x565176d97060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 0x565176d97060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3eed9d640 1 -- 172.21.15.96:0/81382056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565176d98460 con 0x565176d8e3e0 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.979+0000 7fb3edd9b640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 0x565176d97060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.980+0000 7fb3eed9d640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006010 tx=0x7fb3e40056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a737859fc1afad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.980+0000 7fb3d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.980+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4009e90 con 0x565176d8e3e0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.980+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e80cdaa0 con 0x565176d8e3e0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.980+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb3e40035f0 con 0x565176d8e3e0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.981+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3e4033050 con 0x565176d8e3e0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.981+0000 7fb3f7c72740 10 client.25730 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.981+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565176d99850 con 0x565176d8e3e0 2024-09-10T22:51:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.981+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e80cdaa0 con 0x565176d8e3e0 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb3e4004550 con 0x565176d8e3e0 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 10 client.25730 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 20 client.25730 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9a230 0x565176d82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 4 client.25730 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:51:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.982+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565176cf00f0 con 0x565176d9a230 2024-09-10T22:51:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.983+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9a230 0x565176d82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.985+0000 7fb3ee59c640 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9a230 0x565176d82410 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b639c19e3c7351 server_cookie=6d868fd40480c7df in_seq=0 out_seq=0 2024-09-10T22:51:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.985+0000 7fb3d5ffb640 10 client.25730 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3d5ffb640 1 -- 172.21.15.96:0/81382056 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565176cf00f0 con 0x565176d9a230 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3d5ffb640 10 client.25730 handle_command_reply: tid=0 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 client.25730 shutdown 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 msgr2=0x565176d97c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006010 tx=0x7fb3e40056a0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 shutdown_connections 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x565176d9a230 0x565176d82410 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565176d8e3e0 0x565176d97c30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x565176d8b9a0 0x565176d97060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:06 smithi096 ceph-mon[30869]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 864 op/s 2024-09-10T22:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 --2- 172.21.15.96:0/81382056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565176d89070 0x565176d969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3d5ffb640 0 client.25730 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.998+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 >> 172.21.15.96:0/81382056 conn(0x565176d841c0 msgr2=0x565176d84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.999+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 shutdown_connections 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.999+0000 7fb3f7c72740 1 -- 172.21.15.96:0/81382056 wait complete. 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:06.999+0000 7fb3f7c72740 20 client.25730 trim_cache size 0 max 16384 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:51:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 msgr2=0x7fb3d80b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 0x7fb3d80b7660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00058c0 tx=0x7fb3e000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 msgr2=0x7fb3f0071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3e802acb0 tx=0x7fb3e8030de0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 shutdown_connections 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3f0105ac0 0x7fb3f00722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb3d80b51a0 0x7fb3d80b7660 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3f0108670 0x7fb3f0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 --2- 172.21.15.96:0/2755959328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f006b470 0x7fb3f0071c80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 >> 172.21.15.96:0/2755959328 conn(0x7fb3f00fd140 msgr2=0x7fb3f0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 shutdown_connections 2024-09-10T22:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:07.000+0000 7fb3f4ea9640 1 -- 172.21.15.96:0/2755959328 wait complete. 2024-09-10T22:51:08.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:08 smithi121 ceph-mon[35119]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 686 op/s 2024-09-10T22:51:08.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:08 smithi117 ceph-mon[35081]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 686 op/s 2024-09-10T22:51:09.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:08 smithi096 ceph-mon[30869]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 MiB/s wr, 686 op/s 2024-09-10T22:51:10.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:10 smithi121 ceph-mon[35119]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 886 op/s 2024-09-10T22:51:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:10 smithi117 ceph-mon[35081]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 886 op/s 2024-09-10T22:51:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 215.0 s, 4012.0 tps, lat 25.114 ms stddev 27.638 2024-09-10T22:51:11.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:10 smithi096 ceph-mon[30869]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 886 op/s 2024-09-10T22:51:12.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:12 smithi121 ceph-mon[35119]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:51:12.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:12 smithi117 ceph-mon[35081]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:51:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:12 smithi096 ceph-mon[30869]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-10T22:51:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:14 smithi121 ceph-mon[35119]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 860 op/s 2024-09-10T22:51:14.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:14 smithi117 ceph-mon[35081]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 860 op/s 2024-09-10T22:51:15.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:14 smithi096 ceph-mon[30869]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 860 op/s 2024-09-10T22:51:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 220.0 s, 3277.8 tps, lat 30.169 ms stddev 34.733 2024-09-10T22:51:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:16 smithi117 ceph-mon[35081]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T22:51:17.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:16 smithi096 ceph-mon[30869]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T22:51:17.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:16 smithi121 ceph-mon[35119]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T22:51:17.543 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:51:17.910 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:18.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 -- 172.21.15.96:0/1370054210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 msgr2=0x7f625410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 --2- 172.21.15.96:0/1370054210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f625410cae0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6248009950 tx=0x7f624802a480 comp rx=0 tx=0).stop 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 -- 172.21.15.96:0/1370054210 shutdown_connections 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 --2- 172.21.15.96:0/1370054210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f625410d130 0x7f625410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 --2- 172.21.15.96:0/1370054210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f625410cae0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 --2- 172.21.15.96:0/1370054210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6254103cd0 0x7f62541040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 -- 172.21.15.96:0/1370054210 >> 172.21.15.96:0/1370054210 conn(0x7f62540fd770 msgr2=0x7f62540ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 -- 172.21.15.96:0/1370054210 shutdown_connections 2024-09-10T22:51:18.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.456+0000 7f6258c08640 1 -- 172.21.15.96:0/1370054210 wait complete. 2024-09-10T22:51:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.458+0000 7f6258c08640 1 Processor -- start 2024-09-10T22:51:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.458+0000 7f6258c08640 1 -- start start 2024-09-10T22:51:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f62541997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f625259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f625259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59142/0 (socket says 172.21.15.96:59142) 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 0x7f625419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6254112da0 con 0x7f62541045f0 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6254112c20 con 0x7f625410d130 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f6258c08640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6254112f20 con 0x7f6254103cd0 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.459+0000 7f625259c640 1 -- 172.21.15.96:0/2610606739 learned_addr learned my addr 172.21.15.96:0/2610606739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f6252d9d640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 0x7f625419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f62541997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f625259c640 1 -- 172.21.15.96:0/2610606739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 msgr2=0x7f625419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f625259c640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 0x7f625419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f625259c640 1 -- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 msgr2=0x7f62541997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f625259c640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f62541997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f625259c640 1 -- 172.21.15.96:0/2610606739 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625419f320 con 0x7f6254103cd0 2024-09-10T22:51:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.460+0000 7f6252d9d640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 0x7f625419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.461+0000 7f625259c640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f624400c920 tx=0x7f624400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.462+0000 7f62437fe640 1 -- 172.21.15.96:0/2610606739 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244007c70 con 0x7f6254103cd0 2024-09-10T22:51:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.462+0000 7f62437fe640 1 -- 172.21.15.96:0/2610606739 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6244007e10 con 0x7f6254103cd0 2024-09-10T22:51:18.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.462+0000 7f62437fe640 1 -- 172.21.15.96:0/2610606739 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244018820 con 0x7f6254103cd0 2024-09-10T22:51:18.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.462+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625419f610 con 0x7f6254103cd0 2024-09-10T22:51:18.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.462+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6254101a30 con 0x7f6254103cd0 2024-09-10T22:51:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.465+0000 7f625afdd740 1 Processor -- start 2024-09-10T22:51:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 -- start start 2024-09-10T22:51:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdbf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbd35e50 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbd25280 con 0x55a6dbdbf9c0 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f625afdd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbd29090 con 0x55a6dbdd4160 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59152/0 (socket says 172.21.15.96:59152) 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.466+0000 7f6251d9b640 1 -- 172.21.15.96:0/2865203565 learned_addr learned my addr 172.21.15.96:0/2865203565 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6252d9d640 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdbf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 -- 172.21.15.96:0/2865203565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf9c0 msgr2=0x55a6dbdd3c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdd3c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 -- 172.21.15.96:0/2865203565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 msgr2=0x55a6dbdbf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdbf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 -- 172.21.15.96:0/2865203565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dba83650 con 0x55a6dbdd4160 2024-09-10T22:51:18.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.467+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6248000c00 tx=0x7f624802a980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe160dd53b7816fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.468+0000 7f6240ff9640 1 -- 172.21.15.96:0/2865203565 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248049070 con 0x55a6dbdd4160 2024-09-10T22:51:18.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.468+0000 7f6240ff9640 1 -- 172.21.15.96:0/2865203565 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62480048c0 con 0x55a6dbdd4160 2024-09-10T22:51:18.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.468+0000 7f62437fe640 1 -- 172.21.15.96:0/2610606739 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f624400f450 con 0x7f6254103cd0 2024-09-10T22:51:18.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.468+0000 7f6240ff9640 1 -- 172.21.15.96:0/2865203565 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248044660 con 0x55a6dbdd4160 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 -- 172.21.15.96:0/2865203565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 msgr2=0x55a6dbdd6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6248000c00 tx=0x7f624802a980 comp rx=0 tx=0).stop 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 -- 172.21.15.96:0/2865203565 shutdown_connections 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd6550 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 --2- 172.21.15.96:0/2865203565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdbf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f625afdd740 1 -- 172.21.15.96:0/2865203565 >> 172.21.15.96:0/2865203565 conn(0x55a6dbdbced0 msgr2=0x55a6dbdbe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f62437fe640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 0x7f623c0b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.469+0000 7f62437fe640 1 -- 172.21.15.96:0/2610606739 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f62440e5be0 con 0x7f6254103cd0 2024-09-10T22:51:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.471+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 0x7f623c0b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.471+0000 7f625afdd740 1 -- 172.21.15.96:0/2865203565 shutdown_connections 2024-09-10T22:51:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.472+0000 7f625afdd740 1 -- 172.21.15.96:0/2865203565 wait complete. 2024-09-10T22:51:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.472+0000 7f625afdd740 1 -- start start 2024-09-10T22:51:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.472+0000 7f625afdd740 1 Processor -- start 2024-09-10T22:51:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdc9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbb86700 con 0x55a6dbdd4160 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbcc0110 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6dbdc5710 con 0x55a6dbdbf9c0 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625afdd740 10 client.0 _ll_register_callbacks cb 0x55a6dbcd9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f6251d9b640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 0x7f623c0b7860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6248005c40 tx=0x7f6248005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40008/0 (socket says 172.21.15.96:40008) 2024-09-10T22:51:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.473+0000 7f625259c640 1 -- 172.21.15.96:0/1410593520 learned_addr learned my addr 172.21.15.96:0/1410593520 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f6251d9b640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdc9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 -- 172.21.15.96:0/1410593520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 msgr2=0x55a6dbdc9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdc9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 -- 172.21.15.96:0/1410593520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdd4160 msgr2=0x55a6dbdd3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 -- 172.21.15.96:0/1410593520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dbcedc40 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f625259c640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f624400c8f0 tx=0x7f62440abeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615224d52a9950a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.474+0000 7f6239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.475+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62440ac3a0 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.475+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f62440b54a0 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.475+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62480048c0 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.475+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62440b0e80 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.476+0000 7f625afdd740 10 client.25736 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.476+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6dbdfc110 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.476+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62480048c0 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.477+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f62440ab660 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 10 client.25736 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:18.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:18.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 20 client.25736 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:18.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdfd8c0 0x55a6dbdffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 4 client.25736 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:18.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.478+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6dbd37da0 con 0x55a6dbdfd8c0 2024-09-10T22:51:18.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.479+0000 7f6251d9b640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdfd8c0 0x55a6dbdffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.480+0000 7f6251d9b640 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdfd8c0 0x55a6dbdffd10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798b07a9471db86a server_cookie=14cba0fb363d922f in_seq=0 out_seq=0 2024-09-10T22:51:18.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.480+0000 7f6239ffb640 10 client.25736 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.481+0000 7f6239ffb640 1 -- 172.21.15.96:0/1410593520 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6dbd37da0 con 0x55a6dbdfd8c0 2024-09-10T22:51:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.481+0000 7f6239ffb640 10 client.25736 handle_command_reply: tid=0 2024-09-10T22:51:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.481+0000 7f625afdd740 1 client.25736 shutdown 2024-09-10T22:51:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.484+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 msgr2=0x55a6dbdc9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.484+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f624400c8f0 tx=0x7f62440abeb0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 shutdown_connections 2024-09-10T22:51:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdfd8c0 0x55a6dbdffd10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdc9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 --2- 172.21.15.96:0/1410593520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc9010 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 >> 172.21.15.96:0/1410593520 conn(0x55a6dbdbced0 msgr2=0x55a6dbdc2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.486+0000 7f6239ffb640 0 client.25736 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:18.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.490+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 shutdown_connections 2024-09-10T22:51:18.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.492+0000 7f625afdd740 1 -- 172.21.15.96:0/1410593520 wait complete. 2024-09-10T22:51:18.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.494+0000 7f625afdd740 20 client.25736 trim_cache size 0 max 16384 2024-09-10T22:51:18.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.535+0000 7f625afdd740 1 Processor -- start 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.535+0000 7f625afdd740 1 -- start start 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6254072d10 con 0x55a6dbdbf9c0 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6254074850 con 0x55a6dbdd4160 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625afdd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62540733e0 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59168/0 (socket says 172.21.15.96:59168) 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f625259c640 1 -- 172.21.15.96:0/3028425955 learned_addr learned my addr 172.21.15.96:0/3028425955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:18.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f6251d9b640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.536+0000 7f6252d9d640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 -- 172.21.15.96:0/3028425955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 msgr2=0x55a6dbdc69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:18.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 -- 172.21.15.96:0/3028425955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf9c0 msgr2=0x55a6dbdff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 -- 172.21.15.96:0/3028425955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dbdb8790 con 0x55a6dbdd4160 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f625259c640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f6252d9d640 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd5110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624c009fd0 tx=0x7f624c012c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80dc293868bde1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f623bfff640 1 -- 172.21.15.96:0/3028425955 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c012190 con 0x55a6dbdd4160 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f623bfff640 1 -- 172.21.15.96:0/3028425955 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f624c003e20 con 0x55a6dbdd4160 2024-09-10T22:51:18.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.537+0000 7f623bfff640 1 -- 172.21.15.96:0/3028425955 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004280 con 0x55a6dbdd4160 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.538+0000 7f625afdd740 1 -- 172.21.15.96:0/3028425955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 msgr2=0x55a6dbdd5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.538+0000 7f625afdd740 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd5110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624c009fd0 tx=0x7f624c012c60 comp rx=0 tx=0).stop 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 -- 172.21.15.96:0/3028425955 shutdown_connections 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdd5110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 --2- 172.21.15.96:0/3028425955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbdc69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 -- 172.21.15.96:0/3028425955 >> 172.21.15.96:0/3028425955 conn(0x55a6dbdbced0 msgr2=0x55a6dbdd60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.539+0000 7f625afdd740 1 -- 172.21.15.96:0/3028425955 shutdown_connections 2024-09-10T22:51:18.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.540+0000 7f625afdd740 1 -- 172.21.15.96:0/3028425955 wait complete. 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.540+0000 7f625afdd740 1 -- start start 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.540+0000 7f625afdd740 1 Processor -- start 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbe11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6254074180 con 0x55a6dbdbf0a0 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6254073ab0 con 0x55a6dbdd4160 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6254070dc0 con 0x55a6dbdbf9c0 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f625afdd740 10 client.0 _ll_register_callbacks cb 0x7f624400fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40016/0 (socket says 172.21.15.96:40016) 2024-09-10T22:51:18.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 -- 172.21.15.96:0/1916029350 learned_addr learned my addr 172.21.15.96:0/1916029350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 -- 172.21.15.96:0/1916029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 msgr2=0x55a6dbdca460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdca460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 -- 172.21.15.96:0/1916029350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 msgr2=0x55a6dbe11bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbe11bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.541+0000 7f6252d9d640 1 -- 172.21.15.96:0/1916029350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dbdcb090 con 0x55a6dbdd4160 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.542+0000 7f6252d9d640 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f624c000f80 tx=0x7f624c012580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb8f111b0d2a6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.542+0000 7f6239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.542+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c012290 con 0x55a6dbdd4160 2024-09-10T22:51:18.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.542+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f624c018040 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.542+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c003e20 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.543+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f624c02d050 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.543+0000 7f625afdd740 10 client.25742 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.543+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6dbde9ab0 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.544+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f624c003e20 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.544+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f624c0045d0 con 0x55a6dbdd4160 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 10 client.25742 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 20 client.25742 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdea240 0x55a6dbdec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:18.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 4 client.25742 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:51:18.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6dbd33850 con 0x55a6dbdea240 2024-09-10T22:51:18.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.545+0000 7f625259c640 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdea240 0x55a6dbdec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:18.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.550+0000 7f625259c640 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdea240 0x55a6dbdec660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe879976f6e7e55 server_cookie=149f558d7ed67a35 in_seq=0 out_seq=0 2024-09-10T22:51:18.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.550+0000 7f6239ffb640 10 client.25742 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:18.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.550+0000 7f6239ffb640 1 -- 172.21.15.96:0/1916029350 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55a6dbd33850 con 0x55a6dbdea240 2024-09-10T22:51:18.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.550+0000 7f6239ffb640 10 client.25742 handle_command_reply: tid=0 2024-09-10T22:51:18.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.550+0000 7f625afdd740 1 client.25742 shutdown 2024-09-10T22:51:18.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 msgr2=0x55a6dbdca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f624c000f80 tx=0x7f624c012580 comp rx=0 tx=0).stop 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 shutdown_connections 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a6dbdea240 0x55a6dbdec660 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6dbdd4160 0x55a6dbdca9a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6dbdbf9c0 0x55a6dbdca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 --2- 172.21.15.96:0/1916029350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a6dbdbf0a0 0x55a6dbe11bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 >> 172.21.15.96:0/1916029350 conn(0x55a6dbdbced0 msgr2=0x55a6dbdd60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f6239ffb640 0 client.25742 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 shutdown_connections 2024-09-10T22:51:18.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.554+0000 7f625afdd740 1 -- 172.21.15.96:0/1916029350 wait complete. 2024-09-10T22:51:18.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.555+0000 7f625afdd740 20 client.25742 trim_cache size 0 max 16384 2024-09-10T22:51:18.558 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:51:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:51:18.562 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.557+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 msgr2=0x7f623c0b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.557+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 0x7f623c0b7860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6248005c40 tx=0x7f6248005fd0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.557+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 msgr2=0x7f6254199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.557+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f624400c920 tx=0x7f624400cde0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 shutdown_connections 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f625410d130 0x7f625419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f623c0b53a0 0x7f623c0b7860 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62541045f0 0x7f62541997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 --2- 172.21.15.96:0/2610606739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6254103cd0 0x7f6254199270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:18.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 >> 172.21.15.96:0/2610606739 conn(0x7f62540fd770 msgr2=0x7f62540fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:18.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 shutdown_connections 2024-09-10T22:51:18.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:18.558+0000 7f6258c08640 1 -- 172.21.15.96:0/2610606739 wait complete. 2024-09-10T22:51:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:18 smithi096 ceph-mon[30869]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 716 op/s 2024-09-10T22:51:18.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:18 smithi117 ceph-mon[35081]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 716 op/s 2024-09-10T22:51:19.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:18 smithi121 ceph-mon[35119]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 716 op/s 2024-09-10T22:51:19.426 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:51:19.426 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:51:19.809 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 -- 172.21.15.96:0/1951842643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c103370 msgr2=0x7f7a6c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 --2- 172.21.15.96:0/1951842643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c103370 0x7f7a6c109cf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009c10 tx=0x7f7a6402a6e0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 -- 172.21.15.96:0/1951842643 shutdown_connections 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 --2- 172.21.15.96:0/1951842643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c103370 0x7f7a6c109cf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 --2- 172.21.15.96:0/1951842643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 --2- 172.21.15.96:0/1951842643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c101860 0x7f7a6c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.130+0000 7f7a740b6640 1 -- 172.21.15.96:0/1951842643 >> 172.21.15.96:0/1951842643 conn(0x7f7a6c0fd140 msgr2=0x7f7a6c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- 172.21.15.96:0/1951842643 shutdown_connections 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- 172.21.15.96:0/1951842643 wait complete. 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 Processor -- start 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- start start 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c103370 0x7f7a6c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e5f0 con 0x7f7a6c101860 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e470 con 0x7f7a6c1029b0 2024-09-10T22:51:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.131+0000 7f7a740b6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10e770 con 0x7f7a6c103370 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52546/0 (socket says 172.21.15.96:52546) 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 -- 172.21.15.96:0/3873932480 learned_addr learned my addr 172.21.15.96:0/3873932480 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71651640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a72653640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c103370 0x7f7a6c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 -- 172.21.15.96:0/3873932480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c103370 msgr2=0x7f7a6c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c103370 0x7f7a6c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 -- 172.21.15.96:0/3873932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 msgr2=0x7f7a6c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 -- 172.21.15.96:0/3873932480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c19a850 con 0x7f7a6c101860 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71651640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a71e52640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800d950 tx=0x7f7a6800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a62ffd640 1 -- 172.21.15.96:0/3873932480 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68017030 con 0x7f7a6c101860 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c19ab40 con 0x7f7a6c101860 2024-09-10T22:51:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a62ffd640 1 -- 172.21.15.96:0/3873932480 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a68004020 con 0x7f7a6c101860 2024-09-10T22:51:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a62ffd640 1 -- 172.21.15.96:0/3873932480 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68005560 con 0x7f7a6c101860 2024-09-10T22:51:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.132+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a1210 con 0x7f7a6c101860 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.135+0000 7f7a76e85740 1 Processor -- start 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.135+0000 7f7a76e85740 1 -- start start 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d076630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 0x55fa5d077070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cfe56f0 con 0x55fa5d0761b0 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cfd4b20 con 0x55fa5d073860 2024-09-10T22:51:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a76e85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cfd8930 con 0x55fa5d076bc0 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40046/0 (socket says 172.21.15.96:40046) 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 -- 172.21.15.96:0/720224911 learned_addr learned my addr 172.21.15.96:0/720224911 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a71651640 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 0x55fa5d077070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 -- 172.21.15.96:0/720224911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 msgr2=0x55fa5d077070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 0x55fa5d077070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 -- 172.21.15.96:0/720224911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 msgr2=0x55fa5d076630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d076630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a72653640 1 -- 172.21.15.96:0/720224911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5cf9a530 con 0x55fa5d073860 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a71651640 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 0x55fa5d077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.136+0000 7f7a62ffd640 1 -- 172.21.15.96:0/3873932480 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7a68009a30 con 0x7f7a6c101860 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.137+0000 7f7a72653640 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a64004af0 tx=0x7f7a64005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bedd1982514f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.137+0000 7f7a57fff640 1 -- 172.21.15.96:0/720224911 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64041030 con 0x55fa5d073860 2024-09-10T22:51:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.137+0000 7f7a57fff640 1 -- 172.21.15.96:0/720224911 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a640305e0 con 0x55fa5d073860 2024-09-10T22:51:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.137+0000 7f7a62ffd640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 0x7f7a580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.137+0000 7f7a57fff640 1 -- 172.21.15.96:0/720224911 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64036040 con 0x55fa5d073860 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a62ffd640 1 -- 172.21.15.96:0/3873932480 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7a680e5470 con 0x7f7a6c101860 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a71e52640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 0x7f7a580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- 172.21.15.96:0/720224911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 msgr2=0x55fa5d075c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7a64004af0 tx=0x7f7a64005c50 comp rx=0 tx=0).stop 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- 172.21.15.96:0/720224911 shutdown_connections 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d076bc0 0x55fa5d077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d076630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 --2- 172.21.15.96:0/720224911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d075c70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- 172.21.15.96:0/720224911 >> 172.21.15.96:0/720224911 conn(0x55fa5d06e9b0 msgr2=0x55fa5d06f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- 172.21.15.96:0/720224911 shutdown_connections 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- 172.21.15.96:0/720224911 wait complete. 2024-09-10T22:51:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a71e52640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 0x7f7a580b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a680abc60 tx=0x7f7a68005790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.138+0000 7f7a76e85740 1 -- start start 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 Processor -- start 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d073860 0x55fa5d085670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d0868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cf0cab0 con 0x55fa5d0761b0 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa5cf7a730 con 0x55fa5d076bc0 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d0844a0 con 0x55fa5d073860 2024-09-10T22:51:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a76e85740 10 client.0 _ll_register_callbacks cb 0x55fa5cfce6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52550/0 (socket says 172.21.15.96:52550) 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 -- 172.21.15.96:0/800991887 learned_addr learned my addr 172.21.15.96:0/800991887 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a72653640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d073860 0x55fa5d085670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a72653640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d0868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 -- 172.21.15.96:0/800991887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d073860 msgr2=0x55fa5d085670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d073860 0x55fa5d085670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 -- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 msgr2=0x55fa5d0868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d0868e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.139+0000 7f7a71651640 1 -- 172.21.15.96:0/800991887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5cd36790 con 0x55fa5d0761b0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a71651640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c012d20 tx=0x7f7a5c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ec633d4d242b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c01c5f0 con 0x55fa5d0761b0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a640305e0 con 0x55fa5d0761b0 2024-09-10T22:51:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7a5c01c790 con 0x55fa5d0761b0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c01c990 con 0x55fa5d0761b0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a76e85740 10 client.16203 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa5cdaf620 con 0x55fa5d0761b0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a640305e0 con 0x55fa5d0761b0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.140+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7a5c00ebf0 con 0x55fa5d0761b0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 20 client.16203 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d0933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 4 client.16203 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5cfe7640 con 0x55fa5d090f90 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a72653640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d0933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a72653640 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d0933c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a036a0ba441dc57 server_cookie=c0a20527977c0a00 in_seq=0 out_seq=0 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.141+0000 7f7a557fa640 10 client.16203 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a557fa640 1 -- 172.21.15.96:0/800991887 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa5cfe7640 con 0x55fa5d090f90 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a557fa640 10 client.16203 handle_command_reply: tid=0 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 client.16203 shutdown 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 msgr2=0x55fa5d085d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c012d20 tx=0x7f7a5c0107a0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 shutdown_connections 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d0933c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d0868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a557fa640 0 client.16203 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d0761b0 0x55fa5d085d10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 --2- 172.21.15.96:0/800991887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d073860 0x55fa5d085670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 >> 172.21.15.96:0/800991887 conn(0x55fa5d06e9b0 msgr2=0x55fa5d06dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 shutdown_connections 2024-09-10T22:51:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 1 -- 172.21.15.96:0/800991887 wait complete. 2024-09-10T22:51:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.142+0000 7f7a76e85740 20 client.16203 trim_cache size 0 max 16384 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 Processor -- start 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 -- start start 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d06f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d076bc0 0x55fa5d085670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a7b40 con 0x55fa5d076bc0 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a9680 con 0x55fa5d073860 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.157+0000 7f7a76e85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a8210 con 0x55fa5d0761b0 2024-09-10T22:51:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40072/0 (socket says 172.21.15.96:40072) 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 -- 172.21.15.96:0/3218816109 learned_addr learned my addr 172.21.15.96:0/3218816109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a71651640 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d076bc0 0x55fa5d085670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 -- 172.21.15.96:0/3218816109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 msgr2=0x55fa5d06f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d06f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 -- 172.21.15.96:0/3218816109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d076bc0 msgr2=0x55fa5d085670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d076bc0 0x55fa5d085670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 -- 172.21.15.96:0/3218816109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5d077a70 con 0x55fa5d073860 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a72653640 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7a6402aca0 tx=0x7f7a64033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d031f805c0838d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.158+0000 7f7a577fe640 1 -- 172.21.15.96:0/3218816109 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64036040 con 0x55fa5d073860 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a577fe640 1 -- 172.21.15.96:0/3218816109 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64004040 con 0x55fa5d073860 2024-09-10T22:51:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a577fe640 1 -- 172.21.15.96:0/3218816109 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64041040 con 0x55fa5d073860 2024-09-10T22:51:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 -- 172.21.15.96:0/3218816109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 msgr2=0x55fa5d06f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7a6402aca0 tx=0x7f7a64033040 comp rx=0 tx=0).stop 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 -- 172.21.15.96:0/3218816109 shutdown_connections 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d076bc0 0x55fa5d085670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d06f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3218816109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d073860 0x55fa5d06f210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 -- 172.21.15.96:0/3218816109 >> 172.21.15.96:0/3218816109 conn(0x55fa5d06e9b0 msgr2=0x55fa5d083890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 -- 172.21.15.96:0/3218816109 shutdown_connections 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.159+0000 7f7a76e85740 1 -- 172.21.15.96:0/3218816109 wait complete. 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 -- start start 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 Processor -- start 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d088f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d089520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a8fb0 con 0x55fa5d073860 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a88e0 con 0x55fa5d076bc0 2024-09-10T22:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1a5b40 con 0x55fa5d0761b0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a76e85740 10 client.0 _ll_register_callbacks cb 0x7f7a5c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52562/0 (socket says 172.21.15.96:52562) 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 -- 172.21.15.96:0/3851358088 learned_addr learned my addr 172.21.15.96:0/3851358088 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a71651640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d088f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 -- 172.21.15.96:0/3851358088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 msgr2=0x55fa5d088f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d088f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 -- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 msgr2=0x55fa5d089520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d089520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.160+0000 7f7a72653640 1 -- 172.21.15.96:0/3851358088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5d08a240 con 0x55fa5d073860 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a72653640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7a6403f040 tx=0x7f7a640305e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0354b68153a688 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a557fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6405c070 con 0x55fa5d073860 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7a64030a70 con 0x55fa5d073860 2024-09-10T22:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a64004040 con 0x55fa5d073860 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a64068050 con 0x55fa5d073860 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a76e85740 10 client.16209 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa5d0fe290 con 0x55fa5d073860 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.161+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64004040 con 0x55fa5d073860 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7a6403d700 con 0x55fa5d073860 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 20 client.16209 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d100850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 4 client.16209 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa5cfe30f0 con 0x55fa5d090f90 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a71651640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a557fa640 10 client.16209 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.162+0000 7f7a71651640 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d100850 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2242e3b221da8de5 server_cookie=192db85dac8fc0a7 in_seq=0 out_seq=0 2024-09-10T22:51:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.186+0000 7f7a557fa640 1 -- 172.21.15.96:0/3851358088 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa5cfe30f0 con 0x55fa5d090f90 2024-09-10T22:51:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.186+0000 7f7a557fa640 10 client.16209 handle_command_reply: tid=0 2024-09-10T22:51:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.186+0000 7f7a76e85740 1 client.16209 shutdown 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 msgr2=0x55fa5d08d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7a6403f040 tx=0x7f7a640305e0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 shutdown_connections 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fa5d090f90 0x55fa5d100850 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa5d076bc0 0x55fa5d089520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fa5d0761b0 0x55fa5d088f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a557fa640 0 client.16209 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 --2- 172.21.15.96:0/3851358088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa5d073860 0x55fa5d08d210 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 >> 172.21.15.96:0/3851358088 conn(0x55fa5d06e9b0 msgr2=0x55fa5d0837a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 shutdown_connections 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 1 -- 172.21.15.96:0/3851358088 wait complete. 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.187+0000 7f7a76e85740 20 client.16209 trim_cache size 0 max 16384 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:51:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:51:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 msgr2=0x7f7a580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 0x7f7a580b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a680abc60 tx=0x7f7a68005790 comp rx=0 tx=0).stop 2024-09-10T22:51:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 msgr2=0x7f7a6c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800d950 tx=0x7f7a6800dd50 comp rx=0 tx=0).stop 2024-09-10T22:51:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 shutdown_connections 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a6c1029b0 0x7f7a6c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a580b52e0 0x7f7a580b77a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a6c103370 0x7f7a6c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 --2- 172.21.15.96:0/3873932480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a6c101860 0x7f7a6c198d10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 >> 172.21.15.96:0/3873932480 conn(0x7f7a6c0fd140 msgr2=0x7f7a6c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 shutdown_connections 2024-09-10T22:51:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:20.188+0000 7f7a740b6640 1 -- 172.21.15.96:0/3873932480 wait complete. 2024-09-10T22:51:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.399+0000 7fe67b786640 1 -- 172.21.15.96:0/1691632229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe674101cb0 msgr2=0x7fe674102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.399+0000 7fe67b786640 1 --2- 172.21.15.96:0/1691632229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe674101cb0 0x7fe674102090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe670009900 tx=0x7fe67002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.401+0000 7fe67b786640 1 -- 172.21.15.96:0/1691632229 shutdown_connections 2024-09-10T22:51:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.401+0000 7fe67b786640 1 --2- 172.21.15.96:0/1691632229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6741076c0 0x7fe674107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.401+0000 7fe67b786640 1 --2- 172.21.15.96:0/1691632229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe674108040 0x7fe67410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.401+0000 7fe67b786640 1 --2- 172.21.15.96:0/1691632229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe674101cb0 0x7fe674102090 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe670009900 tx=0x7fe67002a410 comp rx=0 tx=0).stop 2024-09-10T22:51:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.401+0000 7fe67b786640 1 -- 172.21.15.96:0/1691632229 >> 172.21.15.96:0/1691632229 conn(0x7fe6740fd700 msgr2=0x7fe6740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.418+0000 7fe67b786640 1 -- 172.21.15.96:0/1691632229 shutdown_connections 2024-09-10T22:51:20.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.418+0000 7fe67b786640 1 -- 172.21.15.96:0/1691632229 wait complete. 2024-09-10T22:51:20.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.420+0000 7fe67b786640 1 Processor -- start 2024-09-10T22:51:20.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.420+0000 7fe67b786640 1 -- start start 2024-09-10T22:51:20.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.420+0000 7fe67b786640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 0x7fe674199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.420+0000 7fe67b786640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe674108040 0x7fe6741999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe679522640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 0x7fe674199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe679522640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 0x7fe674199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59224/0 (socket says 172.21.15.96:59224) 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe679522640 1 -- 172.21.15.96:0/3505443547 learned_addr learned my addr 172.21.15.96:0/3505443547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 0x7fe67419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe674111cc0 con 0x7fe67419ed60 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe674111b40 con 0x7fe674108040 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.421+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe674111e40 con 0x7fe6741076c0 2024-09-10T22:51:20.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.422+0000 7fe679d23640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 0x7fe67419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.423+0000 7fe679d23640 1 -- 172.21.15.96:0/3505443547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 msgr2=0x7fe674199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.423+0000 7fe679d23640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 0x7fe674199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.423+0000 7fe679d23640 1 -- 172.21.15.96:0/3505443547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe674108040 msgr2=0x7fe6741999c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:51:20.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.423+0000 7fe679d23640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe674108040 0x7fe6741999c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.423+0000 7fe679d23640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67419f8f0 con 0x7fe67419ed60 2024-09-10T22:51:20.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.424+0000 7fe679d23640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 0x7fe67419f210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009fd0 tx=0x7fe66c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.424+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00ce20 con 0x7fe67419ed60 2024-09-10T22:51:20.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.425+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67410a820 con 0x7fe67419ed60 2024-09-10T22:51:20.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.425+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67410ad60 con 0x7fe67419ed60 2024-09-10T22:51:20.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.425+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66c01b440 con 0x7fe67419ed60 2024-09-10T22:51:20.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.426+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c005280 con 0x7fe67419ed60 2024-09-10T22:51:20.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.427+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6580052c0 con 0x7fe67419ed60 2024-09-10T22:51:20.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.432+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe66c01b5f0 con 0x7fe67419ed60 2024-09-10T22:51:20.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.433+0000 7fe66a7fc640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 0x7fe6600bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:20.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.434+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe66c0e63a0 con 0x7fe67419ed60 2024-09-10T22:51:20.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.435+0000 7fe679522640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 0x7fe6600bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:20.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.437+0000 7fe679522640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 0x7fe6600bff80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe670004770 tx=0x7fe6700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:20.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.439+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe66c0ad260 con 0x7fe67419ed60 2024-09-10T22:51:20.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:20 smithi096 ceph-mon[30869]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-09-10T22:51:20.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.766+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6580055b0 con 0x7fe67419ed60 2024-09-10T22:51:20.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.766+0000 7fe66a7fc640 1 -- 172.21.15.96:0/3505443547 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fe66c0acb60 con 0x7fe67419ed60 2024-09-10T22:51:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:51:20.768 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:51:20.769 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:51:20.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 msgr2=0x7fe6600bff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 0x7fe6600bff80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe670004770 tx=0x7fe6700058d0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 msgr2=0x7fe67419f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:20.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 0x7fe67419f210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009fd0 tx=0x7fe66c00ef10 comp rx=0 tx=0).stop 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 shutdown_connections 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe674108040 0x7fe6741999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe6600bdac0 0x7fe6600bff80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe67419ed60 0x7fe67419f210 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 --2- 172.21.15.96:0/3505443547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6741076c0 0x7fe674199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:20.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.769+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 >> 172.21.15.96:0/3505443547 conn(0x7fe6740fd700 msgr2=0x7fe67406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:20.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.770+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 shutdown_connections 2024-09-10T22:51:20.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:20.770+0000 7fe67b786640 1 -- 172.21.15.96:0/3505443547 wait complete. 2024-09-10T22:51:20.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:20 smithi117 ceph-mon[35081]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-09-10T22:51:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 225.0 s, 3893.6 tps, lat 26.010 ms stddev 28.793 2024-09-10T22:51:21.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:20 smithi121 ceph-mon[35119]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 937 op/s 2024-09-10T22:51:21.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:51:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:22.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:21 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3505443547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:22.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:21 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3505443547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:22.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:21 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3505443547' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:51:22.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 -- 172.21.15.96:0/105246548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 msgr2=0x7f00d010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 --2- 172.21.15.96:0/105246548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d010f950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009be0 tx=0x7f00c802a710 comp rx=0 tx=0).stop 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 -- 172.21.15.96:0/105246548 shutdown_connections 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 --2- 172.21.15.96:0/105246548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d010f950 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 --2- 172.21.15.96:0/105246548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d01087d0 0x7f00d0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 --2- 172.21.15.96:0/105246548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d0102dc0 0x7f00d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 -- 172.21.15.96:0/105246548 >> 172.21.15.96:0/105246548 conn(0x7f00d00fe830 msgr2=0x7f00d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 -- 172.21.15.96:0/105246548 shutdown_connections 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.487+0000 7f00d84d1640 1 -- 172.21.15.96:0/105246548 wait complete. 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 Processor -- start 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 -- start start 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 0x7f00d00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 0x7f00d00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0112e10 con 0x7f00d0109150 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0112c90 con 0x7f00d01087d0 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d84d1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0112f90 con 0x7f00d0102dc0 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 0x7f00d00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d5a6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 0x7f00d00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 0x7f00d00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59242/0 (socket says 172.21.15.96:59242) 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d626d640 1 -- 172.21.15.96:0/2695954857 learned_addr learned my addr 172.21.15.96:0/2695954857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.489+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 -- 172.21.15.96:0/2695954857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 msgr2=0x7f00d00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 0x7f00d00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 -- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 msgr2=0x7f00d0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 -- 172.21.15.96:0/2695954857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d01a7640 con 0x7f00d01087d0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 0x7f00d00759e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00c000e9f0 tx=0x7f00c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00c77fe640 1 -- 172.21.15.96:0/2695954857 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c000ccc0 con 0x7f00d01087d0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d01a7930 con 0x7f00d01087d0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00c77fe640 1 -- 172.21.15.96:0/2695954857 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c00108e0 con 0x7f00d01087d0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01a7e40 con 0x7f00d01087d0 2024-09-10T22:51:22.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.490+0000 7f00c77fe640 1 -- 172.21.15.96:0/2695954857 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0018880 con 0x7f00d01087d0 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 Processor -- start 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 -- start start 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 0x55e772a898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 0x55e772a9e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772aa0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7729ffea0 con 0x55e772a894d0 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e7729ef2d0 con 0x55e772a9e590 2024-09-10T22:51:22.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.494+0000 7f00da8a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e7729f30e0 con 0x55e772a89df0 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.495+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 0x55e772a9e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 0x55e772a9e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59244/0 (socket says 172.21.15.96:59244) 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d626d640 1 -- 172.21.15.96:0/1164648786 learned_addr learned my addr 172.21.15.96:0/1164648786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 0x55e772a898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 -- 172.21.15.96:0/1164648786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 msgr2=0x55e772a9e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 0x55e772a9e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 -- 172.21.15.96:0/1164648786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 msgr2=0x55e772aa0980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:22.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772aa0980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 -- 172.21.15.96:0/1164648786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e77274d650 con 0x55e772a894d0 2024-09-10T22:51:22.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.496+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 0x55e772a898b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f00c8034040 tx=0x7f00c80303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404612166f83c4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.497+0000 7f00c4ff9640 1 -- 172.21.15.96:0/1164648786 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8043070 con 0x55e772a894d0 2024-09-10T22:51:22.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.497+0000 7f00c77fe640 1 -- 172.21.15.96:0/2695954857 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f00c0010430 con 0x7f00d01087d0 2024-09-10T22:51:22.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.499+0000 7f00c77fe640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 0x7f00bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.499+0000 7f00c77fe640 1 -- 172.21.15.96:0/2695954857 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f00c00e5e30 con 0x7f00d01087d0 2024-09-10T22:51:22.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.499+0000 7f00c4ff9640 1 -- 172.21.15.96:0/1164648786 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c8030710 con 0x55e772a894d0 2024-09-10T22:51:22.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.499+0000 7f00c4ff9640 1 -- 172.21.15.96:0/1164648786 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8036040 con 0x55e772a894d0 2024-09-10T22:51:22.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.500+0000 7f00da8a6740 1 -- 172.21.15.96:0/1164648786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 msgr2=0x55e772a898b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.500+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 0x55e772a898b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f00c8034040 tx=0x7f00c80303a0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.502+0000 7f00da8a6740 1 -- 172.21.15.96:0/1164648786 shutdown_connections 2024-09-10T22:51:22.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.502+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772aa0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.502+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a89df0 0x55e772a9e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.502+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1164648786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a894d0 0x55e772a898b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.502+0000 7f00da8a6740 1 -- 172.21.15.96:0/1164648786 >> 172.21.15.96:0/1164648786 conn(0x55e772a87300 msgr2=0x55e772a884a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.504+0000 7f00d626d640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 0x7f00bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.506+0000 7f00d626d640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 0x7f00bc0b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55e772a8c650 tx=0x7f00cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.507+0000 7f00da8a6740 1 -- 172.21.15.96:0/1164648786 shutdown_connections 2024-09-10T22:51:22.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.516+0000 7f00da8a6740 1 -- 172.21.15.96:0/1164648786 wait complete. 2024-09-10T22:51:22.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.516+0000 7f00da8a6740 1 -- start start 2024-09-10T22:51:22.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.516+0000 7f00da8a6740 1 Processor -- start 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 0x55e772a93900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772a9dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e772850700 con 0x55e772a89df0 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e77298a160 con 0x55e772a9e590 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e772a8fb20 con 0x55e772a894d0 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.521+0000 7f00da8a6740 10 client.0 _ll_register_callbacks cb 0x55e7729a39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:22.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59248/0 (socket says 172.21.15.96:59248) 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d626d640 1 -- 172.21.15.96:0/835664077 learned_addr learned my addr 172.21.15.96:0/835664077 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 0x55e772a93900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772a9dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 -- 172.21.15.96:0/835664077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 msgr2=0x55e772a93380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 -- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 msgr2=0x55e772a9dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772a9dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.522+0000 7f00d5a6c640 1 -- 172.21.15.96:0/835664077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7729b7c90 con 0x55e772a89df0 2024-09-10T22:51:22.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.523+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 0x55e772a93900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f00c00e3a20 tx=0x7f00c00abeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173d4aa115240f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.523+0000 7f00b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:22.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00b1170 con 0x55e772a89df0 2024-09-10T22:51:22.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c8030710 con 0x55e772a89df0 2024-09-10T22:51:22.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f00c00ab9a0 con 0x55e772a89df0 2024-09-10T22:51:22.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00c00b5c20 con 0x55e772a89df0 2024-09-10T22:51:22.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00da8a6740 10 client.16227 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:22.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.524+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e772ab5530 con 0x55e772a89df0 2024-09-10T22:51:22.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.526+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c8030710 con 0x55e772a89df0 2024-09-10T22:51:22.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f00c00b2300 con 0x55e772a89df0 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 10 client.16227 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 10 client.16227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 20 client.16227 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772ab6930 0x55e772ab8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 4 client.16227 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.527+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e772a01df0 con 0x55e772ab6930 2024-09-10T22:51:22.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.528+0000 7f00d626d640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772ab6930 0x55e772ab8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.531+0000 7f00d626d640 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772ab6930 0x55e772ab8d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab111ccab4b0aeb2 server_cookie=21b787bf2984a018 in_seq=0 out_seq=0 2024-09-10T22:51:22.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.531+0000 7f00b9ffb640 10 client.16227 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:51:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.532+0000 7f00b9ffb640 1 -- 172.21.15.96:0/835664077 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e772a01df0 con 0x55e772ab6930 2024-09-10T22:51:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.532+0000 7f00b9ffb640 10 client.16227 handle_command_reply: tid=0 2024-09-10T22:51:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.532+0000 7f00da8a6740 1 client.16227 shutdown 2024-09-10T22:51:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.535+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 msgr2=0x55e772a93900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.535+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 0x55e772a93900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f00c00e3a20 tx=0x7f00c00abeb0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 shutdown_connections 2024-09-10T22:51:22.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772ab6930 0x55e772ab8d20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a9e590 0x55e772a9dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a89df0 0x55e772a93900 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 --2- 172.21.15.96:0/835664077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 >> 172.21.15.96:0/835664077 conn(0x55e772a87300 msgr2=0x55e772a8cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00b9ffb640 0 client.16227 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.540+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 shutdown_connections 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.541+0000 7f00da8a6740 1 -- 172.21.15.96:0/835664077 wait complete. 2024-09-10T22:51:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.541+0000 7f00da8a6740 20 client.16227 trim_cache size 0 max 16384 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.567+0000 7f00da8a6740 1 Processor -- start 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 -- start start 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a9f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01abf00 con 0x55e772a9e590 2024-09-10T22:51:22.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ada40 con 0x55e772a89df0 2024-09-10T22:51:22.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00da8a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ac5d0 con 0x55e772a894d0 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59252/0 (socket says 172.21.15.96:59252) 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d626d640 1 -- 172.21.15.96:0/1223905801 learned_addr learned my addr 172.21.15.96:0/1223905801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a9f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1223905801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 msgr2=0x55e772a93380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1223905801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 msgr2=0x55e772a9f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a9f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.568+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1223905801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e772a88eb0 con 0x55e772a89df0 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.569+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a938c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00c00a8f00 tx=0x7f00c0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4e909a6cb59e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.569+0000 7f00bbfff640 1 -- 172.21.15.96:0/1223905801 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00b1170 con 0x55e772a89df0 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.569+0000 7f00bbfff640 1 -- 172.21.15.96:0/1223905801 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c00b2300 con 0x55e772a89df0 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.569+0000 7f00bbfff640 1 -- 172.21.15.96:0/1223905801 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00a8090 con 0x55e772a89df0 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- 172.21.15.96:0/1223905801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 msgr2=0x55e772a938c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a938c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00c00a8f00 tx=0x7f00c0009700 comp rx=0 tx=0).stop 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- 172.21.15.96:0/1223905801 shutdown_connections 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a938c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a9f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1223905801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772a93380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- 172.21.15.96:0/1223905801 >> 172.21.15.96:0/1223905801 conn(0x55e772a87300 msgr2=0x55e772ab8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- 172.21.15.96:0/1223905801 shutdown_connections 2024-09-10T22:51:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- 172.21.15.96:0/1223905801 wait complete. 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 -- start start 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.570+0000 7f00da8a6740 1 Processor -- start 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772acaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772acaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00d626d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772acaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59260/0 (socket says 172.21.15.96:59260) 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00d5a6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00d5a6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40150/0 (socket says 172.21.15.96:40150) 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00d626d640 1 -- 172.21.15.96:0/1128554312 learned_addr learned my addr 172.21.15.96:0/1128554312 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:22.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ad370 con 0x55e772a9e590 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01acca0 con 0x55e772a89df0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01a9fb0 con 0x55e772a894d0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.571+0000 7f00da8a6740 10 client.0 _ll_register_callbacks cb 0x7f00c0005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d6a6e640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1128554312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 msgr2=0x55e772acaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772acaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1128554312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 msgr2=0x55e772a902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e772a90a40 con 0x55e772a89df0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00d5a6c640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ac3f0 tx=0x7f00c0005b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff96dc8e947cc6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c00b24b0 con 0x55e772a89df0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.572+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f00c00a8290 con 0x55e772a89df0 2024-09-10T22:51:22.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.573+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c00b2300 con 0x55e772a89df0 2024-09-10T22:51:22.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.573+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f00c00b1170 con 0x55e772a89df0 2024-09-10T22:51:22.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.573+0000 7f00da8a6740 10 client.25772 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.573+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e772b225d0 con 0x55e772a89df0 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c00b2300 con 0x55e772a89df0 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f00c00d3350 con 0x55e772a89df0 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 10 client.25772 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 10 client.25772 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 20 client.25772 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772b22a60 0x55e772b24e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 4 client.25772 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7729fd8a0 con 0x55e772b22a60 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.574+0000 7f00d626d640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772b22a60 0x55e772b24e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.575+0000 7f00b9ffb640 10 client.25772 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.575+0000 7f00d626d640 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772b22a60 0x55e772b24e80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d1d76823e29cb36 server_cookie=47811979bb5bd990 in_seq=0 out_seq=0 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00b9ffb640 1 -- 172.21.15.96:0/1128554312 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7729fd8a0 con 0x55e772b22a60 2024-09-10T22:51:22.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00b9ffb640 10 client.25772 handle_command_reply: tid=0 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 client.25772 shutdown 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 msgr2=0x55e772a8fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ac3f0 tx=0x7f00c0005b60 comp rx=0 tx=0).stop 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 shutdown_connections 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55e772b22a60 0x55e772b24e80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e772a89df0 0x55e772a8fd70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e772a9e590 0x55e772a902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 --2- 172.21.15.96:0/1128554312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e772a894d0 0x55e772acaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00b9ffb640 0 client.25772 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 >> 172.21.15.96:0/1128554312 conn(0x55e772a87300 msgr2=0x55e772ab8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 shutdown_connections 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 1 -- 172.21.15.96:0/1128554312 wait complete. 2024-09-10T22:51:22.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.576+0000 7f00da8a6740 20 client.25772 trim_cache size 0 max 16384 2024-09-10T22:51:22.581 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 msgr2=0x7f00bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 0x7f00bc0b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55e772a8c650 tx=0x7f00cc009260 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 msgr2=0x7f00d00759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 0x7f00d00759e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00c000e9f0 tx=0x7f00c000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 shutdown_connections 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d01087d0 0x7f00d00759e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f00bc0b52e0 0x7f00bc0b77a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00d0109150 0x7f00d0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 --2- 172.21.15.96:0/2695954857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00d0102dc0 0x7f00d00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 >> 172.21.15.96:0/2695954857 conn(0x7f00d00fe830 msgr2=0x7f00d0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 shutdown_connections 2024-09-10T22:51:22.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:22.577+0000 7f00d84d1640 1 -- 172.21.15.96:0/2695954857 wait complete. 2024-09-10T22:51:22.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:22 smithi096 ceph-mon[30869]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 869 op/s 2024-09-10T22:51:23.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:22 smithi121 ceph-mon[35119]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 869 op/s 2024-09-10T22:51:23.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:22 smithi117 ceph-mon[35081]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 869 op/s 2024-09-10T22:51:23.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:51:23.790 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:24.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.329+0000 7f0d984ca640 1 -- 172.21.15.96:0/700075910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d90103cc0 msgr2=0x7f0d901040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.329+0000 7f0d984ca640 1 --2- 172.21.15.96:0/700075910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d90103cc0 0x7f0d901040a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009860 tx=0x7f0d8002a340 comp rx=0 tx=0).stop 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 -- 172.21.15.96:0/700075910 shutdown_connections 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 --2- 172.21.15.96:0/700075910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d901045e0 0x7f0d9010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 --2- 172.21.15.96:0/700075910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9010d120 0x7f0d9010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 --2- 172.21.15.96:0/700075910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d90103cc0 0x7f0d901040a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 -- 172.21.15.96:0/700075910 >> 172.21.15.96:0/700075910 conn(0x7f0d900fd700 msgr2=0x7f0d900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 -- 172.21.15.96:0/700075910 shutdown_connections 2024-09-10T22:51:24.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 -- 172.21.15.96:0/700075910 wait complete. 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.330+0000 7f0d984ca640 1 Processor -- start 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 -- start start 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 0x7f0d9019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9010d120 0x7f0d901a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112d90 con 0x7f0d901045e0 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112c10 con 0x7f0d90103cc0 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d984ca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112f10 con 0x7f0d9010d120 2024-09-10T22:51:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52620/0 (socket says 172.21.15.96:52620) 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 -- 172.21.15.96:0/2153709529 learned_addr learned my addr 172.21.15.96:0/2153709529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d96266640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 0x7f0d9019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d96a67640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9010d120 0x7f0d901a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 -- 172.21.15.96:0/2153709529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9010d120 msgr2=0x7f0d901a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9010d120 0x7f0d901a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 -- 172.21.15.96:0/2153709529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 msgr2=0x7f0d9019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 0x7f0d9019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d95a65640 1 -- 172.21.15.96:0/2153709529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d901a1550 con 0x7f0d901045e0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.331+0000 7f0d96266640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 0x7f0d9019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d95a65640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d750 tx=0x7f0d8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d877fe640 1 -- 172.21.15.96:0/2153709529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0d901045e0 2024-09-10T22:51:24.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d901a1840 con 0x7f0d901045e0 2024-09-10T22:51:24.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d901a7f00 con 0x7f0d901045e0 2024-09-10T22:51:24.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d877fe640 1 -- 172.21.15.96:0/2153709529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c004020 con 0x7f0d901045e0 2024-09-10T22:51:24.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.332+0000 7f0d877fe640 1 -- 172.21.15.96:0/2153709529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c003c00 con 0x7f0d901045e0 2024-09-10T22:51:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.335+0000 7f0d9a89f740 1 Processor -- start 2024-09-10T22:51:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 -- start start 2024-09-10T22:51:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 0x55d32431c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 0x55d324328460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d32432ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d324292ea0 con 0x55d32431c0f0 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d3242822d0 con 0x55d32431ca10 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d3242860e0 con 0x55d3243289a0 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 0x55d32431c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 0x55d32431c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52636/0 (socket says 172.21.15.96:52636) 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 0x55d324328460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96a67640 1 -- 172.21.15.96:0/438943228 learned_addr learned my addr 172.21.15.96:0/438943228 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 -- 172.21.15.96:0/438943228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 msgr2=0x55d32432ad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d32432ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 -- 172.21.15.96:0/438943228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 msgr2=0x55d32431c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 0x55d32431c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.336+0000 7f0d96266640 1 -- 172.21.15.96:0/438943228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d323fe0650 con 0x55d32431ca10 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.337+0000 7f0d96266640 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 0x55d324328460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000c00 tx=0x7f0d80009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18b93b61ef18781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.337+0000 7f0d877fe640 1 -- 172.21.15.96:0/2153709529 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0d8c0041d0 con 0x7f0d901045e0 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.337+0000 7f0d84ff9640 1 -- 172.21.15.96:0/438943228 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80041070 con 0x55d32431ca10 2024-09-10T22:51:24.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.337+0000 7f0d84ff9640 1 -- 172.21.15.96:0/438943228 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d800327f0 con 0x55d32431ca10 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d84ff9640 1 -- 172.21.15.96:0/438943228 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80037040 con 0x55d32431ca10 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d877fe640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 0x7f0d7c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 -- 172.21.15.96:0/438943228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 msgr2=0x55d324328460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d96a67640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 0x7f0d7c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 0x55d324328460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000c00 tx=0x7f0d80009510 comp rx=0 tx=0).stop 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d877fe640 1 -- 172.21.15.96:0/2153709529 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0d8c0e5170 con 0x7f0d901045e0 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 -- 172.21.15.96:0/438943228 shutdown_connections 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431ca10 0x55d324328460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d32432ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/438943228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431c0f0 0x55d32431c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.338+0000 7f0d9a89f740 1 -- 172.21.15.96:0/438943228 >> 172.21.15.96:0/438943228 conn(0x55d324319f20 msgr2=0x55d32431b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.339+0000 7f0d9a89f740 1 -- 172.21.15.96:0/438943228 shutdown_connections 2024-09-10T22:51:24.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.339+0000 7f0d96a67640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 0x7f0d7c0b7820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e660 tx=0x7f0d88010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.339+0000 7f0d9a89f740 1 -- 172.21.15.96:0/438943228 wait complete. 2024-09-10T22:51:24.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.339+0000 7f0d9a89f740 1 -- start start 2024-09-10T22:51:24.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.339+0000 7f0d9a89f740 1 Processor -- start 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.340+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.340+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324325670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.340+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52638/0 (socket says 172.21.15.96:52638) 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d96266640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d96266640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40186/0 (socket says 172.21.15.96:40186) 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3240e3700 con 0x55d3243289a0 2024-09-10T22:51:24.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d32421d160 con 0x55d32431c0f0 2024-09-10T22:51:24.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d324333e20 con 0x55d32431ca10 2024-09-10T22:51:24.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d9a89f740 10 client.0 _ll_register_callbacks cb 0x55d3242369e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:24.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.341+0000 7f0d96a67640 1 -- 172.21.15.96:0/3569578906 learned_addr learned my addr 172.21.15.96:0/3569578906 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:24.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d95a65640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324325670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 -- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 msgr2=0x55d324325670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324325670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 -- 172.21.15.96:0/3569578906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 msgr2=0x55d324328220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 -- 172.21.15.96:0/3569578906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32424ac90 con 0x55d3243289a0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d95a65640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324325670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96266640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d96a67640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800dfe0 tx=0x7f0d88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ae3de81c0c41e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.342+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88018630 con 0x55d3243289a0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.343+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d800327f0 con 0x55d3243289a0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.343+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0d880187d0 con 0x55d3243289a0 2024-09-10T22:51:24.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.343+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d880189d0 con 0x55d3243289a0 2024-09-10T22:51:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.343+0000 7f0d9a89f740 10 client.16245 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.343+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d32433f880 con 0x55d3243289a0 2024-09-10T22:51:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.344+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8802a070 con 0x55d3243289a0 2024-09-10T22:51:24.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.344+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0d88024070 con 0x55d3243289a0 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.344+0000 7f0d9a89f740 10 client.16245 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.344+0000 7f0d9a89f740 10 client.16245 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.345+0000 7f0d9a89f740 20 client.16245 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.345+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d324340d80 0x55d3243431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.345+0000 7f0d9a89f740 4 client.16245 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.345+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d800327f0 con 0x55d324340d80 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.345+0000 7f0d96266640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d324340d80 0x55d3243431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d79ffb640 10 client.16245 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:51:24.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d96266640 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d324340d80 0x55d3243431e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4292fc47f596ce0d server_cookie=e6468820a80fb93c in_seq=0 out_seq=0 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d79ffb640 1 -- 172.21.15.96:0/3569578906 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0d800327f0 con 0x55d324340d80 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d79ffb640 10 client.16245 handle_command_reply: tid=0 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d9a89f740 1 client.16245 shutdown 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 msgr2=0x55d324325c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.346+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800dfe0 tx=0x7f0d88004ab0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 shutdown_connections 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d324340d80 0x55d3243431e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d324325c10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324325670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/3569578906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324328220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 >> 172.21.15.96:0/3569578906 conn(0x55d324319f20 msgr2=0x55d32431fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d79ffb640 0 client.16245 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:51:24.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 shutdown_connections 2024-09-10T22:51:24.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 1 -- 172.21.15.96:0/3569578906 wait complete. 2024-09-10T22:51:24.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.347+0000 7f0d9a89f740 20 client.16245 trim_cache size 0 max 16384 2024-09-10T22:51:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.369+0000 7f0d9a89f740 1 Processor -- start 2024-09-10T22:51:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 -- start start 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d32433d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d3243228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400e320 con 0x55d3243289a0 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400fe60 con 0x55d32431c0f0 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400e9f0 con 0x55d32431ca10 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.370+0000 7f0d95a65640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d96266640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d32433d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59284/0 (socket says 172.21.15.96:59284) 2024-09-10T22:51:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 -- 172.21.15.96:0/1841496399 learned_addr learned my addr 172.21.15.96:0/1841496399 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d96266640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d32433d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40188/0 (socket says 172.21.15.96:40188) 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 -- 172.21.15.96:0/1841496399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 msgr2=0x55d32433d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d32433d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d96a67640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d3243228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 -- 172.21.15.96:0/1841496399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 msgr2=0x55d3243228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d3243228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 -- 172.21.15.96:0/1841496399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32431ac00 con 0x55d32431ca10 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d96a67640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d3243228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d95a65640 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c0ab150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ab9d3ec3141af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d7bfff640 1 -- 172.21.15.96:0/1841496399 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c094020 con 0x55d32431ca10 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.371+0000 7f0d7bfff640 1 -- 172.21.15.96:0/1841496399 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c0ab9e0 con 0x55d32431ca10 2024-09-10T22:51:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.372+0000 7f0d7bfff640 1 -- 172.21.15.96:0/1841496399 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c020040 con 0x55d32431ca10 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.372+0000 7f0d9a89f740 1 -- 172.21.15.96:0/1841496399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 msgr2=0x55d324319760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.372+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c000c00 tx=0x7f0d8c0ab150 comp rx=0 tx=0).stop 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 -- 172.21.15.96:0/1841496399 shutdown_connections 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3243289a0 0x55d3243228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32431ca10 0x55d324319760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/1841496399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d32433d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 -- 172.21.15.96:0/1841496399 >> 172.21.15.96:0/1841496399 conn(0x55d324319f20 msgr2=0x55d324325640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 -- 172.21.15.96:0/1841496399 shutdown_connections 2024-09-10T22:51:24.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.373+0000 7f0d9a89f740 1 -- 172.21.15.96:0/1841496399 wait complete. 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 -- start start 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 Processor -- start 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324339890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431ca10 0x55d324322470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400f790 con 0x55d32431ca10 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400f0c0 con 0x55d32431c0f0 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7400c300 con 0x55d3243289a0 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d9a89f740 10 client.0 _ll_register_callbacks cb 0x7f0d88009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d95a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431ca10 0x55d324322470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d96a67640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59296/0 (socket says 172.21.15.96:59296) 2024-09-10T22:51:24.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.374+0000 7f0d96a67640 1 -- 172.21.15.96:0/759686374 learned_addr learned my addr 172.21.15.96:0/759686374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96266640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324339890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 -- 172.21.15.96:0/759686374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 msgr2=0x55d324339890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324339890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 -- 172.21.15.96:0/759686374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431ca10 msgr2=0x55d324322470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431ca10 0x55d324322470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 -- 172.21.15.96:0/759686374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d324323120 con 0x55d3243289a0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96266640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324339890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d96a67640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800da80 tx=0x7f0d880188e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caab2a082bff9eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88018b70 con 0x55d3243289a0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0d88018d10 con 0x55d3243289a0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.375+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8c0ab9e0 con 0x55d3243289a0 2024-09-10T22:51:24.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.376+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d8802e730 con 0x55d3243289a0 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.376+0000 7f0d9a89f740 10 client.25849 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.376+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3243adbc0 con 0x55d3243289a0 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8802a070 con 0x55d3243289a0 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0d88042050 con 0x55d3243289a0 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 10 client.25849 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 10 client.25849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 20 client.25849 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d3243ae2c0 0x55d3243b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 4 client.25849 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:51:24.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.377+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0d8c0ab9e0 con 0x55d3243ae2c0 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d96266640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d3243ae2c0 0x55d3243b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d96266640 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d3243ae2c0 0x55d3243b06e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b24c48d89368d14 server_cookie=566c461ce32e7aa1 in_seq=0 out_seq=0 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d79ffb640 10 client.25849 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d79ffb640 1 -- 172.21.15.96:0/759686374 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0d8c0ab9e0 con 0x55d3243ae2c0 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d79ffb640 10 client.25849 handle_command_reply: tid=0 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.378+0000 7f0d9a89f740 1 client.25849 shutdown 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 msgr2=0x55d3243229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800da80 tx=0x7f0d880188e0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 shutdown_connections 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55d3243ae2c0 0x55d3243b06e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3243289a0 0x55d3243229b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d79ffb640 0 client.25849 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:51:24.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d32431ca10 0x55d324322470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 --2- 172.21.15.96:0/759686374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d32431c0f0 0x55d324339890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 >> 172.21.15.96:0/759686374 conn(0x55d324319f20 msgr2=0x55d324326ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 shutdown_connections 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 1 -- 172.21.15.96:0/759686374 wait complete. 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.379+0000 7f0d9a89f740 20 client.25849 trim_cache size 0 max 16384 2024-09-10T22:51:24.384 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 msgr2=0x7f0d7c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 0x7f0d7c0b7820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e660 tx=0x7f0d88010040 comp rx=0 tx=0).stop 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 msgr2=0x7f0d9019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d750 tx=0x7f0d8c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 shutdown_connections 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d7c0b5360 0x7f0d7c0b7820 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9010d120 0x7f0d901a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d901045e0 0x7f0d9019b9e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 --2- 172.21.15.96:0/2153709529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d90103cc0 0x7f0d9019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:24.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 >> 172.21.15.96:0/2153709529 conn(0x7f0d900fd700 msgr2=0x7f0d900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:24.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 shutdown_connections 2024-09-10T22:51:24.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:24.380+0000 7f0d984ca640 1 -- 172.21.15.96:0/2153709529 wait complete. 2024-09-10T22:51:24.953 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:24 smithi096 ceph-mon[30869]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 865 op/s 2024-09-10T22:51:25.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:24 smithi121 ceph-mon[35119]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 865 op/s 2024-09-10T22:51:25.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:24 smithi117 ceph-mon[35081]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 865 op/s 2024-09-10T22:51:25.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:51:25.526 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 230.0 s, 3520.0 tps, lat 28.317 ms stddev 30.849 2024-09-10T22:51:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.247+0000 7f441f470640 1 -- 172.21.15.96:0/4276179118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 msgr2=0x7f4418113600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.247+0000 7f441f470640 1 --2- 172.21.15.96:0/4276179118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f4418113600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4404009900 tx=0x7f440402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 -- 172.21.15.96:0/4276179118 shutdown_connections 2024-09-10T22:51:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 --2- 172.21.15.96:0/4276179118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f4418115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 --2- 172.21.15.96:0/4276179118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f4418113600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 --2- 172.21.15.96:0/4276179118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 0x7f4418075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 -- 172.21.15.96:0/4276179118 >> 172.21.15.96:0/4276179118 conn(0x7f44180fe830 msgr2=0x7f4418100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 -- 172.21.15.96:0/4276179118 shutdown_connections 2024-09-10T22:51:26.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.249+0000 7f441f470640 1 -- 172.21.15.96:0/4276179118 wait complete. 2024-09-10T22:51:26.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.250+0000 7f441f470640 1 Processor -- start 2024-09-10T22:51:26.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.250+0000 7f441f470640 1 -- start start 2024-09-10T22:51:26.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 0x7f441819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f44181a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4418119430 con 0x7f4418075540 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44181192b0 con 0x7f4418113b40 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441f470640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44181195b0 con 0x7f4418075e60 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441d20c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 0x7f441819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441ca0b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441ca0b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48986/0 (socket says 172.21.15.96:48986) 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441ca0b640 1 -- 172.21.15.96:0/328133675 learned_addr learned my addr 172.21.15.96:0/328133675 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:26.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.251+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f44181a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 -- 172.21.15.96:0/328133675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 msgr2=0x7f441819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 -- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 msgr2=0x7f441819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 0x7f441819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 -- 172.21.15.96:0/328133675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181a5a10 con 0x7f4418113b40 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441ca0b640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.252+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f44181a5240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f441000e940 tx=0x7f441000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.253+0000 7f44167fc640 1 -- 172.21.15.96:0/328133675 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000ccc0 con 0x7f4418113b40 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.253+0000 7f44167fc640 1 -- 172.21.15.96:0/328133675 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44100035f0 con 0x7f4418113b40 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.253+0000 7f44167fc640 1 -- 172.21.15.96:0/328133675 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000ce60 con 0x7f4418113b40 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.253+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44181a5d00 con 0x7f4418113b40 2024-09-10T22:51:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.253+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181ac3c0 con 0x7f4418113b40 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 Processor -- start 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f44167fc640 1 -- 172.21.15.96:0/328133675 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4410004020 con 0x7f4418113b40 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 -- start start 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6bcd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bd9d80 0x556bb6bdc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bb6b43ea0 con 0x556bb6bd9d80 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bb6b332d0 con 0x556bb6bcddf0 2024-09-10T22:51:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.256+0000 7f4421845740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556bb6b370e0 con 0x556bb6bcd4d0 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.257+0000 7f441d20c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.257+0000 7f441d20c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51318/0 (socket says 172.21.15.96:51318) 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.257+0000 7f441d20c640 1 -- 172.21.15.96:0/749732227 learned_addr learned my addr 172.21.15.96:0/749732227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.257+0000 7f44167fc640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 0x7f44080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.257+0000 7f44167fc640 1 -- 172.21.15.96:0/328133675 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44100e5cb0 con 0x7f4418113b40 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 -- 172.21.15.96:0/749732227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 msgr2=0x556bb6bcd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6bcd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 -- 172.21.15.96:0/749732227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bd9d80 msgr2=0x556bb6bdc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bd9d80 0x556bb6bdc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 -- 172.21.15.96:0/749732227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb6891650 con 0x556bb6bcddf0 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 0x7f44080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.258+0000 7f441d20c640 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f440c00a4b0 tx=0x7f440c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c8140c5a18e987 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.259+0000 7f441da0d640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 0x7f44080b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44100a9760 tx=0x7f4410004580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.259+0000 7f44037fe640 1 -- 172.21.15.96:0/749732227 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c018020 con 0x556bb6bcddf0 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.259+0000 7f44037fe640 1 -- 172.21.15.96:0/749732227 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c014cc0 con 0x556bb6bcddf0 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.259+0000 7f44037fe640 1 -- 172.21.15.96:0/749732227 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c013db0 con 0x556bb6bcddf0 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- 172.21.15.96:0/749732227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 msgr2=0x556bb6bd9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f440c00a4b0 tx=0x7f440c0096d0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- 172.21.15.96:0/749732227 shutdown_connections 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bcddf0 0x556bb6bd9840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bd9d80 0x556bb6bdc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 --2- 172.21.15.96:0/749732227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6bcd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- 172.21.15.96:0/749732227 >> 172.21.15.96:0/749732227 conn(0x556bb6bcb300 msgr2=0x556bb6bcc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- 172.21.15.96:0/749732227 shutdown_connections 2024-09-10T22:51:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- 172.21.15.96:0/749732227 wait complete. 2024-09-10T22:51:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 -- start start 2024-09-10T22:51:26.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.260+0000 7f4421845740 1 Processor -- start 2024-09-10T22:51:26.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6bd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556bb6994700 con 0x556bb6bcd4d0 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bb6ace160 con 0x556bb6bd9d80 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.261+0000 7f4421845740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556bb6bdca00 con 0x556bb6bcddf0 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49010/0 (socket says 172.21.15.96:49010) 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 -- 172.21.15.96:0/304699786 learned_addr learned my addr 172.21.15.96:0/304699786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f4421845740 10 client.0 _ll_register_callbacks cb 0x556bb6ae79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441ca0b640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 -- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 msgr2=0x556bb6bd7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 -- 172.21.15.96:0/304699786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 msgr2=0x556bb6bd9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6bd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 -- 172.21.15.96:0/304699786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb6afbc90 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441ca0b640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f441d20c640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f440c00afd0 tx=0x7f440c013d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2913f075140878f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f4400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.262+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c013ed0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c014cc0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f440c014a20 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f440c009ca0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4421845740 10 client.25858 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bb6be83b0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c01d8e0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f440c01e6d0 con 0x556bb6bcddf0 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4421845740 10 client.25858 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.263+0000 7f4421845740 10 client.25858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:51:26.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.264+0000 7f4421845740 20 client.25858 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:26.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.264+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6be9a80 0x556bb6bebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.264+0000 7f4421845740 4 client.25858 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:26.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.264+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f440c014cc0 con 0x556bb6be9a80 2024-09-10T22:51:26.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.264+0000 7f441ca0b640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6be9a80 0x556bb6bebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.265+0000 7f441ca0b640 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6be9a80 0x556bb6bebf00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c58474d7e02fc48 server_cookie=45fe309cce5b7121 in_seq=0 out_seq=0 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.266+0000 7f4400ff9640 10 client.25858 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.266+0000 7f4400ff9640 1 -- 172.21.15.96:0/304699786 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f440c014cc0 con 0x556bb6be9a80 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.266+0000 7f4400ff9640 10 client.25858 handle_command_reply: tid=0 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.266+0000 7f4421845740 1 client.25858 shutdown 2024-09-10T22:51:26.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 msgr2=0x556bb6bd6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f440c00afd0 tx=0x7f440c013d60 comp rx=0 tx=0).stop 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 shutdown_connections 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6be9a80 0x556bb6bebf00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6b60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 --2- 172.21.15.96:0/304699786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6bd9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 >> 172.21.15.96:0/304699786 conn(0x556bb6bcb300 msgr2=0x556bb6bd0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4400ff9640 0 client.25858 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 shutdown_connections 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 1 -- 172.21.15.96:0/304699786 wait complete. 2024-09-10T22:51:26.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.267+0000 7f4421845740 20 client.25858 trim_cache size 0 max 16384 2024-09-10T22:51:26.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.293+0000 7f4421845740 1 Processor -- start 2024-09-10T22:51:26.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.293+0000 7f4421845740 1 -- start start 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.293+0000 7f4421845740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6be5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.293+0000 7f4421845740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.293+0000 7f4421845740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bcbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.294+0000 7f4421845740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b4950 con 0x556bb6bcddf0 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.294+0000 7f4421845740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b6490 con 0x556bb6bd9d80 2024-09-10T22:51:26.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.294+0000 7f4421845740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b5020 con 0x556bb6bcd4d0 2024-09-10T22:51:26.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.295+0000 7f441d20c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.295+0000 7f441d20c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34006/0 (socket says 172.21.15.96:34006) 2024-09-10T22:51:26.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.295+0000 7f441d20c640 1 -- 172.21.15.96:0/896004166 learned_addr learned my addr 172.21.15.96:0/896004166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:26.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.295+0000 7f441ca0b640 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6be5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.295+0000 7f441ca0b640 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bcbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 -- 172.21.15.96:0/896004166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 msgr2=0x556bb6bcbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bcbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 -- 172.21.15.96:0/896004166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 msgr2=0x556bb6bd6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 -- 172.21.15.96:0/896004166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb6bcdc20 con 0x556bb6bcd4d0 2024-09-10T22:51:26.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.296+0000 7f441ca0b640 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6be5c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4404004770 tx=0x7f4404032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576282a2ab548c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.302+0000 7f4402ffd640 1 -- 172.21.15.96:0/896004166 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440404b040 con 0x556bb6bcd4d0 2024-09-10T22:51:26.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.302+0000 7f4402ffd640 1 -- 172.21.15.96:0/896004166 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404004c50 con 0x556bb6bcd4d0 2024-09-10T22:51:26.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.302+0000 7f4402ffd640 1 -- 172.21.15.96:0/896004166 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404035040 con 0x556bb6bcd4d0 2024-09-10T22:51:26.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.303+0000 7f4421845740 1 -- 172.21.15.96:0/896004166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 msgr2=0x556bb6be5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.304+0000 7f4421845740 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6be5c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4404004770 tx=0x7f4404032f60 comp rx=0 tx=0).stop 2024-09-10T22:51:26.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.307+0000 7f4421845740 1 -- 172.21.15.96:0/896004166 shutdown_connections 2024-09-10T22:51:26.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.307+0000 7f4421845740 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bcbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.307+0000 7f4421845740 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcddf0 0x556bb6bd6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.307+0000 7f4421845740 1 --2- 172.21.15.96:0/896004166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcd4d0 0x556bb6be5c30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.307+0000 7f4421845740 1 -- 172.21.15.96:0/896004166 >> 172.21.15.96:0/896004166 conn(0x556bb6bcb300 msgr2=0x556bb6bd1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.308+0000 7f4421845740 1 -- 172.21.15.96:0/896004166 shutdown_connections 2024-09-10T22:51:26.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.308+0000 7f4421845740 1 -- 172.21.15.96:0/896004166 wait complete. 2024-09-10T22:51:26.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.309+0000 7f4421845740 1 -- start start 2024-09-10T22:51:26.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.309+0000 7f4421845740 1 Processor -- start 2024-09-10T22:51:26.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.310+0000 7f4421845740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6be1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.310+0000 7f4421845740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6be21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.310+0000 7f4421845740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.310+0000 7f441d20c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6be21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.310+0000 7f441d20c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6be21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49036/0 (socket says 172.21.15.96:49036) 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f441d20c640 1 -- 172.21.15.96:0/58650777 learned_addr learned my addr 172.21.15.96:0/58650777 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f441ca0b640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b5dc0 con 0x556bb6bcd4d0 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b56f0 con 0x556bb6bd9d80 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b2930 con 0x556bb6bcddf0 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f4421845740 10 client.0 _ll_register_callbacks cb 0x7f440c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.311+0000 7f441ca0b640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6be1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 -- 172.21.15.96:0/58650777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 msgr2=0x556bb6be21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6be21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 -- 172.21.15.96:0/58650777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 msgr2=0x556bb6be1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6be1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb6bd42d0 con 0x556bb6bd9d80 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f441ca0b640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd3d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4404004770 tx=0x7f4404031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0bfb1370f1c1792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.312+0000 7f4400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:26.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.314+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404035070 con 0x556bb6bd9d80 2024-09-10T22:51:26.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.314+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4404005380 con 0x556bb6bd9d80 2024-09-10T22:51:26.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.314+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4404004c50 con 0x556bb6bd9d80 2024-09-10T22:51:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4404005180 con 0x556bb6bd9d80 2024-09-10T22:51:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 10 client.25805 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bb6bf1400 con 0x556bb6bd9d80 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404004c50 con 0x556bb6bd9d80 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4404032030 con 0x556bb6bd9d80 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 10 client.25805 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 10 client.25805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 20 client.25805 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6c67a20 0x556bb6c69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 4 client.25805 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:51:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.316+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bb6b45df0 con 0x556bb6c67a20 2024-09-10T22:51:26.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.317+0000 7f441d20c640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6c67a20 0x556bb6c69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.318+0000 7f441d20c640 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6c67a20 0x556bb6c69e10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a517ece4f72afd7 server_cookie=435248bd11030949 in_seq=0 out_seq=0 2024-09-10T22:51:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.318+0000 7f4400ff9640 10 client.25805 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:51:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.318+0000 7f4400ff9640 1 -- 172.21.15.96:0/58650777 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bb6b45df0 con 0x556bb6c67a20 2024-09-10T22:51:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.318+0000 7f4400ff9640 10 client.25805 handle_command_reply: tid=0 2024-09-10T22:51:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.318+0000 7f4421845740 1 client.25805 shutdown 2024-09-10T22:51:26.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.320+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 msgr2=0x556bb6bd3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.320+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd3d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4404004770 tx=0x7f4404031400 comp rx=0 tx=0).stop 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 shutdown_connections 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x556bb6c67a20 0x556bb6c69e10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bb6bd9d80 0x556bb6bd3d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556bb6bcddf0 0x556bb6be21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 --2- 172.21.15.96:0/58650777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556bb6bcd4d0 0x556bb6be1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 >> 172.21.15.96:0/58650777 conn(0x556bb6bcb300 msgr2=0x556bb6bcbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4400ff9640 0 client.25805 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:51:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 shutdown_connections 2024-09-10T22:51:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 1 -- 172.21.15.96:0/58650777 wait complete. 2024-09-10T22:51:26.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.321+0000 7f4421845740 20 client.25805 trim_cache size 0 max 16384 2024-09-10T22:51:26.324 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 msgr2=0x7f44080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 0x7f44080b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f44100a9760 tx=0x7f4410004580 comp rx=0 tx=0).stop 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 msgr2=0x7f44181a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f44181a5240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f441000e940 tx=0x7f441000ee00 comp rx=0 tx=0).stop 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 shutdown_connections 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418113b40 0x7f44181a5240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44080b5360 0x7f44080b7820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418075e60 0x7f441819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 --2- 172.21.15.96:0/328133675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418075540 0x7f441819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:26.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 >> 172.21.15.96:0/328133675 conn(0x7f44180fe830 msgr2=0x7f44180ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:26.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 shutdown_connections 2024-09-10T22:51:26.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:26.323+0000 7f441f470640 1 -- 172.21.15.96:0/328133675 wait complete. 2024-09-10T22:51:26.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:26 smithi096 ceph-mon[30869]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-09-10T22:51:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:26 smithi121 ceph-mon[35119]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-09-10T22:51:27.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:26 smithi117 ceph-mon[35081]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 825 op/s 2024-09-10T22:51:27.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:51:27.814 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:28.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d5f7fe640 1 -- 172.21.15.96:0/1083267492 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54035020 con 0x7f9d6010b350 2024-09-10T22:51:28.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 -- 172.21.15.96:0/1083267492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010b350 msgr2=0x7f9d60111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1083267492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010b350 0x7f9d60111b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009900 tx=0x7f9d5402a400 comp rx=0 tx=0).stop 2024-09-10T22:51:28.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 -- 172.21.15.96:0/1083267492 shutdown_connections 2024-09-10T22:51:28.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1083267492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010b350 0x7f9d60111b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1083267492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d6010a9d0 0x7f9d6010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.386+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1083267492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d60104fc0 0x7f9d601053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.387+0000 7f9d671f4640 1 -- 172.21.15.96:0/1083267492 >> 172.21.15.96:0/1083267492 conn(0x7f9d601009f0 msgr2=0x7f9d60102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.391+0000 7f9d671f4640 1 -- 172.21.15.96:0/1083267492 shutdown_connections 2024-09-10T22:51:28.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.392+0000 7f9d671f4640 1 -- 172.21.15.96:0/1083267492 wait complete. 2024-09-10T22:51:28.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.393+0000 7f9d671f4640 1 Processor -- start 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 -- start start 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010a9d0 0x7f9d6010eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d6010b350 0x7f9d6010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601150a0 con 0x7f9d60104fc0 2024-09-10T22:51:28.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60114f20 con 0x7f9d6010a9d0 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d671f4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60115220 con 0x7f9d6010b350 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d65791640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d6010b350 0x7f9d6010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d64f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d64f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34040/0 (socket says 172.21.15.96:34040) 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.396+0000 7f9d64f90640 1 -- 172.21.15.96:0/1716832862 learned_addr learned my addr 172.21.15.96:0/1716832862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.397+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010a9d0 0x7f9d6010eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.398+0000 7f9d64f90640 1 -- 172.21.15.96:0/1716832862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d6010b350 msgr2=0x7f9d6010f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.398+0000 7f9d64f90640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d6010b350 0x7f9d6010f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.398+0000 7f9d64f90640 1 -- 172.21.15.96:0/1716832862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010a9d0 msgr2=0x7f9d6010eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.398+0000 7f9d64f90640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010a9d0 0x7f9d6010eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.398+0000 7f9d64f90640 1 -- 172.21.15.96:0/1716832862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6010f950 con 0x7f9d60104fc0 2024-09-10T22:51:28.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.401+0000 7f9d64f90640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ba50 tx=0x7f9d5800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.402+0000 7f9d5dffb640 1 -- 172.21.15.96:0/1716832862 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58019070 con 0x7f9d60104fc0 2024-09-10T22:51:28.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.402+0000 7f9d5dffb640 1 -- 172.21.15.96:0/1716832862 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58014440 con 0x7f9d60104fc0 2024-09-10T22:51:28.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.402+0000 7f9d5dffb640 1 -- 172.21.15.96:0/1716832862 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580075b0 con 0x7f9d60104fc0 2024-09-10T22:51:28.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.402+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601a16e0 con 0x7f9d60104fc0 2024-09-10T22:51:28.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.402+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601a1ba0 con 0x7f9d60104fc0 2024-09-10T22:51:28.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.407+0000 7f9d5dffb640 1 -- 172.21.15.96:0/1716832862 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9d580145f0 con 0x7f9d60104fc0 2024-09-10T22:51:28.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.407+0000 7f9d695c9740 1 Processor -- start 2024-09-10T22:51:28.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.407+0000 7f9d695c9740 1 -- start start 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e45308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2e44a6ea0 con 0x55b2e45304d0 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2e44962d0 con 0x55b2e4530df0 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d695c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2e449a0e0 con 0x55b2e453cd80 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49064/0 (socket says 172.21.15.96:49064) 2024-09-10T22:51:28.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.408+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2088954243 learned_addr learned my addr 172.21.15.96:0/2088954243 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:28.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5dffb640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 0x7f9d480b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d64f90640 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2088954243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 msgr2=0x55b2e453c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2088954243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 msgr2=0x55b2e45308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e45308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2088954243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e41f4650 con 0x55b2e453cd80 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009a40 tx=0x7f9d50009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c54d54cd9b3fea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.409+0000 7f9d5dffb640 1 -- 172.21.15.96:0/1716832862 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9d580e5160 con 0x7f9d60104fc0 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.410+0000 7f9d65791640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 0x7f9d480b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.410+0000 7f9d46ffd640 1 -- 172.21.15.96:0/2088954243 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50017070 con 0x55b2e453cd80 2024-09-10T22:51:28.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.410+0000 7f9d46ffd640 1 -- 172.21.15.96:0/2088954243 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d50013470 con 0x55b2e453cd80 2024-09-10T22:51:28.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 -- 172.21.15.96:0/2088954243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 msgr2=0x55b2e453f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009a40 tx=0x7f9d50009f00 comp rx=0 tx=0).stop 2024-09-10T22:51:28.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d65791640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 0x7f9d480b78f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006010 tx=0x7f9d54031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 -- 172.21.15.96:0/2088954243 shutdown_connections 2024-09-10T22:51:28.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453cd80 0x55b2e453f170 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009a40 tx=0x7f9d50009f00 comp rx=0 tx=0).stop 2024-09-10T22:51:28.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2088954243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e45308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.411+0000 7f9d695c9740 1 -- 172.21.15.96:0/2088954243 >> 172.21.15.96:0/2088954243 conn(0x55b2e452e300 msgr2=0x55b2e452f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.412+0000 7f9d695c9740 1 -- 172.21.15.96:0/2088954243 shutdown_connections 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.412+0000 7f9d695c9740 1 -- 172.21.15.96:0/2088954243 wait complete. 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.413+0000 7f9d695c9740 1 -- start start 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.413+0000 7f9d695c9740 1 Processor -- start 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.413+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e4539d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4548dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2e42f7700 con 0x55b2e45304d0 2024-09-10T22:51:28.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2e4431160 con 0x55b2e4530df0 2024-09-10T22:51:28.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2e453fa50 con 0x55b2e453a7c0 2024-09-10T22:51:28.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d695c9740 10 client.0 _ll_register_callbacks cb 0x55b2e444a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51392/0 (socket says 172.21.15.96:51392) 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2837869772 learned_addr learned my addr 172.21.15.96:0/2837869772 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.414+0000 7f9d65791640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4548dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 msgr2=0x55b2e4548dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4548dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2837869772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 msgr2=0x55b2e4539d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e4539d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d5ffff640 1 -- 172.21.15.96:0/2837869772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e445ec90 con 0x55b2e4530df0 2024-09-10T22:51:28.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.415+0000 7f9d65791640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4548dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:28.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.416+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009430 tx=0x7f9d500046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d54d192a1cc062 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.416+0000 7f9d3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:28.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.416+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50022070 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.416+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9d50012c80 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.416+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2e44a8df0 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.417+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d50017040 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.417+0000 7f9d695c9740 10 client.25817 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.417+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2e454bac0 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b2e44a8df0 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9d500134d0 con 0x55b2e4530df0 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 10 client.25817 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 10 client.25817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 20 client.25817 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e454ce10 0x55b2e454f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 4 client.25817 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:28.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.418+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e44a48a0 con 0x55b2e454ce10 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.419+0000 7f9d64f90640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e454ce10 0x55b2e454f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.419+0000 7f9d64f90640 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e454ce10 0x55b2e454f200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcc24de8fe0c95c1 server_cookie=cc8522913f2f9330 in_seq=0 out_seq=0 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.419+0000 7f9d3ffff640 10 client.25817 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d3ffff640 1 -- 172.21.15.96:0/2837869772 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2e44a48a0 con 0x55b2e454ce10 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d3ffff640 10 client.25817 handle_command_reply: tid=0 2024-09-10T22:51:28.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 client.25817 shutdown 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 msgr2=0x55b2e453a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009430 tx=0x7f9d500046b0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 shutdown_connections 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e454ce10 0x55b2e454f200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e453a280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4548dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 --2- 172.21.15.96:0/2837869772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e45304d0 0x55b2e4539d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 >> 172.21.15.96:0/2837869772 conn(0x55b2e452e300 msgr2=0x55b2e4533ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.420+0000 7f9d3ffff640 0 client.25817 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.421+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 shutdown_connections 2024-09-10T22:51:28.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.421+0000 7f9d695c9740 1 -- 172.21.15.96:0/2837869772 wait complete. 2024-09-10T22:51:28.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.421+0000 7f9d695c9740 20 client.25817 trim_cache size 0 max 16384 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 Processor -- start 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 -- start start 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e45304d0 0x55b2e452d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4549210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60073ff0 con 0x55b2e4530df0 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601c7570 con 0x55b2e45304d0 2024-09-10T22:51:28.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34070/0 (socket says 172.21.15.96:34070) 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d695c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d600746c0 con 0x55b2e453a7c0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.439+0000 7f9d5ffff640 1 -- 172.21.15.96:0/1902613714 learned_addr learned my addr 172.21.15.96:0/1902613714 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d64f90640 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e45304d0 0x55b2e452d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d65791640 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4549210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 -- 172.21.15.96:0/1902613714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 msgr2=0x55b2e4549210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4549210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 -- 172.21.15.96:0/1902613714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e45304d0 msgr2=0x55b2e452d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e45304d0 0x55b2e452d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 -- 172.21.15.96:0/1902613714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e452ed80 con 0x55b2e4530df0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006c70 tx=0x7f9d50004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a50050e6dbe13e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.440+0000 7f9d467fc640 1 -- 172.21.15.96:0/1902613714 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50022040 con 0x55b2e4530df0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.441+0000 7f9d467fc640 1 -- 172.21.15.96:0/1902613714 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d50035070 con 0x55b2e4530df0 2024-09-10T22:51:28.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.441+0000 7f9d467fc640 1 -- 172.21.15.96:0/1902613714 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d500306e0 con 0x55b2e4530df0 2024-09-10T22:51:28.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- 172.21.15.96:0/1902613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 msgr2=0x55b2e4548cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9d50006c70 tx=0x7f9d50004640 comp rx=0 tx=0).stop 2024-09-10T22:51:28.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- 172.21.15.96:0/1902613714 shutdown_connections 2024-09-10T22:51:28.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e453a7c0 0x55b2e4549210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e4530df0 0x55b2e4548cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 --2- 172.21.15.96:0/1902613714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e45304d0 0x55b2e452d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- 172.21.15.96:0/1902613714 >> 172.21.15.96:0/1902613714 conn(0x55b2e452e300 msgr2=0x55b2e452cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- 172.21.15.96:0/1902613714 shutdown_connections 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- 172.21.15.96:0/1902613714 wait complete. 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 -- start start 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.442+0000 7f9d695c9740 1 Processor -- start 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e4540aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e453a7c0 0x55b2e45410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d64f90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d64f90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49078/0 (socket says 172.21.15.96:49078) 2024-09-10T22:51:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d64f90640 1 -- 172.21.15.96:0/4012461877 learned_addr learned my addr 172.21.15.96:0/4012461877 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601c6ea0 con 0x55b2e453a7c0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60074d90 con 0x55b2e4530df0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60071fd0 con 0x55b2e45304d0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d65791640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e453a7c0 0x55b2e45410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e4540aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.443+0000 7f9d695c9740 10 client.0 _ll_register_callbacks cb 0x7f9d50009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 -- 172.21.15.96:0/4012461877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 msgr2=0x55b2e4540aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e4540aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 -- 172.21.15.96:0/4012461877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e453a7c0 msgr2=0x55b2e45410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e453a7c0 0x55b2e45410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e45b8a00 con 0x55b2e45304d0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d64f90640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a8040 tx=0x7f9d58004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973f7177af699b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:28.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580a7040 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d50035070 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9d58014930 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d580aba70 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d695c9740 10 client.25876 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2e45b9b80 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d50035070 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.444+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9d580b4ac0 con 0x55b2e45304d0 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 10 client.25876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 20 client.25876 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e45ba2b0 0x55b2e45bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 4 client.25876 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2e44a1cd0 con 0x55b2e45ba2b0 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.445+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e45ba2b0 0x55b2e45bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.446+0000 7f9d3ffff640 10 client.25876 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.446+0000 7f9d5ffff640 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e45ba2b0 0x55b2e45bc6d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d39bd9bc6b7bcf89 server_cookie=63da6d99a33ae98a in_seq=0 out_seq=0 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.446+0000 7f9d3ffff640 1 -- 172.21.15.96:0/4012461877 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2e44a1cd0 con 0x55b2e45ba2b0 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.446+0000 7f9d3ffff640 10 client.25876 handle_command_reply: tid=0 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 client.25876 shutdown 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 msgr2=0x55b2e45455d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d580a8040 tx=0x7f9d58004a40 comp rx=0 tx=0).stop 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 shutdown_connections 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2e45ba2b0 0x55b2e45bc6d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2e4530df0 0x55b2e4540aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2e453a7c0 0x55b2e45410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 --2- 172.21.15.96:0/4012461877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2e45304d0 0x55b2e45455d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d3ffff640 0 client.25876 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 >> 172.21.15.96:0/4012461877 conn(0x55b2e452e300 msgr2=0x55b2e452d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 shutdown_connections 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.447+0000 7f9d695c9740 1 -- 172.21.15.96:0/4012461877 wait complete. 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d695c9740 20 client.25876 trim_cache size 0 max 16384 2024-09-10T22:51:28.454 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:51:28.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 msgr2=0x7f9d480b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 0x7f9d480b78f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006010 tx=0x7f9d54031400 comp rx=0 tx=0).stop 2024-09-10T22:51:28.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 msgr2=0x7f9d601118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:28.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ba50 tx=0x7f9d5800be50 comp rx=0 tx=0).stop 2024-09-10T22:51:28.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 shutdown_connections 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d6010a9d0 0x7f9d6010eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9d480b5430 0x7f9d480b78f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d6010b350 0x7f9d6010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 --2- 172.21.15.96:0/1716832862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d60104fc0 0x7f9d601118e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.448+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 >> 172.21.15.96:0/1716832862 conn(0x7f9d601009f0 msgr2=0x7f9d60102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.449+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 shutdown_connections 2024-09-10T22:51:28.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:28.449+0000 7f9d671f4640 1 -- 172.21.15.96:0/1716832862 wait complete. 2024-09-10T22:51:28.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:28 smithi096 ceph-mon[30869]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T22:51:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:28 smithi121 ceph-mon[35119]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T22:51:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:28 smithi117 ceph-mon[35081]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 697 op/s 2024-09-10T22:51:29.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:51:29.582 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 -- 172.21.15.96:0/3682293455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 msgr2=0x7f8080101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 --2- 172.21.15.96:0/3682293455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080101e80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8078009ca0 tx=0x7f807802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 -- 172.21.15.96:0/3682293455 shutdown_connections 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 --2- 172.21.15.96:0/3682293455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80801023c0 0x7f8080106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 --2- 172.21.15.96:0/3682293455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8080106ee0 0x7f8080107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 --2- 172.21.15.96:0/3682293455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080101e80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.125+0000 7f80866f1640 1 -- 172.21.15.96:0/3682293455 >> 172.21.15.96:0/3682293455 conn(0x7f80800fd730 msgr2=0x7f80800ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.126+0000 7f80866f1640 1 -- 172.21.15.96:0/3682293455 shutdown_connections 2024-09-10T22:51:30.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.126+0000 7f80866f1640 1 -- 172.21.15.96:0/3682293455 wait complete. 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 Processor -- start 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 -- start start 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80801023c0 0x7f8080074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8080119420 con 0x7f80801023c0 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80801192a0 con 0x7f8080106ee0 2024-09-10T22:51:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f80866f1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80801195a0 con 0x7f8080101aa0 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f807ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f8084c8e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f8084c8e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51422/0 (socket says 172.21.15.96:51422) 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.127+0000 7f8084c8e640 1 -- 172.21.15.96:0/3086136971 learned_addr learned my addr 172.21.15.96:0/3086136971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 -- 172.21.15.96:0/3086136971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 msgr2=0x7f8080074780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080074780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 -- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80801023c0 msgr2=0x7f8080074cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80801023c0 0x7f8080074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 -- 172.21.15.96:0/3086136971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808006e3a0 con 0x7f8080106ee0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.128+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8070009fd0 tx=0x7f807000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.129+0000 7f807d7fa640 1 -- 172.21.15.96:0/3086136971 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070011830 con 0x7f8080106ee0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.129+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808006e690 con 0x7f8080106ee0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.129+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8080073690 con 0x7f8080106ee0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.129+0000 7f807d7fa640 1 -- 172.21.15.96:0/3086136971 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8070016070 con 0x7f8080106ee0 2024-09-10T22:51:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.129+0000 7f807d7fa640 1 -- 172.21.15.96:0/3086136971 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070010de0 con 0x7f8080106ee0 2024-09-10T22:51:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.133+0000 7f807d7fa640 1 -- 172.21.15.96:0/3086136971 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f807001b4a0 con 0x7f8080106ee0 2024-09-10T22:51:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.133+0000 7f8088ac6740 1 Processor -- start 2024-09-10T22:51:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.133+0000 7f8088ac6740 1 -- start start 2024-09-10T22:51:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc098c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55afc0902ea0 con 0x55afc098c0f0 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55afc08f22d0 con 0x55afc098ca10 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f8088ac6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55afc08f60e0 con 0x55afc09989a0 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc098c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51438/0 (socket says 172.21.15.96:51438) 2024-09-10T22:51:30.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 -- 172.21.15.96:0/3331365922 learned_addr learned my addr 172.21.15.96:0/3331365922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:30.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807f7fe640 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 -- 172.21.15.96:0/3331365922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 msgr2=0x55afc099ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 -- 172.21.15.96:0/3331365922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 msgr2=0x55afc098c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc098c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.134+0000 7f807ffff640 1 -- 172.21.15.96:0/3331365922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc0650650 con 0x55afc098ca10 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.135+0000 7f807ffff640 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f807802add0 tx=0x7f80780303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279998b18b89a3ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.136+0000 7f80667fc640 1 -- 172.21.15.96:0/3331365922 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078030800 con 0x55afc098ca10 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.136+0000 7f80667fc640 1 -- 172.21.15.96:0/3331365922 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8078036030 con 0x55afc098ca10 2024-09-10T22:51:30.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.136+0000 7f80667fc640 1 -- 172.21.15.96:0/3331365922 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078030800 con 0x55afc098ca10 2024-09-10T22:51:30.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.136+0000 7f807d7fa640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 0x7f80680b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- 172.21.15.96:0/3331365922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 msgr2=0x55afc0998460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f807802add0 tx=0x7f80780303a0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- 172.21.15.96:0/3331365922 shutdown_connections 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0998460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3331365922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc098c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- 172.21.15.96:0/3331365922 >> 172.21.15.96:0/3331365922 conn(0x55afc0989f20 msgr2=0x55afc098b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f807f7fe640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 0x7f80680b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f807d7fa640 1 -- 172.21.15.96:0/3086136971 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f80700e5000 con 0x7f8080106ee0 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- 172.21.15.96:0/3331365922 shutdown_connections 2024-09-10T22:51:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- 172.21.15.96:0/3331365922 wait complete. 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f8088ac6740 1 -- start start 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.137+0000 7f807f7fe640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 0x7f80680b78a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8074009460 tx=0x7f80740096b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 Processor -- start 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 0x55afc0998210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 0x55afc0995660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55afc0753700 con 0x55afc098ca10 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55afc088d160 con 0x55afc098c0f0 2024-09-10T22:51:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55afc099b5c0 con 0x55afc09989a0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8088ac6740 10 client.0 _ll_register_callbacks cb 0x55afc08a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8084c8e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f807ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 0x55afc0998210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f807f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 0x55afc0995660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8084c8e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49126/0 (socket says 172.21.15.96:49126) 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.138+0000 7f8084c8e640 1 -- 172.21.15.96:0/3383463494 learned_addr learned my addr 172.21.15.96:0/3383463494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 -- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 msgr2=0x55afc0998210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 0x55afc0998210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 -- 172.21.15.96:0/3383463494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 msgr2=0x55afc0995660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 0x55afc0995660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 -- 172.21.15.96:0/3383463494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc08bac90 con 0x55afc09989a0 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f807ffff640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 0x55afc0998210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f807f7fe640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 0x55afc0995660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:30.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8084c8e640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f807000bde0 tx=0x7f80700105b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a8cad9a298473d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f805f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700add40 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8078036030 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f80700aba70 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80700acf10 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8088ac6740 10 client.25888 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.139+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afc09a6f30 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8078036030 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f80700ab4e0 con 0x55afc09989a0 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 10 client.25888 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 20 client.25888 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:30.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc09a8340 0x55afc09aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 4 client.25888 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:30.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc0904df0 con 0x55afc09a8340 2024-09-10T22:51:30.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f807ffff640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc09a8340 0x55afc09aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f807ffff640 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc09a8340 0x55afc09aa7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee204b895c276ff server_cookie=9e34b1ba32303d3c in_seq=0 out_seq=0 2024-09-10T22:51:30.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.140+0000 7f805f7fe640 10 client.25888 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.141+0000 7f805f7fe640 1 -- 172.21.15.96:0/3383463494 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55afc0904df0 con 0x55afc09a8340 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.141+0000 7f805f7fe640 10 client.25888 handle_command_reply: tid=0 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.141+0000 7f8088ac6740 1 client.25888 shutdown 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 msgr2=0x55afc0995be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f807000bde0 tx=0x7f80700105b0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 shutdown_connections 2024-09-10T22:51:30.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc09a8340 0x55afc09aa7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0995be0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098ca10 0x55afc0995660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 --2- 172.21.15.96:0/3383463494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098c0f0 0x55afc0998210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 >> 172.21.15.96:0/3383463494 conn(0x55afc0989f20 msgr2=0x55afc098fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f805f7fe640 0 client.25888 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 shutdown_connections 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 1 -- 172.21.15.96:0/3383463494 wait complete. 2024-09-10T22:51:30.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.142+0000 7f8088ac6740 20 client.25888 trim_cache size 0 max 16384 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 Processor -- start 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 -- start start 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc09a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0992870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0992db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00e320 con 0x55afc098c0f0 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00fe60 con 0x55afc098ca10 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8088ac6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00e9f0 con 0x55afc09989a0 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f807ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc09a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f807f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0992870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f807ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc09a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34110/0 (socket says 172.21.15.96:34110) 2024-09-10T22:51:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f807ffff640 1 -- 172.21.15.96:0/4263551641 learned_addr learned my addr 172.21.15.96:0/4263551641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.158+0000 7f8084c8e640 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0992db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 -- 172.21.15.96:0/4263551641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 msgr2=0x55afc0992db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0992db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 -- 172.21.15.96:0/4263551641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 msgr2=0x55afc09a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc09a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 -- 172.21.15.96:0/4263551641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc098ac00 con 0x55afc098ca10 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.159+0000 7f807f7fe640 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0992870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f808006c170 tx=0x7f80740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9cacfadbcf23a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.160+0000 7f8065ffb640 1 -- 172.21.15.96:0/4263551641 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807402c630 con 0x55afc098ca10 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.160+0000 7f8065ffb640 1 -- 172.21.15.96:0/4263551641 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80740228c0 con 0x55afc098ca10 2024-09-10T22:51:30.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.160+0000 7f8065ffb640 1 -- 172.21.15.96:0/4263551641 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074021a60 con 0x55afc098ca10 2024-09-10T22:51:30.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- 172.21.15.96:0/4263551641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 msgr2=0x55afc0992870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0992870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f808006c170 tx=0x7f80740046c0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- 172.21.15.96:0/4263551641 shutdown_connections 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc0992870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc0992db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 --2- 172.21.15.96:0/4263551641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc09a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- 172.21.15.96:0/4263551641 >> 172.21.15.96:0/4263551641 conn(0x55afc0989f20 msgr2=0x55afc098af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- 172.21.15.96:0/4263551641 shutdown_connections 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- 172.21.15.96:0/4263551641 wait complete. 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.161+0000 7f8088ac6740 1 -- start start 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 Processor -- start 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc099c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc099cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f807ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc099c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f807ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc099c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34114/0 (socket says 172.21.15.96:34114) 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f807ffff640 1 -- 172.21.15.96:0/1751090120 learned_addr learned my addr 172.21.15.96:0/1751090120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00f790 con 0x55afc098c0f0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00f0c0 con 0x55afc098ca10 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f806c00c300 con 0x55afc09989a0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f807f7fe640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc099cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8084c8e640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.162+0000 7f8088ac6740 10 client.0 _ll_register_callbacks cb 0x7f80700a8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 -- 172.21.15.96:0/1751090120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 msgr2=0x55afc099d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 -- 172.21.15.96:0/1751090120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 msgr2=0x55afc099c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc099c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc0a14330 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f807f7fe640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc099cde0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55afc099b5e0 tx=0x7f8074021430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=759a740208044f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f805f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074034030 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80740228c0 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8074022a70 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80740226e0 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f8088ac6740 10 client.25841 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.163+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afc0a155d0 con 0x55afc098ca10 2024-09-10T22:51:30.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80740228c0 con 0x55afc098ca10 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f807402c430 con 0x55afc098ca10 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 10 client.25841 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 20 client.25841 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc0a15d50 0x55afc0a18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 4 client.25841 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afc09008a0 con 0x55afc0a15d50 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.164+0000 7f807ffff640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc0a15d50 0x55afc0a18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f807ffff640 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc0a15d50 0x55afc0a18170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e10fc1561fed05 server_cookie=c872dc5037f3fd39 in_seq=0 out_seq=0 2024-09-10T22:51:30.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f805f7fe640 10 client.25841 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f805f7fe640 1 -- 172.21.15.96:0/1751090120 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afc09008a0 con 0x55afc0a15d50 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f805f7fe640 10 client.25841 handle_command_reply: tid=0 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f8088ac6740 1 client.25841 shutdown 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 msgr2=0x55afc099cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.165+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc099cde0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55afc099b5e0 tx=0x7f8074021430 comp rx=0 tx=0).stop 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 shutdown_connections 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55afc0a15d50 0x55afc0a18170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f805f7fe640 0 client.25841 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afc098ca10 0x55afc099cde0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afc09989a0 0x55afc099d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 --2- 172.21.15.96:0/1751090120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55afc098c0f0 0x55afc099c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 >> 172.21.15.96:0/1751090120 conn(0x55afc0989f20 msgr2=0x55afc0996c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 shutdown_connections 2024-09-10T22:51:30.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 1 -- 172.21.15.96:0/1751090120 wait complete. 2024-09-10T22:51:30.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.166+0000 7f8088ac6740 20 client.25841 trim_cache size 0 max 16384 2024-09-10T22:51:30.173 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.167+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 msgr2=0x7f80680b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.167+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 0x7f80680b78a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8074009460 tx=0x7f80740096b0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 msgr2=0x7f808006dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8070009fd0 tx=0x7f807000b8e0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 shutdown_connections 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080106ee0 0x7f808006dae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80680b53e0 0x7f80680b78a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80801023c0 0x7f8080074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 --2- 172.21.15.96:0/3086136971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080101aa0 0x7f8080074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:30.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 >> 172.21.15.96:0/3086136971 conn(0x7f80800fd730 msgr2=0x7f8080104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:30.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 shutdown_connections 2024-09-10T22:51:30.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:30.168+0000 7f80866f1640 1 -- 172.21.15.96:0/3086136971 wait complete. 2024-09-10T22:51:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 235.0 s, 3294.6 tps, lat 30.295 ms stddev 33.197 2024-09-10T22:51:30.961 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:30 smithi096 ceph-mon[30869]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 871 op/s 2024-09-10T22:51:31.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:30 smithi121 ceph-mon[35119]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 871 op/s 2024-09-10T22:51:31.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:30 smithi117 ceph-mon[35081]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 871 op/s 2024-09-10T22:51:32.004 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:51:32.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:51:32.411 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:33.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:32 smithi096 ceph-mon[30869]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 753 op/s 2024-09-10T22:51:33.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:33.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:32 smithi121 ceph-mon[35119]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 753 op/s 2024-09-10T22:51:33.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:33.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:32 smithi117 ceph-mon[35081]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 753 op/s 2024-09-10T22:51:33.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:51:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 -- 172.21.15.96:0/4284614023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc1029b0 msgr2=0x7f88bc102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/4284614023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc1029b0 0x7f88bc102e30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009930 tx=0x7f88ac02a420 comp rx=0 tx=0).stop 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 -- 172.21.15.96:0/4284614023 shutdown_connections 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/4284614023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc103370 0x7f88bc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/4284614023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc1029b0 0x7f88bc102e30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/4284614023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 0x7f88bc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 -- 172.21.15.96:0/4284614023 >> 172.21.15.96:0/4284614023 conn(0x7f88bc0fd140 msgr2=0x7f88bc0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 -- 172.21.15.96:0/4284614023 shutdown_connections 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.316+0000 7f88c3f0e640 1 -- 172.21.15.96:0/4284614023 wait complete. 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 Processor -- start 2024-09-10T22:51:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 -- start start 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 0x7f88bc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc1029b0 0x7f88bc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc10d4d0 con 0x7f88bc103370 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc10d350 con 0x7f88bc101860 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c3f0e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc10d650 con 0x7f88bc1029b0 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34134/0 (socket says 172.21.15.96:34134) 2024-09-10T22:51:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 -- 172.21.15.96:0/2139307478 learned_addr learned my addr 172.21.15.96:0/2139307478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c1caa640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 0x7f88bc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c14a9640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc1029b0 0x7f88bc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 -- 172.21.15.96:0/2139307478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc1029b0 msgr2=0x7f88bc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc1029b0 0x7f88bc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 -- 172.21.15.96:0/2139307478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 msgr2=0x7f88bc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 0x7f88bc198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 -- 172.21.15.96:0/2139307478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc19a950 con 0x7f88bc103370 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.317+0000 7f88c24ab640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f88b400eb10 tx=0x7f88b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.318+0000 7f88b2ffd640 1 -- 172.21.15.96:0/2139307478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004b20 con 0x7f88bc103370 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.318+0000 7f88b2ffd640 1 -- 172.21.15.96:0/2139307478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4004cc0 con 0x7f88bc103370 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.318+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc19ac40 con 0x7f88bc103370 2024-09-10T22:51:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.318+0000 7f88b2ffd640 1 -- 172.21.15.96:0/2139307478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4010670 con 0x7f88bc103370 2024-09-10T22:51:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.318+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a1310 con 0x7f88bc103370 2024-09-10T22:51:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.321+0000 7f88c6cdb740 1 Processor -- start 2024-09-10T22:51:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.322+0000 7f88b2ffd640 1 -- 172.21.15.96:0/2139307478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f88b4005020 con 0x7f88bc103370 2024-09-10T22:51:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.322+0000 7f88c6cdb740 1 -- start start 2024-09-10T22:51:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 0x5607e35ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5607e35306f0 con 0x5607e35bc7d0 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607e351fb20 con 0x5607e35bd110 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5607e3523930 con 0x5607e35cc470 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c14a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 0x5607e35ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c14a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51490/0 (socket says 172.21.15.96:51490) 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 -- 172.21.15.96:0/3915074878 learned_addr learned my addr 172.21.15.96:0/3915074878 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 -- 172.21.15.96:0/3915074878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 msgr2=0x5607e35ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 0x5607e35ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 -- 172.21.15.96:0/3915074878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 msgr2=0x5607e35bcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35bcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c1caa640 1 -- 172.21.15.96:0/3915074878 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e34e5530 con 0x5607e35bd110 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88c14a9640 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 0x5607e35ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.323+0000 7f88b2ffd640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 0x7f88a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.324+0000 7f88b2ffd640 1 -- 172.21.15.96:0/2139307478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f88b40e60c0 con 0x7f88bc103370 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.324+0000 7f88c14a9640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 0x7f88a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.324+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88b800bd50 tx=0x7f88b80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6583055ce39f929b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.324+0000 7f88a7fff640 1 -- 172.21.15.96:0/3915074878 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8005220 con 0x5607e35bd110 2024-09-10T22:51:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.324+0000 7f88a7fff640 1 -- 172.21.15.96:0/3915074878 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b800be40 con 0x5607e35bd110 2024-09-10T22:51:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.325+0000 7f88a7fff640 1 -- 172.21.15.96:0/3915074878 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80117c0 con 0x5607e35bd110 2024-09-10T22:51:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.325+0000 7f88c14a9640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 0x7f88a80b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.325+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3915074878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 msgr2=0x5607e35cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.325+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88b800bd50 tx=0x7f88b80107a0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3915074878 shutdown_connections 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bd110 0x5607e35cbf30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35cc470 0x5607e35ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3915074878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35bcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3915074878 >> 172.21.15.96:0/3915074878 conn(0x5607e35b9930 msgr2=0x5607e35ba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3915074878 shutdown_connections 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.326+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3915074878 wait complete. 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.327+0000 7f88c6cdb740 1 -- start start 2024-09-10T22:51:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.327+0000 7f88c6cdb740 1 Processor -- start 2024-09-10T22:51:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 0x5607e35d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5607e3457ab0 con 0x5607e35bd110 2024-09-10T22:51:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607e34c5730 con 0x5607e35cc470 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5607e35cf460 con 0x5607e35bc7d0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c6cdb740 10 client.0 _ll_register_callbacks cb 0x5607e35196b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49192/0 (socket says 172.21.15.96:49192) 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 -- 172.21.15.96:0/1259389892 learned_addr learned my addr 172.21.15.96:0/1259389892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c24ab640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 0x5607e35d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 -- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 msgr2=0x5607e35d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 -- 172.21.15.96:0/1259389892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 msgr2=0x5607e35d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.328+0000 7f88c1caa640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 0x5607e35d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c1caa640 1 -- 172.21.15.96:0/1259389892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e3281790 con 0x5607e35bc7d0 2024-09-10T22:51:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c24ab640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 0x5607e35d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c1caa640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006010 tx=0x7f88b8011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d014572c0e801ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b802a060 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f88b8025660 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607e3532640 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88b8011e60 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c6cdb740 10 client.25906 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607e32fa620 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.329+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5607e3532640 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f88b802f070 con 0x5607e35bc7d0 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 10 client.25906 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 20 client.25906 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e35dc240 0x5607e35de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 4 client.25906 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e352e0f0 con 0x5607e35dc240 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e35dc240 0x5607e35de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e35dc240 0x5607e35de680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725a021a9d19e8b3 server_cookie=4d9ea8309feec5bc in_seq=0 out_seq=0 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.330+0000 7f88a57fa640 10 client.25906 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.331+0000 7f88a57fa640 1 -- 172.21.15.96:0/1259389892 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607e352e0f0 con 0x5607e35dc240 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.331+0000 7f88a57fa640 10 client.25906 handle_command_reply: tid=0 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.331+0000 7f88c6cdb740 1 client.25906 shutdown 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.331+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 msgr2=0x5607e35d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.331+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006010 tx=0x7f88b8011a30 comp rx=0 tx=0).stop 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 shutdown_connections 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e35dc240 0x5607e35de680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bd110 0x5607e35d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1259389892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bc7d0 0x5607e35d06e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 >> 172.21.15.96:0/1259389892 conn(0x5607e35b9930 msgr2=0x5607e35b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88a57fa640 0 client.25906 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 shutdown_connections 2024-09-10T22:51:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1259389892 wait complete. 2024-09-10T22:51:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.332+0000 7f88c6cdb740 20 client.25906 trim_cache size 0 max 16384 2024-09-10T22:51:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.357+0000 7f88c6cdb740 1 Processor -- start 2024-09-10T22:51:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.358+0000 7f88c6cdb740 1 -- start start 2024-09-10T22:51:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.358+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 0x5607e35ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35cc470 0x5607e35b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 0x5607e35ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 0x5607e35ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51494/0 (socket says 172.21.15.96:51494) 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c1caa640 1 -- 172.21.15.96:0/1770795363 learned_addr learned my addr 172.21.15.96:0/1770795363 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a7cb0 con 0x5607e35cc470 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a97f0 con 0x5607e35bc7d0 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.359+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a8380 con 0x5607e35bd110 2024-09-10T22:51:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c24ab640 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35cc470 0x5607e35b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c14a9640 1 -- 172.21.15.96:0/1770795363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 msgr2=0x5607e35ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 0x5607e35ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c14a9640 1 -- 172.21.15.96:0/1770795363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35cc470 msgr2=0x5607e35b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35cc470 0x5607e35b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.360+0000 7f88c14a9640 1 -- 172.21.15.96:0/1770795363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e35b52d0 con 0x5607e35bd110 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.362+0000 7f88c14a9640 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35c8620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88ac031440 tx=0x7f88ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd628ca16c7783c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.362+0000 7f88a77fe640 1 -- 172.21.15.96:0/1770795363 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac035040 con 0x5607e35bd110 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.362+0000 7f88a77fe640 1 -- 172.21.15.96:0/1770795363 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac05d440 con 0x5607e35bd110 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.362+0000 7f88a77fe640 1 -- 172.21.15.96:0/1770795363 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac03c6a0 con 0x5607e35bd110 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.363+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 msgr2=0x5607e35c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.363+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35c8620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88ac031440 tx=0x7f88ac009260 comp rx=0 tx=0).stop 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 shutdown_connections 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35cc470 0x5607e35b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35c8620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/1770795363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35bc7d0 0x5607e35ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 >> 172.21.15.96:0/1770795363 conn(0x5607e35b9930 msgr2=0x5607e35c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 shutdown_connections 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.364+0000 7f88c6cdb740 1 -- 172.21.15.96:0/1770795363 wait complete. 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.365+0000 7f88c6cdb740 1 -- start start 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.365+0000 7f88c6cdb740 1 Processor -- start 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.365+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34170/0 (socket says 172.21.15.96:34170) 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a9120 con 0x5607e35bc7d0 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a8a50 con 0x5607e35cc470 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc1a5c90 con 0x5607e35bd110 2024-09-10T22:51:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c6cdb740 10 client.0 _ll_register_callbacks cb 0x7f88ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 -- 172.21.15.96:0/3198520731 learned_addr learned my addr 172.21.15.96:0/3198520731 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c14a9640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c24ab640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 -- 172.21.15.96:0/3198520731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 msgr2=0x5607e35d5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35d5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 -- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 msgr2=0x5607e35d68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 -- 172.21.15.96:0/3198520731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e35d70d0 con 0x5607e35bc7d0 2024-09-10T22:51:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c24ab640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88c1caa640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f88b80117d0 tx=0x7f88b8030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f830ced337f5d45b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.366+0000 7f88a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8035070 con 0x5607e35bc7d0 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac05d440 con 0x5607e35bc7d0 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f88b8025870 con 0x5607e35bc7d0 2024-09-10T22:51:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88b8011aa0 con 0x5607e35bc7d0 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 10 client.16296 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607e35d8440 con 0x5607e35bc7d0 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac05d440 con 0x5607e35bc7d0 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f88b8042030 con 0x5607e35bc7d0 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 20 client.16296 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e3649560 0x5607e364b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 4 client.16296 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607e352b520 con 0x5607e3649560 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.367+0000 7f88c14a9640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e3649560 0x5607e364b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.368+0000 7f88a57fa640 10 client.16296 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.369+0000 7f88c14a9640 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e3649560 0x5607e364b980 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74bd0c347c1af9 server_cookie=4b3e6097ab76c612 in_seq=0 out_seq=0 2024-09-10T22:51:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.389+0000 7f88a57fa640 1 -- 172.21.15.96:0/3198520731 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607e352b520 con 0x5607e3649560 2024-09-10T22:51:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.389+0000 7f88a57fa640 10 client.16296 handle_command_reply: tid=0 2024-09-10T22:51:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.389+0000 7f88c6cdb740 1 client.16296 shutdown 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 msgr2=0x5607e35d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f88b80117d0 tx=0x7f88b8030db0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 shutdown_connections 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5607e3649560 0x5607e364b980 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607e35cc470 0x5607e35d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607e35bd110 0x5607e35d5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 --2- 172.21.15.96:0/3198520731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e35bc7d0 0x5607e35d5630 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 >> 172.21.15.96:0/3198520731 conn(0x5607e35b9930 msgr2=0x5607e35b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88a57fa640 0 client.16296 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 shutdown_connections 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 1 -- 172.21.15.96:0/3198520731 wait complete. 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.391+0000 7f88c6cdb740 20 client.16296 trim_cache size 0 max 16384 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:51:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:51:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.392+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 msgr2=0x7f88a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.392+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 0x7f88a80b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac000bc0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.392+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 msgr2=0x7f88bc19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.392+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f88b400eb10 tx=0x7f88b400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 shutdown_connections 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88a80b52e0 0x7f88a80b77a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc103370 0x7f88bc19a090 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88bc1029b0 0x7f88bc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 --2- 172.21.15.96:0/2139307478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88bc101860 0x7f88bc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 >> 172.21.15.96:0/2139307478 conn(0x7f88bc0fd140 msgr2=0x7f88bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 shutdown_connections 2024-09-10T22:51:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:33.394+0000 7f88c3f0e640 1 -- 172.21.15.96:0/2139307478 wait complete. 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.495+0000 7f95b6858640 1 -- 172.21.15.96:0/920291818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 msgr2=0x7f95b011e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.495+0000 7f95b6858640 1 --2- 172.21.15.96:0/920291818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b011e490 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009ca0 tx=0x7f95a802a960 comp rx=0 tx=0).stop 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.496+0000 7f95b6858640 1 -- 172.21.15.96:0/920291818 shutdown_connections 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.496+0000 7f95b6858640 1 --2- 172.21.15.96:0/920291818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 0x7f95b010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.496+0000 7f95b6858640 1 --2- 172.21.15.96:0/920291818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b011e490 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.496+0000 7f95b6858640 1 --2- 172.21.15.96:0/920291818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.496+0000 7f95b6858640 1 -- 172.21.15.96:0/920291818 >> 172.21.15.96:0/920291818 conn(0x7f95b006dfc0 msgr2=0x7f95b006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.500+0000 7f95b6858640 1 -- 172.21.15.96:0/920291818 shutdown_connections 2024-09-10T22:51:33.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.500+0000 7f95b6858640 1 -- 172.21.15.96:0/920291818 wait complete. 2024-09-10T22:51:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 Processor -- start 2024-09-10T22:51:33.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 -- start start 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b011b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 0x7f95b011ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b0115860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112140 con 0x7f95b010a150 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0111fc0 con 0x7f95b010a530 2024-09-10T22:51:33.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.501+0000 7f95b6858640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01122c0 con 0x7f95b011c0a0 2024-09-10T22:51:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b011b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b011b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34182/0 (socket says 172.21.15.96:34182) 2024-09-10T22:51:33.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95affff640 1 -- 172.21.15.96:0/1068103114 learned_addr learned my addr 172.21.15.96:0/1068103114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 0x7f95b011ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b0115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 -- 172.21.15.96:0/1068103114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 msgr2=0x7f95b011ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 0x7f95b011ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 -- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 msgr2=0x7f95b011b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b011b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.502+0000 7f95b4df5640 1 -- 172.21.15.96:0/1068103114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b0116120 con 0x7f95b011c0a0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b0115860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95a802af50 tx=0x7f95a802fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95ad7fa640 1 -- 172.21.15.96:0/1068103114 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8034030 con 0x7f95b011c0a0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95ad7fa640 1 -- 172.21.15.96:0/1068103114 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a8030b10 con 0x7f95b011c0a0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b01163b0 con 0x7f95b011c0a0 2024-09-10T22:51:33.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95ad7fa640 1 -- 172.21.15.96:0/1068103114 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004640 con 0x7f95b011c0a0 2024-09-10T22:51:33.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.503+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b01bd210 con 0x7f95b011c0a0 2024-09-10T22:51:33.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.507+0000 7f95ad7fa640 1 -- 172.21.15.96:0/1068103114 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f95a8041070 con 0x7f95b011c0a0 2024-09-10T22:51:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.509+0000 7f95b8c2d740 1 Processor -- start 2024-09-10T22:51:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 -- start start 2024-09-10T22:51:33.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793deb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793deb790 0x563793dedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563793d55e60 con 0x563793deb790 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563793d45290 con 0x563793ddf800 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563793d490a0 con 0x563793ddeee0 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95ad7fa640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 0x7f95980b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49238/0 (socket says 172.21.15.96:49238) 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95affff640 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793deb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95af7fe640 1 -- 172.21.15.96:0/1259511051 learned_addr learned my addr 172.21.15.96:0/1259511051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793deb790 0x563793dedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 0x7f95980b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.510+0000 7f95ad7fa640 1 -- 172.21.15.96:0/1068103114 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f95a8106490 con 0x7f95b011c0a0 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 -- 172.21.15.96:0/1259511051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 msgr2=0x563793deb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793deb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 -- 172.21.15.96:0/1259511051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793deb790 msgr2=0x563793dedb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793deb790 0x563793dedb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 -- 172.21.15.96:0/1259511051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563793aa3930 con 0x563793ddeee0 2024-09-10T22:51:33.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95a4013950 tx=0x7f95a400e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47366942ed990603 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95967fc640 1 -- 172.21.15.96:0/1259511051 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a400b910 con 0x563793ddeee0 2024-09-10T22:51:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95967fc640 1 -- 172.21.15.96:0/1259511051 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a4005020 con 0x563793ddeee0 2024-09-10T22:51:33.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.511+0000 7f95967fc640 1 -- 172.21.15.96:0/1259511051 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004a00 con 0x563793ddeee0 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.512+0000 7f95b4df5640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 0x7f95980b7860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95a804b000 tx=0x7f95a80d1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.512+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1259511051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 msgr2=0x563793ddf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.512+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95a4013950 tx=0x7f95a400e430 comp rx=0 tx=0).stop 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1259511051 shutdown_connections 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793deb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793deb790 0x563793dedb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1259511051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793ddeee0 0x563793ddf2c0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95a4013950 tx=0x7f95a400e430 comp rx=0 tx=0).stop 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1259511051 >> 172.21.15.96:0/1259511051 conn(0x563793ddcd10 msgr2=0x563793dddeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1259511051 shutdown_connections 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.513+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1259511051 wait complete. 2024-09-10T22:51:33.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.514+0000 7f95b8c2d740 1 -- start start 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.514+0000 7f95b8c2d740 1 Processor -- start 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793de8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 0x563793df77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95affff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95affff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49254/0 (socket says 172.21.15.96:49254) 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793de8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793de8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51536/0 (socket says 172.21.15.96:51536) 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95affff640 1 -- 172.21.15.96:0/1567515672 learned_addr learned my addr 172.21.15.96:0/1567515672 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563793ba6890 con 0x563793de9400 2024-09-10T22:51:33.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563793cdfd40 con 0x563793ddf800 2024-09-10T22:51:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563793dee3e0 con 0x563793deb790 2024-09-10T22:51:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.515+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 0x563793df77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 -- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 msgr2=0x563793de8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793de8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 -- 172.21.15.96:0/1567515672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 msgr2=0x563793df77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 0x563793df77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563793df7e90 con 0x563793deb790 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 0x563793df77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95b8c2d740 10 client.0 _ll_register_callbacks cb 0x563793d43c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f95affff640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95a000bc20 tx=0x7f95a00039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b84424d9a290ff38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f958f7fe640 10 client.25924 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.516+0000 7f958f7fe640 1 -- 172.21.15.96:0/1567515672 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011650 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.517+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a4005020 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.517+0000 7f958f7fe640 1 -- 172.21.15.96:0/1567515672 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f95a00117f0 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.518+0000 7f958f7fe640 1 -- 172.21.15.96:0/1567515672 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f95a00119f0 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.518+0000 7f95b8c2d740 10 client.25924 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.518+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x563793dfa6a0 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.518+0000 7f958f7fe640 1 -- 172.21.15.96:0/1567515672 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f95a0011b90 con 0x563793deb790 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.518+0000 7f95b8c2d740 10 client.25924 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95b8c2d740 10 client.25924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95b8c2d740 20 client.25924 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793dfbf00 0x563793dfe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95b8c2d740 4 client.25924 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95a4005020 con 0x563793dfbf00 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.519+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793dfbf00 0x563793dfe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.520+0000 7f958f7fe640 10 client.25924 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.520+0000 7f95af7fe640 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793dfbf00 0x563793dfe3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac693b4baff7029 server_cookie=875c45565829633f in_seq=0 out_seq=0 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f958f7fe640 1 -- 172.21.15.96:0/1567515672 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f95a4005020 con 0x563793dfbf00 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f958f7fe640 10 client.25924 handle_command_reply: tid=0 2024-09-10T22:51:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 client.25924 shutdown 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 msgr2=0x563793de8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95a000bc20 tx=0x7f95a00039c0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 shutdown_connections 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793dfbf00 0x563793dfe3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793de9400 0x563793df77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793deb790 0x563793de8ec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/1567515672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793de8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 >> 172.21.15.96:0/1567515672 conn(0x563793ddcd10 msgr2=0x563793ddbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.521+0000 7f958f7fe640 0 client.25924 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.522+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 shutdown_connections 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.522+0000 7f95b8c2d740 1 -- 172.21.15.96:0/1567515672 wait complete. 2024-09-10T22:51:33.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.522+0000 7f95b8c2d740 20 client.25924 trim_cache size 0 max 16384 2024-09-10T22:51:33.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.549+0000 7f95b8c2d740 1 Processor -- start 2024-09-10T22:51:33.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 -- start start 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 0x563793df76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 0x563793e03cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00e330 con 0x563793ddf800 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00fe70 con 0x563793ddcd10 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00ea00 con 0x563793de9400 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 0x563793e03cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 0x563793e03cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34202/0 (socket says 172.21.15.96:34202) 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95affff640 1 -- 172.21.15.96:0/125195015 learned_addr learned my addr 172.21.15.96:0/125195015 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95af7fe640 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 0x563793df76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.550+0000 7f95af7fe640 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 -- 172.21.15.96:0/125195015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 msgr2=0x563793e04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 -- 172.21.15.96:0/125195015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 msgr2=0x563793e03cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 0x563793e03cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 -- 172.21.15.96:0/125195015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563793ddd790 con 0x563793ddcd10 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.551+0000 7f95af7fe640 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 0x563793df76a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95a4012000 tx=0x7f95a4013f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daeae322fa240229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.552+0000 7f9595ffb640 1 -- 172.21.15.96:0/125195015 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4004020 con 0x563793ddcd10 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.552+0000 7f9595ffb640 1 -- 172.21.15.96:0/125195015 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a402e040 con 0x563793ddcd10 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.552+0000 7f9595ffb640 1 -- 172.21.15.96:0/125195015 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40052b0 con 0x563793ddcd10 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 -- 172.21.15.96:0/125195015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 msgr2=0x563793df76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 0x563793df76a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f95a4012000 tx=0x7f95a4013f90 comp rx=0 tx=0).stop 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 -- 172.21.15.96:0/125195015 shutdown_connections 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddf800 0x563793e03cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/125195015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddcd10 0x563793df76a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.553+0000 7f95b8c2d740 1 -- 172.21.15.96:0/125195015 >> 172.21.15.96:0/125195015 conn(0x563793deabe0 msgr2=0x563793ddbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.554+0000 7f95b8c2d740 1 -- 172.21.15.96:0/125195015 shutdown_connections 2024-09-10T22:51:33.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.554+0000 7f95b8c2d740 1 -- 172.21.15.96:0/125195015 wait complete. 2024-09-10T22:51:33.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.554+0000 7f95b8c2d740 1 -- start start 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 Processor -- start 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 0x563793df1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793df22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e67ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00f7a0 con 0x563793ddcd10 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00f0d0 con 0x563793ddf800 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f959c00c310 con 0x563793de9400 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95b8c2d740 10 client.0 _ll_register_callbacks cb 0x7f95a0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 0x563793df1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 0x563793df1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34212/0 (socket says 172.21.15.96:34212) 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.555+0000 7f95af7fe640 1 -- 172.21.15.96:0/2341259258 learned_addr learned my addr 172.21.15.96:0/2341259258 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:33.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e67ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95affff640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793df22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 -- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 msgr2=0x563793df22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793df22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 -- 172.21.15.96:0/2341259258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 msgr2=0x563793df1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 0x563793df1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 -- 172.21.15.96:0/2341259258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563793e68450 con 0x563793de9400 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95affff640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793df22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f95af7fe640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e67ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95a4013730 tx=0x7f95a4016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cedb86d1a17591bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f958f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.556+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a403c060 con 0x563793de9400 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.557+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f95a400b910 con 0x563793de9400 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.557+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a402e040 con 0x563793de9400 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.557+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f95a4004450 con 0x563793de9400 2024-09-10T22:51:33.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.557+0000 7f95b8c2d740 10 client.25930 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:33.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.557+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563793e697a0 con 0x563793de9400 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a40035f0 con 0x563793de9400 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f95a4041070 con 0x563793de9400 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 10 client.25930 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 20 client.25930 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793deb790 0x563793e6bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 4 client.25930 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.558+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f95a402e040 con 0x563793deb790 2024-09-10T22:51:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.559+0000 7f95affff640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793deb790 0x563793e6bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:33.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.561+0000 7f95affff640 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793deb790 0x563793e6bd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e075ee0f3aec35b6 server_cookie=46ad4a2a9b0c7ee1 in_seq=0 out_seq=0 2024-09-10T22:51:33.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.561+0000 7f958f7fe640 10 client.25930 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f958f7fe640 1 -- 172.21.15.96:0/2341259258 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f95a402e040 con 0x563793deb790 2024-09-10T22:51:33.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f958f7fe640 10 client.25930 handle_command_reply: tid=0 2024-09-10T22:51:33.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 client.25930 shutdown 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 msgr2=0x563793e67ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e67ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95a4013730 tx=0x7f95a4016040 comp rx=0 tx=0).stop 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 shutdown_connections 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563793deb790 0x563793e6bd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563793ddf800 0x563793df22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563793de9400 0x563793e67ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 --2- 172.21.15.96:0/2341259258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563793ddcd10 0x563793df1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 >> 172.21.15.96:0/2341259258 conn(0x563793deabe0 msgr2=0x563793df6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.563+0000 7f958f7fe640 0 client.25930 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.564+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 shutdown_connections 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.564+0000 7f95b8c2d740 1 -- 172.21.15.96:0/2341259258 wait complete. 2024-09-10T22:51:33.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.564+0000 7f95b8c2d740 20 client.25930 trim_cache size 0 max 16384 2024-09-10T22:51:33.570 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:51:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:51:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108", 2024-09-10T22:51:33.570 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:51:33.570 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 msgr2=0x7f95980b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 0x7f95980b7860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f95a804b000 tx=0x7f95a80d1040 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 msgr2=0x7f95b0115860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b0115860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95a802af50 tx=0x7f95a802fb00 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 shutdown_connections 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b010a530 0x7f95b011ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.565+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f95980b53a0 0x7f95980b7860 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.566+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b011c0a0 0x7f95b0115860 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.566+0000 7f95b6858640 1 --2- 172.21.15.96:0/1068103114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95b010a150 0x7f95b011b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.566+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 >> 172.21.15.96:0/1068103114 conn(0x7f95b006dfc0 msgr2=0x7f95b0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:33.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.566+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 shutdown_connections 2024-09-10T22:51:33.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:33.566+0000 7f95b6858640 1 -- 172.21.15.96:0/1068103114 wait complete. 2024-09-10T22:51:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:51:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:51:34.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:51:34.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:51:34.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:51:34.383 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:51:34.764 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:51:35.053 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 761 op/s 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.054 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:34 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 761 op/s 2024-09-10T22:51:35.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:51:35.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:51:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:51:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:34 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 761 op/s 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:34 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:51:35.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.336+0000 7f81bf985640 1 -- 172.21.15.96:0/2818508570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 msgr2=0x7f81b8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.336+0000 7f81bf985640 1 --2- 172.21.15.96:0/2818508570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 0x7f81b8107ae0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a802a410 comp rx=0 tx=0).stop 2024-09-10T22:51:35.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 -- 172.21.15.96:0/2818508570 shutdown_connections 2024-09-10T22:51:35.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 --2- 172.21.15.96:0/2818508570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b8108020 0x7f81b810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 --2- 172.21.15.96:0/2818508570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 0x7f81b8107ae0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 --2- 172.21.15.96:0/2818508570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 -- 172.21.15.96:0/2818508570 >> 172.21.15.96:0/2818508570 conn(0x7f81b80fd700 msgr2=0x7f81b80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.337+0000 7f81bf985640 1 -- 172.21.15.96:0/2818508570 shutdown_connections 2024-09-10T22:51:35.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.338+0000 7f81bf985640 1 -- 172.21.15.96:0/2818508570 wait complete. 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.338+0000 7f81bf985640 1 Processor -- start 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.338+0000 7f81bf985640 1 -- start start 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 0x7f81b810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b8108020 0x7f81b810d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8110210 con 0x7f81b81076a0 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8110090 con 0x7f81b8101c90 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bf985640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8110390 con 0x7f81b8108020 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51554/0 (socket says 172.21.15.96:51554) 2024-09-10T22:51:35.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 -- 172.21.15.96:0/1215127540 learned_addr learned my addr 172.21.15.96:0/1215127540 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 -- 172.21.15.96:0/1215127540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b8108020 msgr2=0x7f81b810d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b8108020 0x7f81b810d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 -- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 msgr2=0x7f81b810d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 0x7f81b810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.339+0000 7f81bd721640 1 -- 172.21.15.96:0/1215127540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b81abaa0 con 0x7f81b8101c90 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.340+0000 7f81bd721640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81b400d950 tx=0x7f81b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.340+0000 7f81ae7fc640 1 -- 172.21.15.96:0/1215127540 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4017070 con 0x7f81b8101c90 2024-09-10T22:51:35.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.340+0000 7f81ae7fc640 1 -- 172.21.15.96:0/1215127540 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4009c30 con 0x7f81b8101c90 2024-09-10T22:51:35.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.340+0000 7f81ae7fc640 1 -- 172.21.15.96:0/1215127540 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400f620 con 0x7f81b8101c90 2024-09-10T22:51:35.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.341+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b81abd90 con 0x7f81b8101c90 2024-09-10T22:51:35.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.341+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b81ac1c0 con 0x7f81b8101c90 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 Processor -- start 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 -- start start 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e08a9c0 0x564c4e096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e096950 0x564c4e098d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c4e000e50 con 0x564c4e096950 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564c4dff0280 con 0x564c4e08a0a0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.345+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c4dff4090 con 0x564c4e08a9c0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51558/0 (socket says 172.21.15.96:51558) 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 -- 172.21.15.96:0/2733611770 learned_addr learned my addr 172.21.15.96:0/2733611770 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81ae7fc640 1 -- 172.21.15.96:0/1215127540 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f81b4023060 con 0x7f81b8101c90 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e096950 0x564c4e098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bd721640 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e08a9c0 0x564c4e096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 -- 172.21.15.96:0/2733611770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e08a9c0 msgr2=0x564c4e096410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e08a9c0 0x564c4e096410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 -- 172.21.15.96:0/2733611770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e096950 msgr2=0x564c4e098d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e096950 0x564c4e098d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 -- 172.21.15.96:0/2733611770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4dd4e650 con 0x564c4e08a0a0 2024-09-10T22:51:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.346+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009d60 tx=0x7f81b000d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830e64f63fd56315 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81a37fe640 1 -- 172.21.15.96:0/2733611770 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0003970 con 0x564c4e08a0a0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81a37fe640 1 -- 172.21.15.96:0/2733611770 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b0004020 con 0x564c4e08a0a0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81a37fe640 1 -- 172.21.15.96:0/2733611770 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0010830 con 0x564c4e08a0a0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81ae7fc640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 0x7f81a40b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81bd721640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 0x7f81a40b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.347+0000 7f81ae7fc640 1 -- 172.21.15.96:0/1215127540 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81b40e5160 con 0x7f81b8101c90 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2733611770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 msgr2=0x564c4e08a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009d60 tx=0x7f81b000d6b0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81bd721640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 0x7f81a40b78e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81b40a8b90 tx=0x7f81b40ea040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2733611770 shutdown_connections 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e096950 0x564c4e098d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e08a9c0 0x564c4e096410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2733611770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a0a0 0x564c4e08a480 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2733611770 >> 172.21.15.96:0/2733611770 conn(0x564c4e087ed0 msgr2=0x564c4e089070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.348+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2733611770 shutdown_connections 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.350+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2733611770 wait complete. 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.350+0000 7f81c1d5a740 1 -- start start 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.350+0000 7f81c1d5a740 1 Processor -- start 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e096280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e093d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c4de51700 con 0x564c4e08a0a0 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564c4df8b110 con 0x564c4e08a9c0 2024-09-10T22:51:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c4e0995d0 con 0x564c4e096950 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51560/0 (socket says 172.21.15.96:51560) 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 -- 172.21.15.96:0/2490482325 learned_addr learned my addr 172.21.15.96:0/2490482325 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81c1d5a740 10 client.0 _ll_register_callbacks cb 0x564c4dfa4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e093d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e096280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 -- 172.21.15.96:0/2490482325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 msgr2=0x564c4e093d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e093d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 -- 172.21.15.96:0/2490482325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 msgr2=0x564c4e096280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e096280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 -- 172.21.15.96:0/2490482325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4dfb8c40 con 0x564c4e08a9c0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d335283743e0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.351+0000 7f81a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8040070 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81a8003a00 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b0004020 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81a8035040 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 10 client.25886 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c4e0a4f50 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b0004020 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f81a8045460 con 0x564c4e08a9c0 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 10 client.25886 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 20 client.25886 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e0a6410 0x564c4e0a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 4 client.25886 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4e002da0 con 0x564c4e0a6410 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.352+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e0a6410 0x564c4e0a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.353+0000 7f81a0ff9640 10 client.25886 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.353+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e0a6410 0x564c4e0a8800 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7711ae3af746506f server_cookie=81f0a886b5cfdb35 in_seq=0 out_seq=0 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.354+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2490482325 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c4e002da0 con 0x564c4e0a6410 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.354+0000 7f81a0ff9640 10 client.25886 handle_command_reply: tid=0 2024-09-10T22:51:35.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.354+0000 7f81c1d5a740 1 client.25886 shutdown 2024-09-10T22:51:35.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.354+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 msgr2=0x564c4e0936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.354+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a8004270 comp rx=0 tx=0).stop 2024-09-10T22:51:35.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 shutdown_connections 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e0a6410 0x564c4e0a8800 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e0936d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e093d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2490482325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e096280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 >> 172.21.15.96:0/2490482325 conn(0x564c4e087ed0 msgr2=0x564c4e08da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81a0ff9640 0 client.25886 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 shutdown_connections 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.355+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2490482325 wait complete. 2024-09-10T22:51:35.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.356+0000 7f81c1d5a740 20 client.25886 trim_cache size 0 max 16384 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 Processor -- start 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 -- start start 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e095410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e097d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b4900 con 0x564c4e08a0a0 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b6440 con 0x564c4e08a9c0 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.380+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b4fd0 con 0x564c4e096950 2024-09-10T22:51:35.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bcf20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e095410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34236/0 (socket says 172.21.15.96:34236) 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 -- 172.21.15.96:0/576772496 learned_addr learned my addr 172.21.15.96:0/576772496 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e097d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 -- 172.21.15.96:0/576772496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 msgr2=0x564c4e097d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e097d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 -- 172.21.15.96:0/576772496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 msgr2=0x564c4e095410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e095410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 -- 172.21.15.96:0/576772496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4e0940c0 con 0x564c4e08a0a0 2024-09-10T22:51:35.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bcf20640 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:35.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81bdf22640 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009d60 tx=0x7f81b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db3bd804fee4b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81a2ffd640 1 -- 172.21.15.96:0/576772496 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00048c0 con 0x564c4e08a0a0 2024-09-10T22:51:35.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81a2ffd640 1 -- 172.21.15.96:0/576772496 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b0025040 con 0x564c4e08a0a0 2024-09-10T22:51:35.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.381+0000 7f81a2ffd640 1 -- 172.21.15.96:0/576772496 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0010c30 con 0x564c4e08a0a0 2024-09-10T22:51:35.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- 172.21.15.96:0/576772496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 msgr2=0x564c4e0a2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009d60 tx=0x7f81b0005fd0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- 172.21.15.96:0/576772496 shutdown_connections 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e097d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/576772496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e0a2830 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- 172.21.15.96:0/576772496 >> 172.21.15.96:0/576772496 conn(0x564c4e087ed0 msgr2=0x564c4e094d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- 172.21.15.96:0/576772496 shutdown_connections 2024-09-10T22:51:35.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- 172.21.15.96:0/576772496 wait complete. 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.382+0000 7f81c1d5a740 1 -- start start 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 Processor -- start 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e09c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e09cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b5d70 con 0x564c4e08a0a0 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b56a0 con 0x564c4e08a9c0 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81b28e0 con 0x564c4e096950 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81c1d5a740 10 client.0 _ll_register_callbacks cb 0x7f81a8005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:35.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34242/0 (socket says 172.21.15.96:34242) 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 -- 172.21.15.96:0/2495596909 learned_addr learned my addr 172.21.15.96:0/2495596909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e09c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e09cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 -- 172.21.15.96:0/2495596909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 msgr2=0x564c4e09cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e09cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 -- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 msgr2=0x564c4e09c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e09c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bdf22640 1 -- 172.21.15.96:0/2495596909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4e09d3a0 con 0x564c4e08a0a0 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.383+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e09c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81bdf22640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000f80 tx=0x7f81b0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac57f37e5308dc0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:35.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0010c70 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b0025040 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f81b0003970 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81b0010e10 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 10 client.16329 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c4e113fe0 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b0025040 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f81b0005000 con 0x564c4e08a0a0 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 10 client.16329 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 20 client.16329 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e114460 0x564c4e116850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:35.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 4 client.16329 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.384+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c4dffe850 con 0x564c4e114460 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e114460 0x564c4e116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81bcf20640 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e114460 0x564c4e116850 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29cc4c38a00fa001 server_cookie=f31088e338d7c7b9 in_seq=0 out_seq=0 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81a0ff9640 10 client.16329 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81a0ff9640 1 -- 172.21.15.96:0/2495596909 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x564c4dffe850 con 0x564c4e114460 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81a0ff9640 10 client.16329 handle_command_reply: tid=0 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 client.16329 shutdown 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 msgr2=0x564c4e09ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f81b0000f80 tx=0x7f81b0005ee0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 shutdown_connections 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c4e114460 0x564c4e116850 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c4e08a9c0 0x564c4e09c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c4e096950 0x564c4e09cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81a0ff9640 0 client.16329 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 --2- 172.21.15.96:0/2495596909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c4e08a0a0 0x564c4e09ee10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.385+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 >> 172.21.15.96:0/2495596909 conn(0x564c4e087ed0 msgr2=0x564c4e095520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.386+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 shutdown_connections 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.386+0000 7f81c1d5a740 1 -- 172.21.15.96:0/2495596909 wait complete. 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.386+0000 7f81c1d5a740 20 client.16329 trim_cache size 0 max 16384 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:51:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108", 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:51:35.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 msgr2=0x7f81a40b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 0x7f81a40b78e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81b40a8b90 tx=0x7f81b40ea040 comp rx=0 tx=0).stop 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 msgr2=0x7f81b810cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81b400d950 tx=0x7f81b400de10 comp rx=0 tx=0).stop 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 shutdown_connections 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f81a40b5420 0x7f81a40b78e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81b8108020 0x7f81b810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81b81076a0 0x7f81b810d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 --2- 172.21.15.96:0/1215127540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b8101c90 0x7f81b810cb20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:35.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 >> 172.21.15.96:0/1215127540 conn(0x7f81b80fd700 msgr2=0x7f81b80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:35.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 shutdown_connections 2024-09-10T22:51:35.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:51:35.387+0000 7f81bf985640 1 -- 172.21.15.96:0/1215127540 wait complete. 2024-09-10T22:51:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 240.0 s, 3194.8 tps, lat 31.416 ms stddev 33.791 2024-09-10T22:51:36.167 INFO:tasks.cephfs.filesystem:scrub status for tag:1bf4032f-07a5-46ac-8fb1-5f28fcdd5108 - {'path': '/', 'tag': '1bf4032f-07a5-46ac-8fb1-5f28fcdd5108', 'options': 'recursive,force'} 2024-09-10T22:51:37.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:36 smithi096 ceph-mon[30869]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-10T22:51:37.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:36 smithi121 ceph-mon[35119]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-10T22:51:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:36 smithi117 ceph-mon[35081]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-10T22:51:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:38 smithi096 ceph-mon[30869]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-09-10T22:51:39.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:38 smithi121 ceph-mon[35119]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-09-10T22:51:39.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:38 smithi117 ceph-mon[35081]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-09-10T22:51:40.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:39 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:51:40.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:39 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:51:40.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:39 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:51:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 245.0 s, 3775.6 tps, lat 26.486 ms stddev 28.446 2024-09-10T22:51:41.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:40 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:51:41.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:40 smithi096 ceph-mon[30869]: pgmap v478: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 9.9 MiB/s wr, 938 op/s 2024-09-10T22:51:41.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:40 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:51:41.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:40 smithi121 ceph-mon[35119]: pgmap v478: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 9.9 MiB/s wr, 938 op/s 2024-09-10T22:51:41.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:40 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:51:41.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:40 smithi117 ceph-mon[35081]: pgmap v478: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 9.9 MiB/s wr, 938 op/s 2024-09-10T22:51:43.077 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:42 smithi117 ceph-mon[35081]: pgmap v479: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.1 MiB/s wr, 841 op/s 2024-09-10T22:51:43.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:42 smithi096 ceph-mon[30869]: pgmap v479: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.1 MiB/s wr, 841 op/s 2024-09-10T22:51:43.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:42 smithi121 ceph-mon[35119]: pgmap v479: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.1 MiB/s wr, 841 op/s 2024-09-10T22:51:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:44 smithi096 ceph-mon[30869]: pgmap v480: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-10T22:51:45.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:44 smithi121 ceph-mon[35119]: pgmap v480: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-10T22:51:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:44 smithi117 ceph-mon[35081]: pgmap v480: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-09-10T22:51:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 250.0 s, 3245.0 tps, lat 30.668 ms stddev 31.714 2024-09-10T22:51:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.532+0000 7f47995d3640 1 -- 172.21.15.96:0/1998971820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 msgr2=0x7f4794102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.532+0000 7f47995d3640 1 --2- 172.21.15.96:0/1998971820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f4794102e40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4788009900 tx=0x7f478802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.532+0000 7f47995d3640 1 -- 172.21.15.96:0/1998971820 shutdown_connections 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.532+0000 7f47995d3640 1 --2- 172.21.15.96:0/1998971820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f4794102e40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 --2- 172.21.15.96:0/1998971820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794103380 0x7f4794109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 --2- 172.21.15.96:0/1998971820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794101870 0x7f4794101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 -- 172.21.15.96:0/1998971820 >> 172.21.15.96:0/1998971820 conn(0x7f47940fd130 msgr2=0x7f47940ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 -- 172.21.15.96:0/1998971820 shutdown_connections 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 -- 172.21.15.96:0/1998971820 wait complete. 2024-09-10T22:51:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 Processor -- start 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.533+0000 7f47995d3640 1 -- start start 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794101870 0x7f4794198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f47941991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f479410d510 con 0x7f4794103380 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f479410d390 con 0x7f47941029c0 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47995d3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f479410d690 con 0x7f4794101870 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f4792ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794101870 0x7f4794198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f47941991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58862/0 (socket says 172.21.15.96:58862) 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47937fe640 1 -- 172.21.15.96:0/1189359659 learned_addr learned my addr 172.21.15.96:0/1189359659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 -- 172.21.15.96:0/1189359659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794101870 msgr2=0x7f4794198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794101870 0x7f4794198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 -- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 msgr2=0x7f4794199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47927fc640 1 -- 172.21.15.96:0/1189359659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479419a690 con 0x7f47941029c0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.534+0000 7f47937fe640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f47927fc640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f47941991a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f478802a910 tx=0x7f4788003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f477bfff640 1 -- 172.21.15.96:0/1189359659 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478803f030 con 0x7f47941029c0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f477bfff640 1 -- 172.21.15.96:0/1189359659 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4788032bf0 con 0x7f47941029c0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f477bfff640 1 -- 172.21.15.96:0/1189359659 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788035070 con 0x7f47941029c0 2024-09-10T22:51:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479419a920 con 0x7f47941029c0 2024-09-10T22:51:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.535+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a0e70 con 0x7f47941029c0 2024-09-10T22:51:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.539+0000 7f477bfff640 1 -- 172.21.15.96:0/1189359659 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f478803a3c0 con 0x7f47941029c0 2024-09-10T22:51:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.539+0000 7f479c47d740 1 Processor -- start 2024-09-10T22:51:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 -- start start 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb857b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb857c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efb84ea6f0 con 0x55efb857b1b0 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efb84d9b20 con 0x55efb8578860 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f479c47d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55efb84dd930 con 0x55efb857bbc0 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f477bfff640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 0x7f477c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.540+0000 7f47937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:34146/0 (socket says 172.21.15.96:34146) 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 -- 172.21.15.96:0/2047775327 learned_addr learned my addr 172.21.15.96:0/2047775327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 0x7f477c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f4792ffd640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb857b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f477bfff640 1 -- 172.21.15.96:0/1189359659 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f47881068a0 con 0x7f47941029c0 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47927fc640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb857c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 -- 172.21.15.96:0/2047775327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 msgr2=0x55efb857c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb857c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 -- 172.21.15.96:0/2047775327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 msgr2=0x55efb857b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb857b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 -- 172.21.15.96:0/2047775327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb849f530 con 0x55efb8578860 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47927fc640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb857c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f478c011580 tx=0x7f478c011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03269c49ab4d1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47937fe640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 0x7f477c0b78a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f478c011f40 tx=0x7f478c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47797fa640 1 -- 172.21.15.96:0/2047775327 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c023070 con 0x55efb8578860 2024-09-10T22:51:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47797fa640 1 -- 172.21.15.96:0/2047775327 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c0135b0 con 0x55efb8578860 2024-09-10T22:51:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.541+0000 7f47797fa640 1 -- 172.21.15.96:0/2047775327 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c027400 con 0x55efb8578860 2024-09-10T22:51:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 -- 172.21.15.96:0/2047775327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 msgr2=0x55efb857ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f478c011580 tx=0x7f478c011a40 comp rx=0 tx=0).stop 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 -- 172.21.15.96:0/2047775327 shutdown_connections 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb857c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb857b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 --2- 172.21.15.96:0/2047775327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb857ac70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 -- 172.21.15.96:0/2047775327 >> 172.21.15.96:0/2047775327 conn(0x55efb85739b0 msgr2=0x55efb8574740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 -- 172.21.15.96:0/2047775327 shutdown_connections 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.542+0000 7f479c47d740 1 -- 172.21.15.96:0/2047775327 wait complete. 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 -- start start 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 Processor -- start 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb858a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efb8411ab0 con 0x55efb857b1b0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efb847f730 con 0x55efb8578860 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55efb857cad0 con 0x55efb857bbc0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f479c47d740 10 client.0 _ll_register_callbacks cb 0x55efb84d36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37878/0 (socket says 172.21.15.96:37878) 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 -- 172.21.15.96:0/348751383 learned_addr learned my addr 172.21.15.96:0/348751383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47927fc640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f4792ffd640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb858a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 -- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 msgr2=0x55efb858a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb858a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 -- 172.21.15.96:0/348751383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 msgr2=0x55efb858ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.543+0000 7f47937fe640 1 -- 172.21.15.96:0/348751383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb823b790 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f47937fe640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f478c020d90 tx=0x7f478c020dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd8072e4022550a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4792ffd640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb858a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4776ff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c01c040 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478c0135b0 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f478c01f420 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f478c023040 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f479c47d740 10 client.25954 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efb82b4620 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.544+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c036070 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f478c00f040 con 0x55efb857bbc0 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 10 client.25954 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 20 client.25954 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb8595c70 0x55efb8598060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 4 client.25954 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.545+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f478c0135b0 con 0x55efb8595c70 2024-09-10T22:51:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.546+0000 7f4792ffd640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb8595c70 0x55efb8598060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f4792ffd640 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb8595c70 0x55efb8598060 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d100ce7dd403b97 server_cookie=da5c46d4d6eee90b in_seq=0 out_seq=0 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f4776ff5640 10 client.25954 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f4776ff5640 1 -- 172.21.15.96:0/348751383 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f478c0135b0 con 0x55efb8595c70 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f4776ff5640 10 client.25954 handle_command_reply: tid=0 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f479c47d740 1 client.25954 shutdown 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 msgr2=0x55efb858b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f478c020d90 tx=0x7f478c020dc0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 shutdown_connections 2024-09-10T22:51:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.547+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb8595c70 0x55efb8598060 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb858b8f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 --2- 172.21.15.96:0/348751383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb858a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f4776ff5640 0 client.25954 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 >> 172.21.15.96:0/348751383 conn(0x55efb85739b0 msgr2=0x55efb8572bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 shutdown_connections 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 1 -- 172.21.15.96:0/348751383 wait complete. 2024-09-10T22:51:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.548+0000 7f479c47d740 20 client.25954 trim_cache size 0 max 16384 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.570+0000 7f479c47d740 1 Processor -- start 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.570+0000 7f479c47d740 1 -- start start 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.570+0000 7f479c47d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb85889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f479c47d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb8588f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f479c47d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f479c47d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a7b00 con 0x55efb857b1b0 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f479c47d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a9640 con 0x55efb8578860 2024-09-10T22:51:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f479c47d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a81d0 con 0x55efb857bbc0 2024-09-10T22:51:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f47937fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f47937fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37880/0 (socket says 172.21.15.96:37880) 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.571+0000 7f47937fe640 1 -- 172.21.15.96:0/681247386 learned_addr learned my addr 172.21.15.96:0/681247386 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47927fc640 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb8588f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47937fe640 1 -- 172.21.15.96:0/681247386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 msgr2=0x55efb85889e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47937fe640 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb85889e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47937fe640 1 -- 172.21.15.96:0/681247386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 msgr2=0x55efb8588f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47937fe640 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb8588f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47937fe640 1 -- 172.21.15.96:0/681247386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb8585ca0 con 0x55efb857bbc0 2024-09-10T22:51:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.572+0000 7f47927fc640 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb8588f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.573+0000 7f47937fe640 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f478c009270 tx=0x7f478c011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951fb984d71d13f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.573+0000 7f4778ff9640 1 -- 172.21.15.96:0/681247386 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c0135b0 con 0x55efb857bbc0 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.573+0000 7f4778ff9640 1 -- 172.21.15.96:0/681247386 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c036040 con 0x55efb857bbc0 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.573+0000 7f4778ff9640 1 -- 172.21.15.96:0/681247386 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c005330 con 0x55efb857bbc0 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- 172.21.15.96:0/681247386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 msgr2=0x55efb85896b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f478c009270 tx=0x7f478c011a40 comp rx=0 tx=0).stop 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- 172.21.15.96:0/681247386 shutdown_connections 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb857bbc0 0x55efb85896b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb8588f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 --2- 172.21.15.96:0/681247386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb8578860 0x55efb85889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- 172.21.15.96:0/681247386 >> 172.21.15.96:0/681247386 conn(0x55efb85739b0 msgr2=0x55efb856eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- 172.21.15.96:0/681247386 shutdown_connections 2024-09-10T22:51:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- 172.21.15.96:0/681247386 wait complete. 2024-09-10T22:51:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.574+0000 7f479c47d740 1 -- start start 2024-09-10T22:51:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 Processor -- start 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb8578860 0x55efb8592110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a8f70 con 0x55efb857b1b0 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a88a0 con 0x55efb857bbc0 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a5ae0 con 0x55efb8578860 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f479c47d740 10 client.0 _ll_register_callbacks cb 0x7f47840097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f47937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f47937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:34172/0 (socket says 172.21.15.96:34172) 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.575+0000 7f47937fe640 1 -- 172.21.15.96:0/2323241278 learned_addr learned my addr 172.21.15.96:0/2323241278 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 -- 172.21.15.96:0/2323241278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb8578860 msgr2=0x55efb8592110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb8578860 0x55efb8592110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 -- 172.21.15.96:0/2323241278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 msgr2=0x55efb858de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 -- 172.21.15.96:0/2323241278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb858f0c0 con 0x55efb857bbc0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f47937fe640 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f478c010010 tx=0x7f478c0093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5688fd451d5c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f4776ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c04c020 con 0x55efb857bbc0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f478c0050b0 con 0x55efb857bbc0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.576+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478c036040 con 0x55efb857bbc0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f478c0052b0 con 0x55efb857bbc0 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 10 client.25904 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efb8602e40 con 0x55efb857bbc0 2024-09-10T22:51:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c046b40 con 0x55efb857bbc0 2024-09-10T22:51:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f478c044070 con 0x55efb857bbc0 2024-09-10T22:51:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 10 client.25904 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 10 client.25904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 20 client.25904 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb86037c0 0x55efb8605be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 4 client.25904 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f478c036040 con 0x55efb86037c0 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.577+0000 7f4792ffd640 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb86037c0 0x55efb8605be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.578+0000 7f4792ffd640 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb86037c0 0x55efb8605be0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=956980462d258caf server_cookie=e2a712b302e0bb5e in_seq=0 out_seq=0 2024-09-10T22:51:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.578+0000 7f4776ff5640 10 client.25904 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.604+0000 7f4776ff5640 1 -- 172.21.15.96:0/2323241278 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f478c036040 con 0x55efb86037c0 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.604+0000 7f4776ff5640 10 client.25904 handle_command_reply: tid=0 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.604+0000 7f479c47d740 1 client.25904 shutdown 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 msgr2=0x55efb858e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f478c010010 tx=0x7f478c0093c0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 shutdown_connections 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55efb86037c0 0x55efb8605be0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efb857bbc0 0x55efb858e3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efb857b1b0 0x55efb858de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 --2- 172.21.15.96:0/2323241278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55efb8578860 0x55efb8592110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 >> 172.21.15.96:0/2323241278 conn(0x55efb85739b0 msgr2=0x55efb85896b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f4776ff5640 0 client.25904 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 shutdown_connections 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 1 -- 172.21.15.96:0/2323241278 wait complete. 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.605+0000 7f479c47d740 20 client.25904 trim_cache size 0 max 16384 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:51:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 msgr2=0x7f477c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 0x7f477c0b78a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f478c011f40 tx=0x7f478c01e000 comp rx=0 tx=0).stop 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 msgr2=0x7f47941991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f47941991a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f478802a910 tx=0x7f4788003f80 comp rx=0 tx=0).stop 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 shutdown_connections 2024-09-10T22:51:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47941029c0 0x7f47941991a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f477c0b53e0 0x7f477c0b78a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4794103380 0x7f4794199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 --2- 172.21.15.96:0/1189359659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4794101870 0x7f4794198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 >> 172.21.15.96:0/1189359659 conn(0x7f47940fd130 msgr2=0x7f479406b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 shutdown_connections 2024-09-10T22:51:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:46.606+0000 7f47995d3640 1 -- 172.21.15.96:0/1189359659 wait complete. 2024-09-10T22:51:47.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:46 smithi096 ceph-mon[30869]: pgmap v481: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-09-10T22:51:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:46 smithi121 ceph-mon[35119]: pgmap v481: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-09-10T22:51:47.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:46 smithi117 ceph-mon[35081]: pgmap v481: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 11 MiB/s wr, 896 op/s 2024-09-10T22:51:49.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:48 smithi096 ceph-mon[30869]: pgmap v482: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-09-10T22:51:49.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:48 smithi121 ceph-mon[35119]: pgmap v482: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-09-10T22:51:49.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:48 smithi117 ceph-mon[35081]: pgmap v482: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 9.6 MiB/s wr, 749 op/s 2024-09-10T22:51:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 255.0 s, 2937.8 tps, lat 34.047 ms stddev 38.314 2024-09-10T22:51:51.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:50 smithi096 ceph-mon[30869]: pgmap v483: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T22:51:51.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:50 smithi121 ceph-mon[35119]: pgmap v483: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T22:51:51.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:50 smithi117 ceph-mon[35081]: pgmap v483: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T22:51:53.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:52 smithi096 ceph-mon[30869]: pgmap v484: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-10T22:51:53.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:52 smithi121 ceph-mon[35119]: pgmap v484: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-10T22:51:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:52 smithi117 ceph-mon[35081]: pgmap v484: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 752 op/s 2024-09-10T22:51:55.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:54 smithi096 ceph-mon[30869]: pgmap v485: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 818 op/s 2024-09-10T22:51:55.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:54 smithi121 ceph-mon[35119]: pgmap v485: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 818 op/s 2024-09-10T22:51:55.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:54 smithi117 ceph-mon[35081]: pgmap v485: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.9 MiB/s wr, 818 op/s 2024-09-10T22:51:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 260.0 s, 3029.8 tps, lat 33.174 ms stddev 35.288 2024-09-10T22:51:57.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:56 smithi096 ceph-mon[30869]: pgmap v486: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 744 op/s 2024-09-10T22:51:57.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:56 smithi121 ceph-mon[35119]: pgmap v486: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 744 op/s 2024-09-10T22:51:57.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:56 smithi117 ceph-mon[35081]: pgmap v486: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 744 op/s 2024-09-10T22:51:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:51:58 smithi096 ceph-mon[30869]: pgmap v487: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-09-10T22:51:59.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:51:58 smithi121 ceph-mon[35119]: pgmap v487: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-09-10T22:51:59.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:51:58 smithi117 ceph-mon[35081]: pgmap v487: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 -- 172.21.15.96:0/2237304883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a241029b0 msgr2=0x7f4a24102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 --2- 172.21.15.96:0/2237304883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a241029b0 0x7f4a24102e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009930 tx=0x7f4a1802a420 comp rx=0 tx=0).stop 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 -- 172.21.15.96:0/2237304883 shutdown_connections 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 --2- 172.21.15.96:0/2237304883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 --2- 172.21.15.96:0/2237304883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a241029b0 0x7f4a24102e30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 --2- 172.21.15.96:0/2237304883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a24101860 0x7f4a24101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 -- 172.21.15.96:0/2237304883 >> 172.21.15.96:0/2237304883 conn(0x7f4a240fd140 msgr2=0x7f4a240ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 -- 172.21.15.96:0/2237304883 shutdown_connections 2024-09-10T22:51:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.742+0000 7f4a28816640 1 -- 172.21.15.96:0/2237304883 wait complete. 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 Processor -- start 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 -- start start 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 0x7f4a24197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 0x7f4a241982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2410d500 con 0x7f4a24101860 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2410d380 con 0x7f4a24103370 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a28816640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2410d680 con 0x7f4a241029b0 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 0x7f4a24197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 0x7f4a24197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52722/0 (socket says 172.21.15.96:52722) 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a2259c640 1 -- 172.21.15.96:0/884082296 learned_addr learned my addr 172.21.15.96:0/884082296 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.743+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 0x7f4a241982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 -- 172.21.15.96:0/884082296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 msgr2=0x7f4a241982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 0x7f4a241982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 -- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 msgr2=0x7f4a24197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 0x7f4a24197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 -- 172.21.15.96:0/884082296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a241997d0 con 0x7f4a24103370 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.744+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24198f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00eb00 tx=0x7f4a1c009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.745+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 0x7f4a241982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:51:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.745+0000 7f4a137fe640 1 -- 172.21.15.96:0/884082296 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00baf0 con 0x7f4a24103370 2024-09-10T22:51:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.745+0000 7f4a137fe640 1 -- 172.21.15.96:0/884082296 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c00bc90 con 0x7f4a24103370 2024-09-10T22:51:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.745+0000 7f4a137fe640 1 -- 172.21.15.96:0/884082296 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c017410 con 0x7f4a24103370 2024-09-10T22:51:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.746+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a24199ac0 con 0x7f4a24103370 2024-09-10T22:51:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.746+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2406aa10 con 0x7f4a24103370 2024-09-10T22:51:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.749+0000 7f4a137fe640 1 -- 172.21.15.96:0/884082296 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4a1c004e70 con 0x7f4a24103370 2024-09-10T22:51:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.749+0000 7f4a2b5e7740 1 Processor -- start 2024-09-10T22:51:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 -- start start 2024-09-10T22:51:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a137fe640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 0x7f4a0c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 0x5625f99d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 0x5625f99dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5625f99496f0 con 0x5625f99d7860 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5625f9938b20 con 0x5625f99dcbd0 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5625f993c930 con 0x5625f99da190 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a137fe640 1 -- 172.21.15.96:0/884082296 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4a1c0e7290 con 0x7f4a24103370 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 0x5625f99d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 0x7f4a0c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56560/0 (socket says 172.21.15.96:56560) 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 0x5625f99d9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52726/0 (socket says 172.21.15.96:52726) 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a22d9d640 1 -- 172.21.15.96:0/4121485350 learned_addr learned my addr 172.21.15.96:0/4121485350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.750+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 0x5625f99dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 -- 172.21.15.96:0/4121485350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 msgr2=0x5625f99df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 -- 172.21.15.96:0/4121485350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 msgr2=0x5625f99d9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 0x5625f99d9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 -- 172.21.15.96:0/4121485350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f98fe530 con 0x5625f99da190 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 0x5625f99dc580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802a940 tx=0x7f4a18003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d7d62442f8ec1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 0x7f4a0c0b78f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005710 tx=0x7f4a18041000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a10ff9640 1 -- 172.21.15.96:0/4121485350 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18035040 con 0x5625f99da190 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a10ff9640 1 -- 172.21.15.96:0/4121485350 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1804d070 con 0x5625f99da190 2024-09-10T22:51:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.751+0000 7f4a10ff9640 1 -- 172.21.15.96:0/4121485350 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1803d6e0 con 0x5625f99da190 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.752+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/4121485350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 msgr2=0x5625f99dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.752+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 0x5625f99dc580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1802a940 tx=0x7f4a18003920 comp rx=0 tx=0).stop 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.752+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/4121485350 shutdown_connections 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.752+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99da190 0x5625f99dc580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/4121485350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99d7860 0x5625f99d9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/4121485350 >> 172.21.15.96:0/4121485350 conn(0x5625f99d29b0 msgr2=0x5625f99d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/4121485350 shutdown_connections 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/4121485350 wait complete. 2024-09-10T22:51:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.753+0000 7f4a2b5e7740 1 -- start start 2024-09-10T22:51:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 Processor -- start 2024-09-10T22:51:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45074/0 (socket says 172.21.15.96:45074) 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a21d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2259c640 1 -- 172.21.15.96:0/1164029177 learned_addr learned my addr 172.21.15.96:0/1164029177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5625f9870ab0 con 0x5625f99da190 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5625f98de730 con 0x5625f99dcbd0 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.754+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5625f99dfb50 con 0x5625f99d7860 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2b5e7740 10 client.0 _ll_register_callbacks cb 0x5625f99326b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 -- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 msgr2=0x5625f99ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 -- 172.21.15.96:0/1164029177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 msgr2=0x5625f99e9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99e9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f969a790 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a14012d20 tx=0x7f4a1400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5564aa7e32596242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400fe20 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1804d070 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4a14010430 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a14010630 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2b5e7740 10 client.25966 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.755+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625f9713620 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1804d070 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1401c830 con 0x5625f99d7860 2024-09-10T22:51:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 10 client.25966 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 10 client.25966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 20 client.25966 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f99f5030 0x5625f99f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 4 client.25966 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f994b640 con 0x5625f99f5030 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.756+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f99f5030 0x5625f99f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.757+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f99f5030 0x5625f99f7460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a862a378a62ca43e server_cookie=1d58df5cf5fd87a3 in_seq=0 out_seq=0 2024-09-10T22:51:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.757+0000 7f4a09ffb640 10 client.25966 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.757+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1164029177 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625f994b640 con 0x5625f99f5030 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.757+0000 7f4a09ffb640 10 client.25966 handle_command_reply: tid=0 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.757+0000 7f4a2b5e7740 1 client.25966 shutdown 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 msgr2=0x5625f99e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a14012d20 tx=0x7f4a1400f790 comp rx=0 tx=0).stop 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 shutdown_connections 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f99f5030 0x5625f99f7460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99e9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a09ffb640 0 client.25966 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1164029177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99e9710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 >> 172.21.15.96:0/1164029177 conn(0x5625f99d29b0 msgr2=0x5625f99d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 shutdown_connections 2024-09-10T22:51:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1164029177 wait complete. 2024-09-10T22:51:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.758+0000 7f4a2b5e7740 20 client.25966 trim_cache size 0 max 16384 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.779+0000 7f4a2b5e7740 1 Processor -- start 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.779+0000 7f4a2b5e7740 1 -- start start 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.779+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.779+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.779+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a7b70 con 0x5625f99da190 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a96b0 con 0x5625f99dcbd0 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a8240 con 0x5625f99d7860 2024-09-10T22:51:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a21d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56574/0 (socket says 172.21.15.96:56574) 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a21d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52750/0 (socket says 172.21.15.96:52750) 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 -- 172.21.15.96:0/2508433323 learned_addr learned my addr 172.21.15.96:0/2508433323 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 -- 172.21.15.96:0/2508433323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 msgr2=0x5625f99cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 -- 172.21.15.96:0/2508433323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 msgr2=0x5625f99d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 -- 172.21.15.96:0/2508433323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f99e9b40 con 0x5625f99dcbd0 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0e4e80 tx=0x7f4a1c0ad600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5280519e02dccce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a0bfff640 1 -- 172.21.15.96:0/2508433323 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0adaf0 con 0x5625f99dcbd0 2024-09-10T22:51:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a0bfff640 1 -- 172.21.15.96:0/2508433323 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c0b25d0 con 0x5625f99dcbd0 2024-09-10T22:51:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.780+0000 7f4a0bfff640 1 -- 172.21.15.96:0/2508433323 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0b6bf0 con 0x5625f99dcbd0 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/2508433323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 msgr2=0x5625f99d7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0e4e80 tx=0x7f4a1c0ad600 comp rx=0 tx=0).stop 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/2508433323 shutdown_connections 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99dcbd0 0x5625f99d7320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/2508433323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99d7860 0x5625f99cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/2508433323 >> 172.21.15.96:0/2508433323 conn(0x5625f99d29b0 msgr2=0x5625f99d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/2508433323 shutdown_connections 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/2508433323 wait complete. 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 -- start start 2024-09-10T22:51:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 Processor -- start 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.782+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 0x5625f99f0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a8fe0 con 0x5625f99da190 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a8910 con 0x5625f99d7860 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a241a5b70 con 0x5625f99dcbd0 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56584/0 (socket says 172.21.15.96:56584) 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 0x5625f99f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 -- 172.21.15.96:0/1972494459 learned_addr learned my addr 172.21.15.96:0/1972494459 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 10 client.0 _ll_register_callbacks cb 0x7f4a14009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 -- 172.21.15.96:0/1972494459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 msgr2=0x5625f99f0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 0x5625f99f0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 -- 172.21.15.96:0/1972494459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 msgr2=0x5625f99f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 -- 172.21.15.96:0/1972494459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f99d9fc0 con 0x5625f99d7860 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a22d9d640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 0x5625f99f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2259c640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401ce40 tx=0x7f4a1401ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a1964cc8a240c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14010430 con 0x5625f99d7860 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4a14018070 con 0x5625f99d7860 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.783+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a1c0b25d0 con 0x5625f99d7860 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a14037050 con 0x5625f99d7860 2024-09-10T22:51:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 10 client.25922 fetch_fsmap learned FSMap version 32 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625f99eacf0 con 0x5625f99d7860 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a1c0b25d0 con 0x5625f99d7860 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4a1402f020 con 0x5625f99d7860 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 10 client.25922 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 20 client.25922 populate_metadata read hostname 'smithi096' 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f9a622b0 0x5625f9a646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 4 client.25922 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.784+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625f99470f0 con 0x5625f9a622b0 2024-09-10T22:51:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.785+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f9a622b0 0x5625f9a646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:51:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.787+0000 7f4a21d9b640 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f9a622b0 0x5625f9a646a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed697a2c1a059a61 server_cookie=686b6c75278eec2c in_seq=0 out_seq=0 2024-09-10T22:51:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.787+0000 7f4a09ffb640 10 client.25922 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.801+0000 7f4a09ffb640 1 -- 172.21.15.96:0/1972494459 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625f99470f0 con 0x5625f9a622b0 2024-09-10T22:51:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.801+0000 7f4a09ffb640 10 client.25922 handle_command_reply: tid=0 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.801+0000 7f4a2b5e7740 1 client.25922 shutdown 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 msgr2=0x5625f99f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401ce40 tx=0x7f4a1401ded0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 shutdown_connections 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5625f9a622b0 0x5625f9a646a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5625f99dcbd0 0x5625f99f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625f99da190 0x5625f99f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a09ffb640 0 client.25922 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 --2- 172.21.15.96:0/1972494459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625f99d7860 0x5625f99f1250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 >> 172.21.15.96:0/1972494459 conn(0x5625f99d29b0 msgr2=0x5625f99d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 shutdown_connections 2024-09-10T22:51:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 1 -- 172.21.15.96:0/1972494459 wait complete. 2024-09-10T22:51:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.802+0000 7f4a2b5e7740 20 client.25922 trim_cache size 0 max 16384 2024-09-10T22:51:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:51:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:51:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:51:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:51:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 msgr2=0x7f4a0c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 0x7f4a0c0b78f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4a18005710 tx=0x7f4a18041000 comp rx=0 tx=0).stop 2024-09-10T22:51:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 msgr2=0x7f4a24198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24198f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00eb00 tx=0x7f4a1c009e40 comp rx=0 tx=0).stop 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 shutdown_connections 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a24103370 0x7f4a24198f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4a0c0b5430 0x7f4a0c0b78f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a241029b0 0x7f4a241982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 --2- 172.21.15.96:0/884082296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a24101860 0x7f4a24197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 >> 172.21.15.96:0/884082296 conn(0x7f4a240fd140 msgr2=0x7f4a24105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 shutdown_connections 2024-09-10T22:51:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:51:59.803+0000 7f4a28816640 1 -- 172.21.15.96:0/884082296 wait complete. 2024-09-10T22:52:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 265.0 s, 2657.2 tps, lat 37.416 ms stddev 43.581 2024-09-10T22:52:01.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:00 smithi096 ceph-mon[30869]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 861 op/s 2024-09-10T22:52:01.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:00 smithi121 ceph-mon[35119]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 861 op/s 2024-09-10T22:52:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:00 smithi117 ceph-mon[35081]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 861 op/s 2024-09-10T22:52:03.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:02 smithi096 ceph-mon[30869]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-09-10T22:52:03.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:02 smithi121 ceph-mon[35119]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-09-10T22:52:03.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:02 smithi117 ceph-mon[35081]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 714 op/s 2024-09-10T22:52:05.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:04 smithi096 ceph-mon[30869]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 708 op/s 2024-09-10T22:52:05.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:04 smithi121 ceph-mon[35119]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 708 op/s 2024-09-10T22:52:05.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:04 smithi117 ceph-mon[35081]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 708 op/s 2024-09-10T22:52:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 270.0 s, 2926.2 tps, lat 34.342 ms stddev 37.068 2024-09-10T22:52:06.168 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:06.552 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:07.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:06 smithi096 ceph-mon[30869]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.148+0000 7fca64c98640 1 -- 172.21.15.96:0/3317676652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca601076a0 msgr2=0x7fca60107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.148+0000 7fca64c98640 1 --2- 172.21.15.96:0/3317676652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca601076a0 0x7fca60107ae0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fca54009900 tx=0x7fca5402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.149+0000 7fca64c98640 1 -- 172.21.15.96:0/3317676652 shutdown_connections 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.149+0000 7fca64c98640 1 --2- 172.21.15.96:0/3317676652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca60108020 0x7fca6010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.149+0000 7fca64c98640 1 --2- 172.21.15.96:0/3317676652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca601076a0 0x7fca60107ae0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.149+0000 7fca64c98640 1 --2- 172.21.15.96:0/3317676652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.149+0000 7fca64c98640 1 -- 172.21.15.96:0/3317676652 >> 172.21.15.96:0/3317676652 conn(0x7fca600fd700 msgr2=0x7fca600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.150+0000 7fca64c98640 1 -- 172.21.15.96:0/3317676652 shutdown_connections 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.150+0000 7fca64c98640 1 -- 172.21.15.96:0/3317676652 wait complete. 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.151+0000 7fca64c98640 1 Processor -- start 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.151+0000 7fca64c98640 1 -- start start 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.151+0000 7fca64c98640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.151+0000 7fca64c98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 0x7fca601997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35400/0 (socket says 172.21.15.96:35400) 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca5e59c640 1 -- 172.21.15.96:0/1135066555 learned_addr learned my addr 172.21.15.96:0/1135066555 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 0x7fca6019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112dd0 con 0x7fca60108020 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112c50 con 0x7fca601076a0 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 0x7fca6019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 0x7fca601997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.152+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112f50 con 0x7fca60101c90 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 -- 172.21.15.96:0/1135066555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 msgr2=0x7fca60199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 -- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 msgr2=0x7fca6019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 0x7fca6019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6019f330 con 0x7fca601076a0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 0x7fca6019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 0x7fca601997c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca5402a910 tx=0x7fca54004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca4f7fe640 1 -- 172.21.15.96:0/1135066555 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54048070 con 0x7fca601076a0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6019f5c0 con 0x7fca601076a0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6010eee0 con 0x7fca601076a0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca4f7fe640 1 -- 172.21.15.96:0/1135066555 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5403ab20 con 0x7fca601076a0 2024-09-10T22:52:07.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.153+0000 7fca4f7fe640 1 -- 172.21.15.96:0/1135066555 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54035050 con 0x7fca601076a0 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.156+0000 7fca4f7fe640 1 -- 172.21.15.96:0/1135066555 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fca5403a670 con 0x7fca601076a0 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.156+0000 7fca6706d740 1 Processor -- start 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.157+0000 7fca4f7fe640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 0x7fca480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.158+0000 7fca4f7fe640 1 -- 172.21.15.96:0/1135066555 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca541099b0 con 0x7fca601076a0 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.158+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 0x7fca480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.158+0000 7fca6706d740 1 -- start start 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c51092b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51092b540 0x55c51092d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c51088ce50 con 0x55c51091ec90 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 0x7fca480b77a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca50009fd0 tx=0x7fca500096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38610/0 (socket says 172.21.15.96:38610) 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 -- 172.21.15.96:0/2311156321 learned_addr learned my addr 172.21.15.96:0/2311156321 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c51092b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51092b540 0x55c51092d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c51087c280 con 0x55c51092b540 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c510880090 con 0x55c51091f5b0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 -- 172.21.15.96:0/2311156321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 msgr2=0x55c51092b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c51092b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 -- 172.21.15.96:0/2311156321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51092b540 msgr2=0x55c51092d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51092b540 0x55c51092d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.159+0000 7fca5e59c640 1 -- 172.21.15.96:0/2311156321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5105da650 con 0x55c51091ec90 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca5e59c640 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bda0 tx=0x7fca580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1affd9be3a61c9ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca4cff9640 1 -- 172.21.15.96:0/2311156321 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5801c680 con 0x55c51091ec90 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca4cff9640 1 -- 172.21.15.96:0/2311156321 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580118f0 con 0x55c51091ec90 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca4cff9640 1 -- 172.21.15.96:0/2311156321 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5801a800 con 0x55c51091ec90 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 msgr2=0x55c51091f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.160+0000 7fca6706d740 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bda0 tx=0x7fca580107a0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 shutdown_connections 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51092b540 0x55c51092d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c51092b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 --2- 172.21.15.96:0/2311156321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51091f070 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 >> 172.21.15.96:0/2311156321 conn(0x55c51091cac0 msgr2=0x55c51091dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 shutdown_connections 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 -- 172.21.15.96:0/2311156321 wait complete. 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 -- start start 2024-09-10T22:52:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.161+0000 7fca6706d740 1 Processor -- start 2024-09-10T22:52:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca6706d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca6706d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510919b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33224/0 (socket says 172.21.15.96:33224) 2024-09-10T22:52:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca5e59c640 1 -- 172.21.15.96:0/1084541031 learned_addr learned my addr 172.21.15.96:0/1084541031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510919b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.162+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c51091a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c5106dd700 con 0x55c51092b540 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c510817110 con 0x55c51091ec90 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c510913f50 con 0x55c51091f5b0 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c51091a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca6706d740 10 client.0 _ll_register_callbacks cb 0x55c510830990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 -- 172.21.15.96:0/1084541031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 msgr2=0x55c510919b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510919b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 -- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 msgr2=0x55c5109195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.163+0000 7fca5ed9d640 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510844c40 con 0x55c51092b540 2024-09-10T22:52:07.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c51091a060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fca50011010 tx=0x7fca50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fb962e8c958bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50028640 con 0x55c51092b540 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510919b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca500287e0 con 0x55c51092b540 2024-09-10T22:52:07.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca580118f0 con 0x55c51092b540 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca500289e0 con 0x55c51092b540 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca6706d740 10 client.16374 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c5109424e0 con 0x55c51092b540 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.164+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580118f0 con 0x55c51092b540 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca50026020 con 0x55c51092b540 2024-09-10T22:52:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 10 client.16374 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 20 client.16374 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c510943880 0x55c510945c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 4 client.16374 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:07.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c51088eda0 con 0x55c510943880 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.165+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c510943880 0x55c510945c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.166+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c510943880 0x55c510945c70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51b887050c976ec server_cookie=584223395b25fc40 in_seq=0 out_seq=0 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.166+0000 7fca45ffb640 10 client.16374 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca45ffb640 1 -- 172.21.15.96:0/1084541031 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c51088eda0 con 0x55c510943880 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca45ffb640 10 client.16374 handle_command_reply: tid=0 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 client.16374 shutdown 2024-09-10T22:52:07.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 msgr2=0x55c51091a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c51091a060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fca50011010 tx=0x7fca50004ab0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 shutdown_connections 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c510943880 0x55c510945c70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c51091a060 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510919b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 --2- 172.21.15.96:0/1084541031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c5109195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 >> 172.21.15.96:0/1084541031 conn(0x55c51091cac0 msgr2=0x55c5109133d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.167+0000 7fca45ffb640 0 client.16374 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.168+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 shutdown_connections 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.168+0000 7fca6706d740 1 -- 172.21.15.96:0/1084541031 wait complete. 2024-09-10T22:52:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.168+0000 7fca6706d740 20 client.16374 trim_cache size 0 max 16384 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.193+0000 7fca6706d740 1 Processor -- start 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.193+0000 7fca6706d740 1 -- start start 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.193+0000 7fca6706d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.193+0000 7fca6706d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 0x55c51092ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.193+0000 7fca6706d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51092b540 0x55c5109167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca6706d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca601abff0 con 0x55c51091ec90 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca6706d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca601adb30 con 0x55c51091f5b0 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca6706d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ac6c0 con 0x55c51092b540 2024-09-10T22:52:07.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38630/0 (socket says 172.21.15.96:38630) 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 -- 172.21.15.96:0/1923442175 learned_addr learned my addr 172.21.15.96:0/1923442175 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 0x55c51092ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5ed9d640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51092b540 0x55c5109167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 -- 172.21.15.96:0/1923442175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51092b540 msgr2=0x55c5109167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51092b540 0x55c5109167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 -- 172.21.15.96:0/1923442175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 msgr2=0x55c51092ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 0x55c51092ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 -- 172.21.15.96:0/1923442175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51091f3e0 con 0x55c51091ec90 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 0x55c51092ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca5e59c640 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca5801a3f0 tx=0x7fca58005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36943ff7a396df53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca47fff640 1 -- 172.21.15.96:0/1923442175 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58005240 con 0x55c51091ec90 2024-09-10T22:52:07.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca47fff640 1 -- 172.21.15.96:0/1923442175 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580053e0 con 0x55c51091ec90 2024-09-10T22:52:07.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.194+0000 7fca47fff640 1 -- 172.21.15.96:0/1923442175 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58011840 con 0x55c51091ec90 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.195+0000 7fca6706d740 1 -- 172.21.15.96:0/1923442175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 msgr2=0x55c51092c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.195+0000 7fca6706d740 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca5801a3f0 tx=0x7fca58005f30 comp rx=0 tx=0).stop 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.196+0000 7fca6706d740 1 -- 172.21.15.96:0/1923442175 shutdown_connections 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.196+0000 7fca6706d740 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091f5b0 0x55c51092ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.196+0000 7fca6706d740 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51092b540 0x55c5109167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.196+0000 7fca6706d740 1 --2- 172.21.15.96:0/1923442175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51091ec90 0x55c51092c930 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.196+0000 7fca6706d740 1 -- 172.21.15.96:0/1923442175 >> 172.21.15.96:0/1923442175 conn(0x55c51091cac0 msgr2=0x55c5109133d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.197+0000 7fca6706d740 1 -- 172.21.15.96:0/1923442175 shutdown_connections 2024-09-10T22:52:07.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.197+0000 7fca6706d740 1 -- 172.21.15.96:0/1923442175 wait complete. 2024-09-10T22:52:07.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.197+0000 7fca6706d740 1 -- start start 2024-09-10T22:52:07.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.197+0000 7fca6706d740 1 Processor -- start 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c510928290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510928810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c510922fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ad460 con 0x55c51092b540 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca601acd90 con 0x55c51091ec90 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca601a9ff0 con 0x55c51091f5b0 2024-09-10T22:52:07.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca6706d740 10 client.0 _ll_register_callbacks cb 0x7fca5000e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca5dd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510928810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca5dd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510928810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35442/0 (socket says 172.21.15.96:35442) 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca5dd9b640 1 -- 172.21.15.96:0/65091639 learned_addr learned my addr 172.21.15.96:0/65091639 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.198+0000 7fca5e59c640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c510928290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 -- 172.21.15.96:0/65091639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 msgr2=0x55c510928810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510928810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 -- 172.21.15.96:0/65091639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 msgr2=0x55c510922fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c510922fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 -- 172.21.15.96:0/65091639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510923810 con 0x55c51091ec90 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca5e59c640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c510928290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca58010ca0 tx=0x7fca580052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c11f15c4bb37f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:07.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.199+0000 7fca45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.200+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5800b7e0 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.200+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca58003800 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.200+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca580053e0 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca58005590 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 10 client.25937 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c5109b1420 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580053e0 con 0x55c51091ec90 2024-09-10T22:52:07.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca58043050 con 0x55c51091ec90 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 10 client.25937 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 20 client.25937 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.201+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c5109b18c0 0x55c5109b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.202+0000 7fca6706d740 4 client.25937 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.202+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c51088a850 con 0x55c5109b18c0 2024-09-10T22:52:07.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.203+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c5109b18c0 0x55c5109b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:07.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.204+0000 7fca5dd9b640 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c5109b18c0 0x55c5109b3ce0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb842b27564c7d74 server_cookie=1f497993a4760e2 in_seq=0 out_seq=0 2024-09-10T22:52:07.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.204+0000 7fca45ffb640 10 client.25937 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca45ffb640 1 -- 172.21.15.96:0/65091639 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55c51088a850 con 0x55c5109b18c0 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca45ffb640 10 client.25937 handle_command_reply: tid=0 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca6706d740 1 client.25937 shutdown 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 msgr2=0x55c510928290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c510928290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca58010ca0 tx=0x7fca580052f0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 shutdown_connections 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.206+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c5109b18c0 0x55c5109b3ce0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c51092b540 0x55c510922fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca45ffb640 0 client.25937 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c51091f5b0 0x55c510928810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 --2- 172.21.15.96:0/65091639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c51091ec90 0x55c510928290 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 >> 172.21.15.96:0/65091639 conn(0x55c51091cac0 msgr2=0x55c510921cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 shutdown_connections 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 1 -- 172.21.15.96:0/65091639 wait complete. 2024-09-10T22:52:07.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca6706d740 20 client.25937 trim_cache size 0 max 16384 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:52:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:52:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:07.215 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 msgr2=0x7fca480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 0x7fca480b77a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca50009fd0 tx=0x7fca500096a0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 msgr2=0x7fca601997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 0x7fca601997c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca5402a910 tx=0x7fca54004270 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.207+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 shutdown_connections 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca601076a0 0x7fca601997c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca480b52e0 0x7fca480b77a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca60108020 0x7fca6019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 --2- 172.21.15.96:0/1135066555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca60101c90 0x7fca60199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 >> 172.21.15.96:0/1135066555 conn(0x7fca600fd700 msgr2=0x7fca600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 shutdown_connections 2024-09-10T22:52:07.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:07.208+0000 7fca64c98640 1 -- 172.21.15.96:0/1135066555 wait complete. 2024-09-10T22:52:07.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:06 smithi121 ceph-mon[35119]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-09-10T22:52:07.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:06 smithi117 ceph-mon[35081]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-09-10T22:52:08.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:52:08.391 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:08.726 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:09.126 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:08 smithi096 ceph-mon[30869]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 -- 172.21.15.96:0/2242951452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5424108040 msgr2=0x7f542410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 --2- 172.21.15.96:0/2242951452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5424108040 0x7f542410e840 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f541c009900 tx=0x7f541c02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 -- 172.21.15.96:0/2242951452 shutdown_connections 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 --2- 172.21.15.96:0/2242951452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54241076c0 0x7f5424107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 --2- 172.21.15.96:0/2242951452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5424108040 0x7f542410e840 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 --2- 172.21.15.96:0/2242951452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424101cb0 0x7f5424102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 -- 172.21.15.96:0/2242951452 >> 172.21.15.96:0/2242951452 conn(0x7f54240fd700 msgr2=0x7f54240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 -- 172.21.15.96:0/2242951452 shutdown_connections 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.242+0000 7f542910c640 1 -- 172.21.15.96:0/2242951452 wait complete. 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.243+0000 7f542910c640 1 Processor -- start 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.243+0000 7f542910c640 1 -- start start 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.243+0000 7f542910c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 0x7f542410dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.243+0000 7f542910c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 0x7f542410e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.243+0000 7f542910c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542910c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112de0 con 0x7f54241076c0 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542910c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112c60 con 0x7f5424101cb0 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542910c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112f60 con 0x7f5424108040 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35462/0 (socket says 172.21.15.96:35462) 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 -- 172.21.15.96:0/525315433 learned_addr learned my addr 172.21.15.96:0/525315433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f5422d9d640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 0x7f542410dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542259c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 0x7f542410e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 -- 172.21.15.96:0/525315433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 msgr2=0x7f542410dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 0x7f542410dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 -- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 msgr2=0x7f542410e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 0x7f542410e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 -- 172.21.15.96:0/525315433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542419c030 con 0x7f5424108040 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542259c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 0x7f542410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f5422d9d640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 0x7f542410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f542359e640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f541c004770 tx=0x7f541c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.244+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c03f030 con 0x7f5424108040 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.245+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542419c260 con 0x7f5424108040 2024-09-10T22:52:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.245+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c004440 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.245+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c035070 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.245+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5424198790 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.246+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5424102090 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.248+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f541c04c060 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.249+0000 7f541bfff640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 0x7f54080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.249+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f541c106150 con 0x7f5424108040 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.250+0000 7f5422d9d640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 0x7f54080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.250+0000 7f5422d9d640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 0x7f54080b77a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f5414005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.251+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f541c0cd0b0 con 0x7f5424108040 2024-09-10T22:52:09.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:08 smithi121 ceph-mon[35119]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:52:09.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:08 smithi117 ceph-mon[35081]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:52:09.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.563+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5424198c10 con 0x7f5424108040 2024-09-10T22:52:09.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.563+0000 7f541bfff640 1 -- 172.21.15.96:0/525315433 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f541c0cc9b0 con 0x7f5424108040 2024-09-10T22:52:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:52:09.565 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:52:09.566 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:52:09.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 msgr2=0x7f54080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 0x7f54080b77a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f5414005f90 comp rx=0 tx=0).stop 2024-09-10T22:52:09.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 msgr2=0x7f542419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:09.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f541c004770 tx=0x7f541c004040 comp rx=0 tx=0).stop 2024-09-10T22:52:09.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 shutdown_connections 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54080b52e0 0x7f54080b77a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5424108040 0x7f542419b860 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54241076c0 0x7f542410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 --2- 172.21.15.96:0/525315433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5424101cb0 0x7f542410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 >> 172.21.15.96:0/525315433 conn(0x7f54240fd700 msgr2=0x7f542410f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 shutdown_connections 2024-09-10T22:52:09.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:09.566+0000 7f542910c640 1 -- 172.21.15.96:0/525315433 wait complete. 2024-09-10T22:52:10.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:09 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/525315433' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:10.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:09 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/525315433' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:10.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:09 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/525315433' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:10.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:52:10.713 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 275.0 s, 2952.8 tps, lat 33.679 ms stddev 35.794 2024-09-10T22:52:11.168 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:10 smithi096 ceph-mon[30869]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T22:52:11.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:10 smithi121 ceph-mon[35119]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T22:52:11.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:10 smithi117 ceph-mon[35081]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T22:52:11.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.272+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1470655053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010bb80 msgr2=0x7f22b01142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.272+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1470655053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010bb80 0x7f22b01142d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009900 tx=0x7f22a402a480 comp rx=0 tx=0).stop 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1470655053 shutdown_connections 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1470655053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b0116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1470655053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010bb80 0x7f22b01142d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1470655053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010b260 0x7f22b010b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1470655053 >> 172.21.15.96:0/1470655053 conn(0x7f22b006dfc0 msgr2=0x7f22b006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1470655053 shutdown_connections 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.273+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1470655053 wait complete. 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 Processor -- start 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 -- start start 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 0x7f22b01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010bb80 0x7f22b01a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22b011a480 con 0x7f22b010b260 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22b011a300 con 0x7f22b010bb80 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22b4b6b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22b011a600 con 0x7f22b0114810 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 0x7f22b01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 0x7f22b01a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38680/0 (socket says 172.21.15.96:38680) 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22aed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35478/0 (socket says 172.21.15.96:35478) 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22add9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010bb80 0x7f22b01a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.274+0000 7f22ae59c640 1 -- 172.21.15.96:0/1392875128 learned_addr learned my addr 172.21.15.96:0/1392875128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 -- 172.21.15.96:0/1392875128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010bb80 msgr2=0x7f22b01a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010bb80 0x7f22b01a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 -- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 msgr2=0x7f22b01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 0x7f22b01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 -- 172.21.15.96:0/1392875128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b01a7b20 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.275+0000 7f22aed9d640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005fd0 tx=0x7f22a800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.277+0000 7f229f7fe640 1 -- 172.21.15.96:0/1392875128 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a800cdf0 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.277+0000 7f229f7fe640 1 -- 172.21.15.96:0/1392875128 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a8005020 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.278+0000 7f229f7fe640 1 -- 172.21.15.96:0/1392875128 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8010690 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.278+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b01a7e10 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.278+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b00716d0 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.281+0000 7f22b6f40740 1 Processor -- start 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.281+0000 7f229f7fe640 1 -- 172.21.15.96:0/1392875128 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f22a8005460 con 0x7f22b0114810 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.281+0000 7f22b6f40740 1 -- start start 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec26b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 0x5626ec279980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626ec1d8ea0 con 0x5626ec277590 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5626ec1c82d0 con 0x5626ec26b600 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22b6f40740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5626ec1cc0e0 con 0x5626ec26ace0 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33292/0 (socket says 172.21.15.96:33292) 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 -- 172.21.15.96:0/3865559199 learned_addr learned my addr 172.21.15.96:0/3865559199 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f229f7fe640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 0x7f22980b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22add9b640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec26b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22add9b640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 0x5626ec279980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22aed9d640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 0x7f22980b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 -- 172.21.15.96:0/3865559199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 msgr2=0x5626ec26b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec26b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 -- 172.21.15.96:0/3865559199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 msgr2=0x5626ec279980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 0x5626ec279980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22ae59c640 1 -- 172.21.15.96:0/3865559199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ebf26650 con 0x5626ec26b600 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f22add9b640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 0x5626ec279980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.282+0000 7f229f7fe640 1 -- 172.21.15.96:0/1392875128 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f22a80e5cf0 con 0x7f22b0114810 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.283+0000 7f22aed9d640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 0x7f22980b78f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22a80a8b20 tx=0x7f22a8070c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.283+0000 7f22ae59c640 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d450 tx=0x7f22a000d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9bec979924ee6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.283+0000 7f229cff9640 1 -- 172.21.15.96:0/3865559199 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004230 con 0x5626ec26b600 2024-09-10T22:52:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.283+0000 7f229cff9640 1 -- 172.21.15.96:0/3865559199 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a00043d0 con 0x5626ec26b600 2024-09-10T22:52:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.283+0000 7f229cff9640 1 -- 172.21.15.96:0/3865559199 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0018650 con 0x5626ec26b600 2024-09-10T22:52:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.284+0000 7f22b6f40740 1 -- 172.21.15.96:0/3865559199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 msgr2=0x5626ec277050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.284+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d450 tx=0x7f22a000d910 comp rx=0 tx=0).stop 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 -- 172.21.15.96:0/3865559199 shutdown_connections 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec277050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec277590 0x5626ec279980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3865559199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec26b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 -- 172.21.15.96:0/3865559199 >> 172.21.15.96:0/3865559199 conn(0x5626ec268b10 msgr2=0x5626ec269cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 -- 172.21.15.96:0/3865559199 shutdown_connections 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 -- 172.21.15.96:0/3865559199 wait complete. 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.286+0000 7f22b6f40740 1 -- start start 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 Processor -- start 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec274250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec2747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626ec029700 con 0x5626ec26b600 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5626ec163160 con 0x5626ec277590 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5626ec27a240 con 0x5626ec26ace0 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22b6f40740 10 client.0 _ll_register_callbacks cb 0x5626ec17c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35494/0 (socket says 172.21.15.96:35494) 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec274250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 -- 172.21.15.96:0/957552336 learned_addr learned my addr 172.21.15.96:0/957552336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec2747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 -- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 msgr2=0x5626ec2747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec2747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 -- 172.21.15.96:0/957552336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 msgr2=0x5626ec274250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.287+0000 7f22add9b640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec274250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f22add9b640 1 -- 172.21.15.96:0/957552336 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ec190c90 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f22add9b640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a40058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7bbb9db8c8e226 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f2295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a00043d0 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.288+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f22a4004480 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22a40035f0 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 10 client.26005 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626ec285ab0 con 0x5626ec26ace0 2024-09-10T22:52:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a4003c40 con 0x5626ec26ace0 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f22a403a040 con 0x5626ec26ace0 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 10 client.26005 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 10 client.26005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 20 client.26005 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec287250 0x5626ec289670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 4 client.26005 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.289+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22a00043d0 con 0x5626ec287250 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.290+0000 7f22ae59c640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec287250 0x5626ec289670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.290+0000 7f2295ffb640 10 client.26005 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:52:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.290+0000 7f22ae59c640 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec287250 0x5626ec289670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac559cde2007b022 server_cookie=96dd81e8f5842ea5 in_seq=0 out_seq=0 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.303+0000 7f2295ffb640 1 -- 172.21.15.96:0/957552336 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f22a00043d0 con 0x5626ec287250 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.303+0000 7f2295ffb640 10 client.26005 handle_command_reply: tid=0 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.303+0000 7f22b6f40740 1 client.26005 shutdown 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.304+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 msgr2=0x5626ec276e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.304+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a40058b0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 shutdown_connections 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec287250 0x5626ec289670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec2747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec274250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 --2- 172.21.15.96:0/957552336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec276e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 >> 172.21.15.96:0/957552336 conn(0x5626ec268b10 msgr2=0x5626ec25f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.307+0000 7f2295ffb640 0 client.26005 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:52:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.308+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 shutdown_connections 2024-09-10T22:52:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.308+0000 7f22b6f40740 1 -- 172.21.15.96:0/957552336 wait complete. 2024-09-10T22:52:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.309+0000 7f22b6f40740 20 client.26005 trim_cache size 0 max 16384 2024-09-10T22:52:11.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.334+0000 7f22b6f40740 1 Processor -- start 2024-09-10T22:52:11.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.334+0000 7f22b6f40740 1 -- start start 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec283470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec269b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec26f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b4790 con 0x5626ec26b600 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b62d0 con 0x5626ec277590 2024-09-10T22:52:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.335+0000 7f22b6f40740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b4e60 con 0x5626ec26ace0 2024-09-10T22:52:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.339+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec269b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.339+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec269b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38706/0 (socket says 172.21.15.96:38706) 2024-09-10T22:52:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.339+0000 7f22ae59c640 1 -- 172.21.15.96:0/3974296608 learned_addr learned my addr 172.21.15.96:0/3974296608 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.339+0000 7f22add9b640 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec283470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.339+0000 7f22add9b640 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec26f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 -- 172.21.15.96:0/3974296608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 msgr2=0x5626ec26f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec26f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 -- 172.21.15.96:0/3974296608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 msgr2=0x5626ec269b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec269b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 -- 172.21.15.96:0/3974296608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ec274b40 con 0x5626ec26ace0 2024-09-10T22:52:11.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.340+0000 7f22add9b640 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec283470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ac10e0c4631fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.341+0000 7f2297fff640 1 -- 172.21.15.96:0/3974296608 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035040 con 0x5626ec26ace0 2024-09-10T22:52:11.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.341+0000 7f2297fff640 1 -- 172.21.15.96:0/3974296608 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a405c070 con 0x5626ec26ace0 2024-09-10T22:52:11.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.341+0000 7f2297fff640 1 -- 172.21.15.96:0/3974296608 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4031d70 con 0x5626ec26ace0 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 -- 172.21.15.96:0/3974296608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 msgr2=0x5626ec283470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec283470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4005b50 comp rx=0 tx=0).stop 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 -- 172.21.15.96:0/3974296608 shutdown_connections 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec277590 0x5626ec26f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec26b600 0x5626ec269b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 --2- 172.21.15.96:0/3974296608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec26ace0 0x5626ec283470 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005ab0 tx=0x7f22a4005b50 comp rx=0 tx=0).stop 2024-09-10T22:52:11.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.342+0000 7f22b6f40740 1 -- 172.21.15.96:0/3974296608 >> 172.21.15.96:0/3974296608 conn(0x5626ec268b10 msgr2=0x5626ec25b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.343+0000 7f22b6f40740 1 -- 172.21.15.96:0/3974296608 shutdown_connections 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.343+0000 7f22b6f40740 1 -- 172.21.15.96:0/3974296608 wait complete. 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.343+0000 7f22b6f40740 1 -- start start 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 Processor -- start 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec277590 0x5626ec27b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec27b5a0 0x5626ec27ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b5c00 con 0x5626ec27b5a0 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b5530 con 0x5626ec26b600 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22b01b27b0 con 0x5626ec277590 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22b6f40740 10 client.0 _ll_register_callbacks cb 0x7f22a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22add9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22add9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33304/0 (socket says 172.21.15.96:33304) 2024-09-10T22:52:11.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22add9b640 1 -- 172.21.15.96:0/230871643 learned_addr learned my addr 172.21.15.96:0/230871643 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.344+0000 7f22add9b640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec27b5a0 0x5626ec27ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.345+0000 7f22add9b640 1 -- 172.21.15.96:0/230871643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec277590 msgr2=0x5626ec27b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.345+0000 7f22add9b640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec277590 0x5626ec27b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.345+0000 7f22add9b640 1 -- 172.21.15.96:0/230871643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec27b5a0 msgr2=0x5626ec27ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.345+0000 7f22add9b640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec27b5a0 0x5626ec27ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.345+0000 7f22add9b640 1 -- 172.21.15.96:0/230871643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ec26b430 con 0x5626ec26b600 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.346+0000 7f22add9b640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22a403dca0 tx=0x7f22a403dba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39e044767e0eaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.346+0000 7f2295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.346+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4035070 con 0x5626ec26b600 2024-09-10T22:52:11.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.346+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f22a4005030 con 0x5626ec26b600 2024-09-10T22:52:11.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.347+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a405c070 con 0x5626ec26b600 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.347+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22a4005230 con 0x5626ec26b600 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.347+0000 7f22b6f40740 10 client.25961 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.347+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626ec2f4360 con 0x5626ec26b600 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.347+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a405c070 con 0x5626ec26b600 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f22a40046c0 con 0x5626ec26b600 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 10 client.25961 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 10 client.25961 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 20 client.25961 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec2f4950 0x5626ec2f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 4 client.25961 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626ec1dadf0 con 0x5626ec2f4950 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.348+0000 7f22ae59c640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec2f4950 0x5626ec2f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f22ae59c640 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec2f4950 0x5626ec2f6d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c10234bdebc8398 server_cookie=f71c254c553cbd39 in_seq=0 out_seq=0 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f2295ffb640 10 client.25961 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f2295ffb640 1 -- 172.21.15.96:0/230871643 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626ec1dadf0 con 0x5626ec2f4950 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f2295ffb640 10 client.25961 handle_command_reply: tid=0 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f22b6f40740 1 client.25961 shutdown 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 msgr2=0x5626ec27fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.349+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22a403dca0 tx=0x7f22a403dba0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 shutdown_connections 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5626ec2f4950 0x5626ec2f6d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626ec27b5a0 0x5626ec27ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626ec277590 0x5626ec27b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f2295ffb640 0 client.25961 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 --2- 172.21.15.96:0/230871643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ec26b600 0x5626ec27fbe0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 >> 172.21.15.96:0/230871643 conn(0x5626ec268b10 msgr2=0x5626ec269b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 shutdown_connections 2024-09-10T22:52:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 1 -- 172.21.15.96:0/230871643 wait complete. 2024-09-10T22:52:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.350+0000 7f22b6f40740 20 client.25961 trim_cache size 0 max 16384 2024-09-10T22:52:11.354 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:52:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 msgr2=0x7f22980b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 0x7f22980b78f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22a80a8b20 tx=0x7f22a8070c90 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 msgr2=0x7f22b01a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005fd0 tx=0x7f22a800ef10 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 shutdown_connections 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b010bb80 0x7f22b01a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f22980b5430 0x7f22980b78f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22b0114810 0x7f22b01a7350 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 --2- 172.21.15.96:0/1392875128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22b010b260 0x7f22b01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 >> 172.21.15.96:0/1392875128 conn(0x7f22b006dfc0 msgr2=0x7f22b010a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 shutdown_connections 2024-09-10T22:52:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:11.351+0000 7f22b4b6b640 1 -- 172.21.15.96:0/1392875128 wait complete. 2024-09-10T22:52:12.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:52:12.539 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 -- 172.21.15.96:0/1076955297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 msgr2=0x7f6b44111420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 --2- 172.21.15.96:0/1076955297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b44111420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009cd0 tx=0x7f6b3c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 -- 172.21.15.96:0/1076955297 shutdown_connections 2024-09-10T22:52:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 --2- 172.21.15.96:0/1076955297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b44111420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 --2- 172.21.15.96:0/1076955297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b4410a100 0x7f6b4410a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 --2- 172.21.15.96:0/1076955297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b44109230 0x7f6b44109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.925+0000 7f6b499d3640 1 -- 172.21.15.96:0/1076955297 >> 172.21.15.96:0/1076955297 conn(0x7f6b440783d0 msgr2=0x7f6b4407a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.926+0000 7f6b499d3640 1 -- 172.21.15.96:0/1076955297 shutdown_connections 2024-09-10T22:52:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.926+0000 7f6b499d3640 1 -- 172.21.15.96:0/1076955297 wait complete. 2024-09-10T22:52:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.926+0000 7f6b499d3640 1 Processor -- start 2024-09-10T22:52:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 -- start start 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b44109230 0x7f6b4419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 0x7f6b4419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44114be0 con 0x7f6b4410aac0 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44114a60 con 0x7f6b4410a100 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b499d3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b44114d60 con 0x7f6b44109230 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38736/0 (socket says 172.21.15.96:38736) 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b437fe640 1 -- 172.21.15.96:0/2582325020 learned_addr learned my addr 172.21.15.96:0/2582325020 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b427fc640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 0x7f6b4419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b44109230 0x7f6b4419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b437fe640 1 -- 172.21.15.96:0/2582325020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b44109230 msgr2=0x7f6b4419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.927+0000 7f6b437fe640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b44109230 0x7f6b4419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b437fe640 1 -- 172.21.15.96:0/2582325020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 msgr2=0x7f6b4419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b437fe640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 0x7f6b4419f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b437fe640 1 -- 172.21.15.96:0/2582325020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b441a0e50 con 0x7f6b4410aac0 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b427fc640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 0x7f6b4419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b437fe640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c033040 tx=0x7f6b3c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b489d1640 1 -- 172.21.15.96:0/2582325020 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004240 con 0x7f6b4410aac0 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b489d1640 1 -- 172.21.15.96:0/2582325020 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3c0305a0 con 0x7f6b4410aac0 2024-09-10T22:52:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b489d1640 1 -- 172.21.15.96:0/2582325020 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c0466e0 con 0x7f6b4410aac0 2024-09-10T22:52:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.928+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b441a10e0 con 0x7f6b4410aac0 2024-09-10T22:52:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.929+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b441a7630 con 0x7f6b4410aac0 2024-09-10T22:52:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.931+0000 7f6b489d1640 1 -- 172.21.15.96:0/2582325020 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f6b3c04e070 con 0x7f6b4410aac0 2024-09-10T22:52:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 Processor -- start 2024-09-10T22:52:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 -- start start 2024-09-10T22:52:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f0bbd0 0x557340f0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557340e786f0 con 0x557340f0bbd0 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557340e67b20 con 0x557340f09190 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b4c881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557340e6b930 con 0x557340f06860 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b489d1640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 0x7f6b2c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33312/0 (socket says 172.21.15.96:33312) 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b42ffd640 1 -- 172.21.15.96:0/3673263649 learned_addr learned my addr 172.21.15.96:0/3673263649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f0bbd0 0x557340f0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b489d1640 1 -- 172.21.15.96:0/2582325020 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6b3c02e030 con 0x7f6b4410aac0 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.932+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 -- 172.21.15.96:0/3673263649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 msgr2=0x557340f08c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f08c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 -- 172.21.15.96:0/3673263649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f0bbd0 msgr2=0x557340f0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f0bbd0 0x557340f0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b437fe640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 0x7f6b2c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 -- 172.21.15.96:0/3673263649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557340e2d530 con 0x557340f09190 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400bd10 tx=0x7f6b340107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72552ff4242b760d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b29ffb640 1 -- 172.21.15.96:0/3673263649 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34004030 con 0x557340f09190 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b29ffb640 1 -- 172.21.15.96:0/3673263649 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b340035f0 con 0x557340f09190 2024-09-10T22:52:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.933+0000 7f6b29ffb640 1 -- 172.21.15.96:0/3673263649 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34011770 con 0x557340f09190 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- 172.21.15.96:0/3673263649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 msgr2=0x557340f0b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400bd10 tx=0x7f6b340107a0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b437fe640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 0x7f6b2c0b78f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009ca0 tx=0x7f6b3c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- 172.21.15.96:0/3673263649 shutdown_connections 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f09190 0x557340f0b580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f0bbd0 0x557340f0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3673263649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- 172.21.15.96:0/3673263649 >> 172.21.15.96:0/3673263649 conn(0x557340f019b0 msgr2=0x557340f02740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- 172.21.15.96:0/3673263649 shutdown_connections 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- 172.21.15.96:0/3673263649 wait complete. 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.934+0000 7f6b4c881740 1 -- start start 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 Processor -- start 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 0x557340f10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f11160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557340d9fab0 con 0x557340f06860 2024-09-10T22:52:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557340e0d730 con 0x557340f0bbd0 2024-09-10T22:52:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557340f0ebb0 con 0x557340f09190 2024-09-10T22:52:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b4c881740 10 client.0 _ll_register_callbacks cb 0x557340e616b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b427fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b427fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38750/0 (socket says 172.21.15.96:38750) 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b427fc640 1 -- 172.21.15.96:0/3179766212 learned_addr learned my addr 172.21.15.96:0/3179766212 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.935+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 0x557340f10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f11160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 -- 172.21.15.96:0/3179766212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 msgr2=0x557340f10590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 0x557340f10590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 -- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 msgr2=0x557340f11160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f11160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 -- 172.21.15.96:0/3179766212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557340bc9790 con 0x557340f06860 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 0x557340f10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b38010e10 tx=0x7f6b38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc7741ea2174363 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38005020 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6b380051c0 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b340035f0 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b38027050 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b4c881740 10 client.16410 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.936+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557340c42620 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b340035f0 con 0x557340f06860 2024-09-10T22:52:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6b3801e070 con 0x557340f06860 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 20 client.16410 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f1b6b0 0x557340f1db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 4 client.16410 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557340e7a640 con 0x557340f1b6b0 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.937+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f1b6b0 0x557340f1db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.938+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f1b6b0 0x557340f1db20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14f97f6fbc91c0f server_cookie=6e646321c7c0c74f in_seq=0 out_seq=0 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.938+0000 7f6b22ffd640 10 client.16410 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b22ffd640 1 -- 172.21.15.96:0/3179766212 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557340e7a640 con 0x557340f1b6b0 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b22ffd640 10 client.16410 handle_command_reply: tid=0 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 client.16410 shutdown 2024-09-10T22:52:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 msgr2=0x557340f0fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b38010e10 tx=0x7f6b38004730 comp rx=0 tx=0).stop 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 shutdown_connections 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f1b6b0 0x557340f1db20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f11160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b22ffd640 0 client.16410 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f09190 0x557340f10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3179766212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f06860 0x557340f0fef0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 >> 172.21.15.96:0/3179766212 conn(0x557340f019b0 msgr2=0x557340f00bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 shutdown_connections 2024-09-10T22:52:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 1 -- 172.21.15.96:0/3179766212 wait complete. 2024-09-10T22:52:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.939+0000 7f6b4c881740 20 client.16410 trim_cache size 0 max 16384 2024-09-10T22:52:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.957+0000 7f6b4c881740 1 Processor -- start 2024-09-10T22:52:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 -- start start 2024-09-10T22:52:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f22420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441b0270 con 0x557340f09190 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441b1db0 con 0x557340f0bbd0 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b4c881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441b0940 con 0x557340f06860 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38760/0 (socket says 172.21.15.96:38760) 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.958+0000 7f6b42ffd640 1 -- 172.21.15.96:0/3938276874 learned_addr learned my addr 172.21.15.96:0/3938276874 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f22420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 -- 172.21.15.96:0/3938276874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 msgr2=0x557340f22420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f22420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 -- 172.21.15.96:0/3938276874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 msgr2=0x557340f22960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 -- 172.21.15.96:0/3938276874 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557340f02450 con 0x557340f0bbd0 2024-09-10T22:52:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.959+0000 7f6b427fc640 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f22ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b38010d70 tx=0x7f6b38005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e1db0e126441c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b297fa640 1 -- 172.21.15.96:0/3938276874 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38024030 con 0x557340f0bbd0 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b297fa640 1 -- 172.21.15.96:0/3938276874 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3801b040 con 0x557340f0bbd0 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b297fa640 1 -- 172.21.15.96:0/3938276874 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800ea90 con 0x557340f0bbd0 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 -- 172.21.15.96:0/3938276874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 msgr2=0x557340f22ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f22ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b38010d70 tx=0x7f6b38005f90 comp rx=0 tx=0).stop 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 -- 172.21.15.96:0/3938276874 shutdown_connections 2024-09-10T22:52:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f0bbd0 0x557340f22ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 --2- 172.21.15.96:0/3938276874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f06860 0x557340f22420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 -- 172.21.15.96:0/3938276874 >> 172.21.15.96:0/3938276874 conn(0x557340f019b0 msgr2=0x557340f04a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 -- 172.21.15.96:0/3938276874 shutdown_connections 2024-09-10T22:52:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.960+0000 7f6b4c881740 1 -- 172.21.15.96:0/3938276874 wait complete. 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 -- start start 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 Processor -- start 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f06860 0x557340f17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 0x557340f16d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441b16e0 con 0x557340f09190 2024-09-10T22:52:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441b1010 con 0x557340f06860 2024-09-10T22:52:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b441ae250 con 0x557340f0bbd0 2024-09-10T22:52:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b4c881740 10 client.0 _ll_register_callbacks cb 0x7f6b38009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b42ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38776/0 (socket says 172.21.15.96:38776) 2024-09-10T22:52:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b42ffd640 1 -- 172.21.15.96:0/4209840471 learned_addr learned my addr 172.21.15.96:0/4209840471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b427fc640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f06860 0x557340f17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.961+0000 7f6b427fc640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 0x557340f16d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 -- 172.21.15.96:0/4209840471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 msgr2=0x557340f16d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 0x557340f16d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 -- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f06860 msgr2=0x557340f17a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f06860 0x557340f17a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 -- 172.21.15.96:0/4209840471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557340f17550 con 0x557340f09190 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b427fc640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 0x557340f16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b3401a000 tx=0x7f6b34003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805592e462e97116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34003c00 con 0x557340f09190 2024-09-10T22:52:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b3801b040 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6b34005040 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b34005240 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b4c881740 10 client.16416 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557340f887c0 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3801b040 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6b3401b420 con 0x557340f09190 2024-09-10T22:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b4c881740 10 client.16416 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.962+0000 7f6b4c881740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b4c881740 20 client.16416 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f88ee0 0x557340f8b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b4c881740 4 client.16416 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557340e760f0 con 0x557340f88ee0 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b427fc640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f88ee0 0x557340f8b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b22ffd640 10 client.16416 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:12.963+0000 7f6b427fc640 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f88ee0 0x557340f8b300 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ecf8403188c61f5 server_cookie=a835c2df22452b31 in_seq=0 out_seq=0 2024-09-10T22:52:12.990 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:12 smithi096 ceph-mon[30869]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-09-10T22:52:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.016+0000 7f6b22ffd640 1 -- 172.21.15.96:0/4209840471 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557340e760f0 con 0x557340f88ee0 2024-09-10T22:52:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.016+0000 7f6b22ffd640 10 client.16416 handle_command_reply: tid=0 2024-09-10T22:52:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.016+0000 7f6b4c881740 1 client.16416 shutdown 2024-09-10T22:52:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.018+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 msgr2=0x557340f16760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.018+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b3401a000 tx=0x7f6b34003680 comp rx=0 tx=0).stop 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.018+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 shutdown_connections 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.018+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557340f88ee0 0x557340f8b300 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b22ffd640 0 client.16416 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557340f0bbd0 0x557340f16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557340f09190 0x557340f16760 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 --2- 172.21.15.96:0/4209840471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557340f06860 0x557340f17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 >> 172.21.15.96:0/4209840471 conn(0x557340f019b0 msgr2=0x557340f060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 shutdown_connections 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 1 -- 172.21.15.96:0/4209840471 wait complete. 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.019+0000 7f6b4c881740 20 client.16416 trim_cache size 0 max 16384 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:52:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:52:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 msgr2=0x7f6b2c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 0x7f6b2c0b78f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009ca0 tx=0x7f6b3c0095c0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 msgr2=0x7f6b441a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c033040 tx=0x7f6b3c003b80 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 shutdown_connections 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4410a100 0x7f6b4419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6b2c0b5430 0x7f6b2c0b78f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b4410aac0 0x7f6b441a0590 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 --2- 172.21.15.96:0/2582325020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b44109230 0x7f6b4419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 >> 172.21.15.96:0/2582325020 conn(0x7f6b440783d0 msgr2=0x7f6b4410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 shutdown_connections 2024-09-10T22:52:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:13.025+0000 7f6b499d3640 1 -- 172.21.15.96:0/2582325020 wait complete. 2024-09-10T22:52:13.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 -- 172.21.15.96:0/2825454191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb878103b70 msgr2=0x7fb878103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2825454191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb878103b70 0x7fb878103f50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb868009900 tx=0x7fb86802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 -- 172.21.15.96:0/2825454191 shutdown_connections 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2825454191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb878104570 0x7fb87810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2825454191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87810d120 0x7fb87810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2825454191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb878103b70 0x7fb878103f50 secure :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb868009900 tx=0x7fb86802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.110+0000 7fb87d75a640 1 -- 172.21.15.96:0/2825454191 >> 172.21.15.96:0/2825454191 conn(0x7fb8780fd700 msgr2=0x7fb8780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.111+0000 7fb87d75a640 1 -- 172.21.15.96:0/2825454191 shutdown_connections 2024-09-10T22:52:13.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.111+0000 7fb87d75a640 1 -- 172.21.15.96:0/2825454191 wait complete. 2024-09-10T22:52:13.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 Processor -- start 2024-09-10T22:52:13.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 -- start start 2024-09-10T22:52:13.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb878104570 0x7fb87819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 0x7fb87819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 0x7fb8781a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb878112d60 con 0x7fb8781a0f50 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb878112be0 con 0x7fb87810d120 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.112+0000 7fb87d75a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb878112ee0 con 0x7fb878104570 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 0x7fb8781a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 0x7fb8781a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38782/0 (socket says 172.21.15.96:38782) 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb8777fe640 1 -- 172.21.15.96:0/2776737971 learned_addr learned my addr 172.21.15.96:0/2776737971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 0x7fb87819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb876ffd640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb878104570 0x7fb87819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 -- 172.21.15.96:0/2776737971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb878104570 msgr2=0x7fb87819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb878104570 0x7fb87819b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 -- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 msgr2=0x7fb8781a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 0x7fb8781a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 -- 172.21.15.96:0/2776737971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8781a1ae0 con 0x7fb87810d120 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb86ffff640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 0x7fb87819bbb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e9f0 tx=0x7fb86000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.113+0000 7fb874ff9640 1 -- 172.21.15.96:0/2776737971 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86000ccc0 con 0x7fb87810d120 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.114+0000 7fb874ff9640 1 -- 172.21.15.96:0/2776737971 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8600108e0 con 0x7fb87810d120 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.114+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8781a80d0 con 0x7fb87810d120 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.114+0000 7fb874ff9640 1 -- 172.21.15.96:0/2776737971 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860018880 con 0x7fb87810d120 2024-09-10T22:52:13.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.114+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a84c0 con 0x7fb87810d120 2024-09-10T22:52:13.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.117+0000 7fb874ff9640 1 -- 172.21.15.96:0/2776737971 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb860010430 con 0x7fb87810d120 2024-09-10T22:52:13.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.117+0000 7fb874ff9640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 0x7fb8640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.118+0000 7fb874ff9640 1 -- 172.21.15.96:0/2776737971 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb8600e5ed0 con 0x7fb87810d120 2024-09-10T22:52:13.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.118+0000 7fb876ffd640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 0x7fb8640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb876ffd640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 0x7fb8640b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb868005940 tx=0x7fb8680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 Processor -- start 2024-09-10T22:52:13.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 -- start start 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 0x55b0e0dea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0d49ea0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0d392d0 con 0x55b0e0dd3df0 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.119+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0d3d0e0 con 0x55b0e0de8590 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33374/0 (socket says 172.21.15.96:33374) 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 0x55b0e0dea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb86ffff640 1 -- 172.21.15.96:0/3170561147 learned_addr learned my addr 172.21.15.96:0/3170561147 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 -- 172.21.15.96:0/3170561147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 msgr2=0x55b0e0de8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 -- 172.21.15.96:0/3170561147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0dd34d0 msgr2=0x55b0e0dd38b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd38b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.120+0000 7fb8777fe640 1 -- 172.21.15.96:0/3170561147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e0a97650 con 0x55b0e0de8590 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.121+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 0x55b0e0dea980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb870007e90 tx=0x7fb87000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be951b9fe43facee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.121+0000 7fb86d7fa640 1 -- 172.21.15.96:0/3170561147 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870010840 con 0x55b0e0de8590 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.121+0000 7fb86d7fa640 1 -- 172.21.15.96:0/3170561147 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700183f0 con 0x55b0e0de8590 2024-09-10T22:52:13.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.121+0000 7fb86d7fa640 1 -- 172.21.15.96:0/3170561147 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870017600 con 0x55b0e0de8590 2024-09-10T22:52:13.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3170561147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 msgr2=0x55b0e0dea980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 0x55b0e0dea980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb870007e90 tx=0x7fb87000f790 comp rx=0 tx=0).stop 2024-09-10T22:52:13.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3170561147 shutdown_connections 2024-09-10T22:52:13.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0de8590 0x55b0e0dea980 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3170561147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3170561147 >> 172.21.15.96:0/3170561147 conn(0x55b0e0dd1300 msgr2=0x55b0e0dd24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3170561147 shutdown_connections 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3170561147 wait complete. 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.122+0000 7fb87fb2f740 1 -- start start 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 Processor -- start 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0de7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0de5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0b9a700 con 0x55b0e0de8590 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0cd4160 con 0x55b0e0dd34d0 2024-09-10T22:52:13.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b0e0deb240 con 0x55b0e0dd3df0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb87fb2f740 10 client.0 _ll_register_callbacks cb 0x55b0e0ced9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0de7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0de7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33378/0 (socket says 172.21.15.96:33378) 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb876ffd640 1 -- 172.21.15.96:0/3187632767 learned_addr learned my addr 172.21.15.96:0/3187632767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0de5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb86ffff640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 -- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 msgr2=0x55b0e0de5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 -- 172.21.15.96:0/3187632767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 msgr2=0x55b0e0de7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0de7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.123+0000 7fb8777fe640 1 -- 172.21.15.96:0/3187632767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e0d01c90 con 0x55b0e0de8590 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0de5840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb87001a040 tx=0x7fb870007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9a12e46636d0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:13.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870026030 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb87001e040 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8700183f0 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb870004c50 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb87fb2f740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.124+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0e0df6bb0 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700183f0 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb8700045c0 con 0x55b0e0de8590 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 10 client.16425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 20 client.16425 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0df8070 0x55b0e0dfa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 4 client.16425 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.125+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e0d4bdf0 con 0x55b0e0df8070 2024-09-10T22:52:13.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.126+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0df8070 0x55b0e0dfa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.126+0000 7fb85e7fc640 10 client.16425 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:52:13.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.126+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0df8070 0x55b0e0dfa460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22c61f43ef453afa server_cookie=1f15db572b8f8f55 in_seq=0 out_seq=0 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3187632767 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0e0d4bdf0 con 0x55b0e0df8070 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb85e7fc640 10 client.16425 handle_command_reply: tid=0 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 client.16425 shutdown 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 msgr2=0x55b0e0de5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0de5840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb87001a040 tx=0x7fb870007b60 comp rx=0 tx=0).stop 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 shutdown_connections 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0df8070 0x55b0e0dfa460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0de5840 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0de5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3187632767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0de7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 >> 172.21.15.96:0/3187632767 conn(0x55b0e0dd1300 msgr2=0x55b0e0dd6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb85e7fc640 0 client.16425 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:52:13.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 shutdown_connections 2024-09-10T22:52:13.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3187632767 wait complete. 2024-09-10T22:52:13.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.127+0000 7fb87fb2f740 20 client.16425 trim_cache size 0 max 16384 2024-09-10T22:52:13.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.144+0000 7fb87fb2f740 1 Processor -- start 2024-09-10T22:52:13.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.144+0000 7fb87fb2f740 1 -- start start 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.144+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781ac700 con 0x55b0e0de8590 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781ae240 con 0x55b0e0dd3df0 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb87fb2f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781acdd0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38794/0 (socket says 172.21.15.96:38794) 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 -- 172.21.15.96:0/3515047925 learned_addr learned my addr 172.21.15.96:0/3515047925 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb86ffff640 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 -- 172.21.15.96:0/3515047925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 msgr2=0x55b0e0dd0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 -- 172.21.15.96:0/3515047925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 msgr2=0x55b0e0df44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 -- 172.21.15.96:0/3515047925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e0de3780 con 0x55b0e0de8590 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:13.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.145+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb870018d80 tx=0x7fb870007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e70d773eb34dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.146+0000 7fb86cff9640 1 -- 172.21.15.96:0/3515047925 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870026040 con 0x55b0e0de8590 2024-09-10T22:52:13.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.146+0000 7fb86cff9640 1 -- 172.21.15.96:0/3515047925 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700183f0 con 0x55b0e0de8590 2024-09-10T22:52:13.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.146+0000 7fb86cff9640 1 -- 172.21.15.96:0/3515047925 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870003dd0 con 0x55b0e0de8590 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3515047925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 msgr2=0x55b0e0dfa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb870018d80 tx=0x7fb870007760 comp rx=0 tx=0).stop 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3515047925 shutdown_connections 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0dfa080 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3515047925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd34d0 0x55b0e0dd0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.147+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3515047925 >> 172.21.15.96:0/3515047925 conn(0x55b0e0dd1300 msgr2=0x55b0e0dea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.148+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3515047925 shutdown_connections 2024-09-10T22:52:13.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.148+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3515047925 wait complete. 2024-09-10T22:52:13.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.148+0000 7fb87fb2f740 1 -- start start 2024-09-10T22:52:13.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.148+0000 7fb87fb2f740 1 Processor -- start 2024-09-10T22:52:13.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33402/0 (socket says 172.21.15.96:33402) 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb876ffd640 1 -- 172.21.15.96:0/3364093617 learned_addr learned my addr 172.21.15.96:0/3364093617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0ded4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb86ffff640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781adb70 con 0x55b0e0de8590 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781ad4a0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781aa740 con 0x55b0e0dd3df0 2024-09-10T22:52:13.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb8777fe640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0ded4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb876ffd640 1 -- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 msgr2=0x55b0e0df0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.149+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb876ffd640 1 -- 172.21.15.96:0/3364093617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 msgr2=0x55b0e0ded4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0ded4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb876ffd640 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e0df1130 con 0x55b0e0dd34d0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb87fb2f740 10 client.0 _ll_register_callbacks cb 0x7fb8600a8d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb876ffd640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb868031440 tx=0x7fb86802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c706592fcb1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8700183f0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3364093617 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868059060 con 0x55b0e0dd34d0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3364093617 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb8680314a0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3364093617 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb868035040 con 0x55b0e0dd34d0 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb87fb2f740 10 client.25997 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.150+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b0e0dd9be0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3364093617 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb8680316e0 con 0x55b0e0dd34d0 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 10 client.25997 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 10 client.25997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 20 client.25997 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0dda200 0x55b0e0ddc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 4 client.25997 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb8700183f0 con 0x55b0e0dda200 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb86ffff640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0dda200 0x55b0e0ddc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb85e7fc640 10 client.25997 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:52:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.151+0000 7fb86ffff640 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0dda200 0x55b0e0ddc680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da96f802f090aa93 server_cookie=6c4b0d50b583c218 in_seq=0 out_seq=0 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb85e7fc640 1 -- 172.21.15.96:0/3364093617 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb8700183f0 con 0x55b0e0dda200 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb85e7fc640 10 client.25997 handle_command_reply: tid=0 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 client.25997 shutdown 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 msgr2=0x55b0e0df0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb868031440 tx=0x7fb86802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 shutdown_connections 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b0e0dda200 0x55b0e0ddc680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0e0de8590 0x55b0e0ded4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb85e7fc640 0 client.25997 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0e0dd3df0 0x55b0e0df0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 --2- 172.21.15.96:0/3364093617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0e0dd34d0 0x55b0e0df0500 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 >> 172.21.15.96:0/3364093617 conn(0x55b0e0dd1300 msgr2=0x55b0e0de68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 shutdown_connections 2024-09-10T22:52:13.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 1 -- 172.21.15.96:0/3364093617 wait complete. 2024-09-10T22:52:13.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.152+0000 7fb87fb2f740 20 client.25997 trim_cache size 0 max 16384 2024-09-10T22:52:13.158 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 msgr2=0x7fb8640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 0x7fb8640b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb868005940 tx=0x7fb8680058d0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 msgr2=0x7fb87819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 0x7fb87819bbb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e9f0 tx=0x7fb86000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 shutdown_connections 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87810d120 0x7fb87819bbb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8640b52e0 0x7fb8640b77a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8781a0f50 0x7fb8781a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 --2- 172.21.15.96:0/2776737971 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb878104570 0x7fb87819b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 >> 172.21.15.96:0/2776737971 conn(0x7fb8780fd700 msgr2=0x7fb8780ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 shutdown_connections 2024-09-10T22:52:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:13.153+0000 7fb87d75a640 1 -- 172.21.15.96:0/2776737971 wait complete. 2024-09-10T22:52:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:12 smithi121 ceph-mon[35119]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-09-10T22:52:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:12 smithi117 ceph-mon[35081]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-09-10T22:52:14.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:52:14.431 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:15.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 -- 172.21.15.96:0/2054053102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84102de0 msgr2=0x7f0f841031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/2054053102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84102de0 0x7f0f841031c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f7401a910 tx=0x7f0f7403b420 comp rx=0 tx=0).stop 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 -- 172.21.15.96:0/2054053102 shutdown_connections 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/2054053102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84109170 0x7f0f8410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/2054053102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/2054053102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84102de0 0x7f0f841031c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 -- 172.21.15.96:0/2054053102 >> 172.21.15.96:0/2054053102 conn(0x7f0f840fe830 msgr2=0x7f0f84100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 -- 172.21.15.96:0/2054053102 shutdown_connections 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.000+0000 7f0f89ad9640 1 -- 172.21.15.96:0/2054053102 wait complete. 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.001+0000 7f0f89ad9640 1 Processor -- start 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.001+0000 7f0f89ad9640 1 -- start start 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 0x7f0f84070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406f400 con 0x7f0f84102de0 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406f280 con 0x7f0f84109170 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f89ad9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406f580 con 0x7f0f841087f0 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 0x7f0f84070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33426/0 (socket says 172.21.15.96:33426) 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 -- 172.21.15.96:0/877860225 learned_addr learned my addr 172.21.15.96:0/877860225 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 -- 172.21.15.96:0/877860225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 msgr2=0x7f0f84071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 -- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 msgr2=0x7f0f84070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 0x7f0f84070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f83fff640 1 -- 172.21.15.96:0/877860225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f84076b30 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.002+0000 7f0f837fe640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 0x7f0f84070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f83fff640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b970 tx=0x7f0f7c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f80ff9640 1 -- 172.21.15.96:0/877860225 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c019070 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f80ff9640 1 -- 172.21.15.96:0/877860225 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c014440 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f80ff9640 1 -- 172.21.15.96:0/877860225 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0075d0 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f84076e20 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.003+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8419b330 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.006+0000 7f0f80ff9640 1 -- 172.21.15.96:0/877860225 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0f7c00cb40 con 0x7f0f84109170 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.007+0000 7f0f8beae740 1 Processor -- start 2024-09-10T22:52:15.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.008+0000 7f0f8beae740 1 -- start start 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.008+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.008+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448aedf0 0x5621448ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.008+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f8beae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562144824ea0 con 0x5621448aedf0 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f8beae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5621448142d0 con 0x5621448bad80 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f8beae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5621448180e0 con 0x5621448ae4d0 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f80ff9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 0x7f0f700b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448aedf0 0x5621448ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f83fff640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 0x7f0f700b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33430/0 (socket says 172.21.15.96:33430) 2024-09-10T22:52:15.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4267332307 learned_addr learned my addr 172.21.15.96:0/4267332307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4267332307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 msgr2=0x5621448ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4267332307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448aedf0 msgr2=0x5621448ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448aedf0 0x5621448ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4267332307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562144572650 con 0x5621448bad80 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c012820 tx=0x7f0f6c009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d9739b03072599 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f83fff640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 0x7f0f700b78f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ab9f0 tx=0x7f0f7c0e8940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f80ff9640 1 -- 172.21.15.96:0/877860225 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0f7c0e51f0 con 0x7f0f84109170 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.009+0000 7f0f79ffb640 1 -- 172.21.15.96:0/4267332307 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c012e60 con 0x5621448bad80 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f79ffb640 1 -- 172.21.15.96:0/4267332307 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c0107d0 con 0x5621448bad80 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f79ffb640 1 -- 172.21.15.96:0/4267332307 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003850 con 0x5621448bad80 2024-09-10T22:52:15.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 -- 172.21.15.96:0/4267332307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 msgr2=0x5621448bd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c012820 tx=0x7f0f6c009720 comp rx=0 tx=0).stop 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 -- 172.21.15.96:0/4267332307 shutdown_connections 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448bd170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448aedf0 0x5621448ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4267332307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.010+0000 7f0f8beae740 1 -- 172.21.15.96:0/4267332307 >> 172.21.15.96:0/4267332307 conn(0x5621448ac300 msgr2=0x5621448ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.011+0000 7f0f8beae740 1 -- 172.21.15.96:0/4267332307 shutdown_connections 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.011+0000 7f0f8beae740 1 -- 172.21.15.96:0/4267332307 wait complete. 2024-09-10T22:52:15.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.011+0000 7f0f8beae740 1 -- start start 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.011+0000 7f0f8beae740 1 Processor -- start 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448b8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562144675700 con 0x5621448bad80 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5621447af160 con 0x5621448aedf0 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5621448bda00 con 0x5621448ae4d0 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f8beae740 10 client.0 _ll_register_callbacks cb 0x5621447c89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33446/0 (socket says 172.21.15.96:33446) 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f837fe640 1 -- 172.21.15.96:0/3124099503 learned_addr learned my addr 172.21.15.96:0/3124099503 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.012+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 -- 172.21.15.96:0/3124099503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 msgr2=0x5621448ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 -- 172.21.15.96:0/3124099503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 msgr2=0x5621448b8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448b8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 -- 172.21.15.96:0/3124099503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621447dcc90 con 0x5621448aedf0 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f837fe640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f7403b940 tx=0x7f0f74004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38afdb9ce0378fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74052070 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6c0107d0 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.013+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0f74004320 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0f7405c050 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 10 client.26009 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5621448c9420 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c0107d0 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0f7404b040 con 0x5621448aedf0 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 10 client.26009 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:15.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 10 client.26009 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 20 client.26009 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5621448ca980 0x5621448ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 4 client.26009 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.014+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562144826df0 con 0x5621448ca980 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.015+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5621448ca980 0x5621448ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.015+0000 7f0f62ffd640 10 client.26009 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.015+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5621448ca980 0x5621448ccd70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4c285fc8ace5b37 server_cookie=6d074f00418b8e14 in_seq=0 out_seq=0 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.016+0000 7f0f62ffd640 1 -- 172.21.15.96:0/3124099503 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562144826df0 con 0x5621448ca980 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.016+0000 7f0f62ffd640 10 client.26009 handle_command_reply: tid=0 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.016+0000 7f0f8beae740 1 client.26009 shutdown 2024-09-10T22:52:15.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 msgr2=0x5621448b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f7403b940 tx=0x7f0f74004d20 comp rx=0 tx=0).stop 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 shutdown_connections 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5621448ca980 0x5621448ccd70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448b7ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448b8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 --2- 172.21.15.96:0/3124099503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 >> 172.21.15.96:0/3124099503 conn(0x5621448ac300 msgr2=0x5621448b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.017+0000 7f0f62ffd640 0 client.26009 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:52:15.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.018+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 shutdown_connections 2024-09-10T22:52:15.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.018+0000 7f0f8beae740 1 -- 172.21.15.96:0/3124099503 wait complete. 2024-09-10T22:52:15.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.018+0000 7f0f8beae740 20 client.26009 trim_cache size 0 max 16384 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.035+0000 7f0f8beae740 1 Processor -- start 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 -- start start 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 0x5621448abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b4870 con 0x5621448ae4d0 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b63b0 con 0x5621448bad80 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f8beae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b4f40 con 0x5621448aedf0 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38854/0 (socket says 172.21.15.96:38854) 2024-09-10T22:52:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f82ffd640 1 -- 172.21.15.96:0/428591226 learned_addr learned my addr 172.21.15.96:0/428591226 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.036+0000 7f0f837fe640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 0x5621448abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 -- 172.21.15.96:0/428591226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 msgr2=0x5621448abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 0x5621448abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 -- 172.21.15.96:0/428591226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 msgr2=0x5621448b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 -- 172.21.15.96:0/428591226 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621448acd80 con 0x5621448ae4d0 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f837fe640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 0x5621448abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0127c0 tx=0x7f0f6c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa76c2c28c008fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f797fa640 1 -- 172.21.15.96:0/428591226 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0107d0 con 0x5621448ae4d0 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f797fa640 1 -- 172.21.15.96:0/428591226 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c012050 con 0x5621448ae4d0 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.037+0000 7f0f797fa640 1 -- 172.21.15.96:0/428591226 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01f820 con 0x5621448ae4d0 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- 172.21.15.96:0/428591226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 msgr2=0x5621448b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0127c0 tx=0x7f0f6c0045d0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- 172.21.15.96:0/428591226 shutdown_connections 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448bad80 0x5621448b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448aedf0 0x5621448abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 --2- 172.21.15.96:0/428591226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448ae4d0 0x5621448b7950 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- 172.21.15.96:0/428591226 >> 172.21.15.96:0/428591226 conn(0x5621448ac300 msgr2=0x5621448b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- 172.21.15.96:0/428591226 shutdown_connections 2024-09-10T22:52:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- 172.21.15.96:0/428591226 wait complete. 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.038+0000 7f0f8beae740 1 -- start start 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 Processor -- start 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b5ce0 con 0x5621448bad80 2024-09-10T22:52:15.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b5610 con 0x5621448aedf0 2024-09-10T22:52:15.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f8beae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841b2870 con 0x5621448ae4d0 2024-09-10T22:52:15.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38860/0 (socket says 172.21.15.96:38860) 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4081257578 learned_addr learned my addr 172.21.15.96:0/4081257578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.039+0000 7f0f837fe640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f8beae740 10 client.0 _ll_register_callbacks cb 0x7f0f74005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4081257578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 msgr2=0x5621448c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 msgr2=0x5621448c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:15.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 -- 172.21.15.96:0/4081257578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621448be190 con 0x5621448bad80 2024-09-10T22:52:15.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f837fe640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:15.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f82ffd640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009dd0 tx=0x7f0f6c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00e92b0d94ac28a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c010880 con 0x5621448bad80 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6c012050 con 0x5621448bad80 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0f6c01b040 con 0x5621448bad80 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0f6c010a20 con 0x5621448bad80 2024-09-10T22:52:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f8beae740 10 client.16446 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.040+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562144937ed0 con 0x5621448bad80 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.041+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c012050 con 0x5621448bad80 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.041+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0f6c02d070 con 0x5621448bad80 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 10 client.16446 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 10 client.16446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:52:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 20 client.16446 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562144938380 0x56214493a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 4 client.16446 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621448228a0 con 0x562144938380 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.043+0000 7f0f837fe640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562144938380 0x56214493a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f62ffd640 10 client.16446 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:52:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f837fe640 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562144938380 0x56214493a7a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90d456ad05bdf51c server_cookie=5173160bef8ddd9d in_seq=0 out_seq=0 2024-09-10T22:52:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f62ffd640 1 -- 172.21.15.96:0/4081257578 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621448228a0 con 0x562144938380 2024-09-10T22:52:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f62ffd640 10 client.16446 handle_command_reply: tid=0 2024-09-10T22:52:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 client.16446 shutdown 2024-09-10T22:52:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 msgr2=0x5621448bdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009dd0 tx=0x7f0f6c009580 comp rx=0 tx=0).stop 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 shutdown_connections 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x562144938380 0x56214493a7a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621448aedf0 0x5621448c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621448bad80 0x5621448bdc50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 --2- 172.21.15.96:0/4081257578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621448ae4d0 0x5621448c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 >> 172.21.15.96:0/4081257578 conn(0x5621448ac300 msgr2=0x5621448b99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f62ffd640 0 client.16446 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:52:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 shutdown_connections 2024-09-10T22:52:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 1 -- 172.21.15.96:0/4081257578 wait complete. 2024-09-10T22:52:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.044+0000 7f0f8beae740 20 client.16446 trim_cache size 0 max 16384 2024-09-10T22:52:15.061 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 msgr2=0x7f0f700b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 0x7f0f700b78f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ab9f0 tx=0x7f0f7c0e8940 comp rx=0 tx=0).stop 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 msgr2=0x7f0f840763b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b970 tx=0x7f0f7c00be30 comp rx=0 tx=0).stop 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 shutdown_connections 2024-09-10T22:52:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f84109170 0x7f0f840763b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0f700b5430 0x7f0f700b78f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f841087f0 0x7f0f84071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 --2- 172.21.15.96:0/877860225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f84102de0 0x7f0f84070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 >> 172.21.15.96:0/877860225 conn(0x7f0f840fe830 msgr2=0x7f0f84100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 shutdown_connections 2024-09-10T22:52:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:15.045+0000 7f0f89ad9640 1 -- 172.21.15.96:0/877860225 wait complete. 2024-09-10T22:52:15.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:14 smithi096 ceph-mon[30869]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-10T22:52:15.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:14 smithi121 ceph-mon[35119]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-10T22:52:15.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:14 smithi117 ceph-mon[35081]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 783 op/s 2024-09-10T22:52:15.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:52:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 280.0 s, 2701.4 tps, lat 37.075 ms stddev 41.092 2024-09-10T22:52:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:16.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 -- 172.21.15.96:0/758878540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84109170 msgr2=0x7f3f8410f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/758878540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84109170 0x7f3f8410f970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009ca0 tx=0x7f3f7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 -- 172.21.15.96:0/758878540 shutdown_connections 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/758878540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84109170 0x7f3f8410f970 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/758878540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f841087f0 0x7f3f84108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/758878540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f84102de0 0x7f3f841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 -- 172.21.15.96:0/758878540 >> 172.21.15.96:0/758878540 conn(0x7f3f840fe830 msgr2=0x7f3f84100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 -- 172.21.15.96:0/758878540 shutdown_connections 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.630+0000 7f3f8bb33640 1 -- 172.21.15.96:0/758878540 wait complete. 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.631+0000 7f3f8bb33640 1 Processor -- start 2024-09-10T22:52:16.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.631+0000 7f3f8bb33640 1 -- start start 2024-09-10T22:52:16.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.631+0000 7f3f8bb33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8bb33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f841087f0 0x7f3f8419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8bb33640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84109170 0x7f3f841a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8bb33640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84115020 con 0x7f3f84102de0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8bb33640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84114ea0 con 0x7f3f841087f0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8bb33640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841151a0 con 0x7f3f84109170 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36544/0 (socket says 172.21.15.96:36544) 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 -- 172.21.15.96:0/2507279908 learned_addr learned my addr 172.21.15.96:0/2507279908 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f890ce640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f841087f0 0x7f3f8419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84109170 0x7f3f841a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 -- 172.21.15.96:0/2507279908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84109170 msgr2=0x7f3f841a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84109170 0x7f3f841a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 -- 172.21.15.96:0/2507279908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f841087f0 msgr2=0x7f3f8419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f841087f0 0x7f3f8419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 -- 172.21.15.96:0/2507279908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f841a1500 con 0x7f3f84102de0 2024-09-10T22:52:16.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.632+0000 7f3f898cf640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9f0 tx=0x7f3f8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.633+0000 7f3f7affd640 1 -- 172.21.15.96:0/2507279908 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000cd80 con 0x7f3f84102de0 2024-09-10T22:52:16.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.633+0000 7f3f7affd640 1 -- 172.21.15.96:0/2507279908 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f800108e0 con 0x7f3f84102de0 2024-09-10T22:52:16.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.633+0000 7f3f7affd640 1 -- 172.21.15.96:0/2507279908 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80018980 con 0x7f3f84102de0 2024-09-10T22:52:16.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.633+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f841a17f0 con 0x7f3f84102de0 2024-09-10T22:52:16.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.633+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a7eb0 con 0x7f3f84102de0 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f7affd640 1 -- 172.21.15.96:0/2507279908 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3f80010430 con 0x7f3f84102de0 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 Processor -- start 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 -- start start 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a86d644ea0 con 0x55a86d6cedf0 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a86d6342d0 con 0x55a86d6ce4d0 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.636+0000 7f3f8df08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a86d6380e0 con 0x55a86d6dad80 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6da840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36554/0 (socket says 172.21.15.96:36554) 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f898cf640 1 -- 172.21.15.96:0/3763369410 learned_addr learned my addr 172.21.15.96:0/3763369410 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f890ce640 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f7affd640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 0x7f3f700b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 -- 172.21.15.96:0/3763369410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 msgr2=0x55a86d6dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 -- 172.21.15.96:0/3763369410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 msgr2=0x55a86d6da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6da840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 -- 172.21.15.96:0/3763369410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86d392650 con 0x55a86d6ce4d0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f7affd640 1 -- 172.21.15.96:0/2507279908 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3f800e5e50 con 0x7f3f84102de0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 0x7f3f700b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.637+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6ce8b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c02ade0 tx=0x7f3f7c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c94b819dd34f1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.638+0000 7f3f6ffff640 1 -- 172.21.15.96:0/3763369410 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c04d070 con 0x55a86d6ce4d0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.638+0000 7f3f6ffff640 1 -- 172.21.15.96:0/3763369410 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c036040 con 0x55a86d6ce4d0 2024-09-10T22:52:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.638+0000 7f3f6ffff640 1 -- 172.21.15.96:0/3763369410 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0044c0 con 0x55a86d6ce4d0 2024-09-10T22:52:16.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.638+0000 7f3f8df08740 1 -- 172.21.15.96:0/3763369410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 msgr2=0x55a86d6ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.638+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6ce8b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c02ade0 tx=0x7f3f7c009870 comp rx=0 tx=0).stop 2024-09-10T22:52:16.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 0x7f3f700b78f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c046000 tx=0x7f3f7c0047a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 -- 172.21.15.96:0/3763369410 shutdown_connections 2024-09-10T22:52:16.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3763369410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6ce8b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 -- 172.21.15.96:0/3763369410 >> 172.21.15.96:0/3763369410 conn(0x55a86d6cc300 msgr2=0x55a86d6cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.639+0000 7f3f8df08740 1 -- 172.21.15.96:0/3763369410 shutdown_connections 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.640+0000 7f3f8df08740 1 -- 172.21.15.96:0/3763369410 wait complete. 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.640+0000 7f3f8df08740 1 -- start start 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.640+0000 7f3f8df08740 1 Processor -- start 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 0x55a86d6d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56168/0 (socket says 172.21.15.96:56168) 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f898cf640 1 -- 172.21.15.96:0/151980556 learned_addr learned my addr 172.21.15.96:0/151980556 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f890ce640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 0x55a86d6d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a86d495700 con 0x55a86d6dad80 2024-09-10T22:52:16.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a86d5cf160 con 0x55a86d6ce4d0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a86d6dda00 con 0x55a86d6cedf0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.641+0000 7f3f8df08740 10 client.0 _ll_register_callbacks cb 0x55a86d5e89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 -- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 msgr2=0x55a86d6d7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 0x55a86d6d7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 -- 172.21.15.96:0/151980556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 msgr2=0x55a86d6d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86d5fcc90 con 0x55a86d6ce4d0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f890ce640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 0x55a86d6d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f898cf640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a8c80 tx=0x7f3f80009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b7d8a9b303b152 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800a8550 con 0x55a86d6ce4d0 2024-09-10T22:52:16.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c036040 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3f80018b20 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f800abe20 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f8df08740 10 client.26027 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.642+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a86d6e9420 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c036040 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3f800a8000 con 0x55a86d6ce4d0 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 10 client.26027 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 20 client.26027 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:16.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d6ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 4 client.26027 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:16.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.643+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86d646df0 con 0x55a86d6ea880 2024-09-10T22:52:16.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.644+0000 7f3f890ce640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d6ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.644+0000 7f3f890ce640 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d6ecc90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f90b2c3534ddf98f server_cookie=cbc2d48a9917b108 in_seq=0 out_seq=0 2024-09-10T22:52:16.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.644+0000 7f3f6d7fa640 10 client.26027 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:52:16.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/151980556 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a86d646df0 con 0x55a86d6ea880 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f6d7fa640 10 client.26027 handle_command_reply: tid=0 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 client.26027 shutdown 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 msgr2=0x55a86d6da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a8c80 tx=0x7f3f80009310 comp rx=0 tx=0).stop 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 shutdown_connections 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6cedf0 0x55a86d6d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d6ecc90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f6d7fa640 0 client.26027 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 --2- 172.21.15.96:0/151980556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6da6b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.645+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 >> 172.21.15.96:0/151980556 conn(0x55a86d6cc300 msgr2=0x55a86d6d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.646+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 shutdown_connections 2024-09-10T22:52:16.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.646+0000 7f3f8df08740 1 -- 172.21.15.96:0/151980556 wait complete. 2024-09-10T22:52:16.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.646+0000 7f3f8df08740 20 client.26027 trim_cache size 0 max 16384 2024-09-10T22:52:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.663+0000 7f3f8df08740 1 Processor -- start 2024-09-10T22:52:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.663+0000 7f3f8df08740 1 -- start start 2024-09-10T22:52:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.663+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f898cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56180/0 (socket says 172.21.15.96:56180) 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f898cf640 1 -- 172.21.15.96:0/3506439458 learned_addr learned my addr 172.21.15.96:0/3506439458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b04b0 con 0x55a86d6cedf0 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b1ff0 con 0x55a86d6ce4d0 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b0b80 con 0x55a86d6dad80 2024-09-10T22:52:16.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 -- 172.21.15.96:0/3506439458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 msgr2=0x55a86d6dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 -- 172.21.15.96:0/3506439458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 msgr2=0x55a86d6cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.664+0000 7f3f890ce640 1 -- 172.21.15.96:0/3506439458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86d6da2e0 con 0x55a86d6cedf0 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.665+0000 7f3f890ce640 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6e6c30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55a86d6ed4a0 tx=0x7f3f7400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e93f31c6e749a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.665+0000 7f3f6f7fe640 1 -- 172.21.15.96:0/3506439458 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74003e20 con 0x55a86d6cedf0 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.665+0000 7f3f6f7fe640 1 -- 172.21.15.96:0/3506439458 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a86d646df0 con 0x55a86d6cedf0 2024-09-10T22:52:16.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.665+0000 7f3f6f7fe640 1 -- 172.21.15.96:0/3506439458 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74024760 con 0x55a86d6cedf0 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 msgr2=0x55a86d6e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6e6c30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55a86d6ed4a0 tx=0x7f3f7400be60 comp rx=0 tx=0).stop 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 shutdown_connections 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6dad80 0x55a86d6dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6cedf0 0x55a86d6e6c30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 --2- 172.21.15.96:0/3506439458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6ce4d0 0x55a86d6cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 >> 172.21.15.96:0/3506439458 conn(0x55a86d6cc300 msgr2=0x55a86d6d9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 shutdown_connections 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- 172.21.15.96:0/3506439458 wait complete. 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 -- start start 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.666+0000 7f3f8df08740 1 Processor -- start 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 0x55a86d6e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b1920 con 0x55a86d6dad80 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b1250 con 0x55a86d6cedf0 2024-09-10T22:52:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841ae4b0 con 0x55a86d6ce4d0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8df08740 10 client.0 _ll_register_callbacks cb 0x7f3f80009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8a0d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56184/0 (socket says 172.21.15.96:56184) 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8a0d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36588/0 (socket says 172.21.15.96:36588) 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 -- 172.21.15.96:0/1433248133 learned_addr learned my addr 172.21.15.96:0/1433248133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f898cf640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 0x55a86d6e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 -- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 msgr2=0x55a86d6e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 0x55a86d6e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 -- 172.21.15.96:0/1433248133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 msgr2=0x55a86d6ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 -- 172.21.15.96:0/1433248133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86d6de390 con 0x55a86d6cedf0 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f8a0d0640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f898cf640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 0x55a86d6e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f890ce640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004650 tx=0x7f3f74005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833abb5cfcdcd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.667+0000 7f3f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7401d420 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a86d646df0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3f7401d5c0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3f7401d7e0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 10 client.26033 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a86d7581a0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a86d646df0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3f74024ba0 con 0x55a86d6cedf0 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 10 client.26033 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 10 client.26033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 20 client.26033 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d75a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 4 client.26033 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.668+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a86d6428a0 con 0x55a86d6ea880 2024-09-10T22:52:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.669+0000 7f3f898cf640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d75a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.669+0000 7f3f6d7fa640 10 client.26033 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:52:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.669+0000 7f3f898cf640 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d75a880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55ff331972ec30ae server_cookie=cbae6d04a2383237 in_seq=0 out_seq=0 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.671+0000 7f3f6d7fa640 1 -- 172.21.15.96:0/1433248133 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a86d6428a0 con 0x55a86d6ea880 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.671+0000 7f3f6d7fa640 10 client.26033 handle_command_reply: tid=0 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.671+0000 7f3f8df08740 1 client.26033 shutdown 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.671+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 msgr2=0x55a86d6e33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.671+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f74004650 tx=0x7f3f74005760 comp rx=0 tx=0).stop 2024-09-10T22:52:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 shutdown_connections 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55a86d6ea880 0x55a86d75a880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d6cedf0 0x55a86d6e33c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a86d6dad80 0x55a86d6ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 --2- 172.21.15.96:0/1433248133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a86d6ce4d0 0x55a86d6e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f6d7fa640 0 client.26033 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 >> 172.21.15.96:0/1433248133 conn(0x55a86d6cc300 msgr2=0x55a86d6ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 shutdown_connections 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 1 -- 172.21.15.96:0/1433248133 wait complete. 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.672+0000 7f3f8df08740 20 client.26033 trim_cache size 0 max 16384 2024-09-10T22:52:16.677 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 msgr2=0x7f3f700b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 0x7f3f700b78f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c046000 tx=0x7f3f7c0047a0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 msgr2=0x7f3f8419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9f0 tx=0x7f3f8000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 shutdown_connections 2024-09-10T22:52:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f841087f0 0x7f3f8419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3f700b5430 0x7f3f700b78f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f84109170 0x7f3f841a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 --2- 172.21.15.96:0/2507279908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f84102de0 0x7f3f8419b450 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 >> 172.21.15.96:0/2507279908 conn(0x7f3f840fe830 msgr2=0x7f3f84100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 shutdown_connections 2024-09-10T22:52:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:16.673+0000 7f3f8bb33640 1 -- 172.21.15.96:0/2507279908 wait complete. 2024-09-10T22:52:17.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:16 smithi096 ceph-mon[30869]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 779 op/s 2024-09-10T22:52:17.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:16 smithi121 ceph-mon[35119]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 779 op/s 2024-09-10T22:52:17.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:16 smithi117 ceph-mon[35081]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.4 MiB/s wr, 779 op/s 2024-09-10T22:52:17.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:52:17.894 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 -- 172.21.15.96:0/3712040851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 msgr2=0x7facac106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 --2- 172.21.15.96:0/3712040851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac106880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 -- 172.21.15.96:0/3712040851 shutdown_connections 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 --2- 172.21.15.96:0/3712040851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac106ed0 0x7facac107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 --2- 172.21.15.96:0/3712040851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac106880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 --2- 172.21.15.96:0/3712040851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac101a90 0x7facac101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 -- 172.21.15.96:0/3712040851 >> 172.21.15.96:0/3712040851 conn(0x7facac0fd700 msgr2=0x7facac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 -- 172.21.15.96:0/3712040851 shutdown_connections 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.416+0000 7facb3d5d640 1 -- 172.21.15.96:0/3712040851 wait complete. 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.417+0000 7facb3d5d640 1 Processor -- start 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.417+0000 7facb3d5d640 1 -- start start 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.417+0000 7facb3d5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 0x7facac19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.417+0000 7facb3d5d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.417+0000 7facb3d5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb3d5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facac119390 con 0x7facac101a90 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56194/0 (socket says 172.21.15.96:56194) 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 -- 172.21.15.96:0/4100237325 learned_addr learned my addr 172.21.15.96:0/4100237325 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facac119210 con 0x7facac106ed0 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7facac119510 con 0x7facac1023b0 2024-09-10T22:52:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb1af9640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 0x7facac19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb12f8640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 -- 172.21.15.96:0/4100237325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 msgr2=0x7facac19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 -- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 msgr2=0x7facac19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 0x7facac19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 -- 172.21.15.96:0/4100237325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac1a5970 con 0x7facac106ed0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb1af9640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 0x7facac19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.418+0000 7facb22fa640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.419+0000 7faca2ffd640 1 -- 172.21.15.96:0/4100237325 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4004b20 con 0x7facac106ed0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.419+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac1a5c60 con 0x7facac106ed0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.419+0000 7faca2ffd640 1 -- 172.21.15.96:0/4100237325 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca4004cc0 con 0x7facac106ed0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.419+0000 7faca2ffd640 1 -- 172.21.15.96:0/4100237325 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4010670 con 0x7facac106ed0 2024-09-10T22:52:18.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.419+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1ac320 con 0x7facac106ed0 2024-09-10T22:52:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.423+0000 7facb6132740 1 Processor -- start 2024-09-10T22:52:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.423+0000 7faca2ffd640 1 -- 172.21.15.96:0/4100237325 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7faca4005020 con 0x7facac106ed0 2024-09-10T22:52:18.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 -- start start 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84afd80 0x55a4b84b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a4b8419ea0 con 0x55a4b84a3df0 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a4b84092d0 con 0x55a4b84afd80 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb6132740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a4b840d0e0 con 0x55a4b84a34d0 2024-09-10T22:52:18.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb1af9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb1af9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36624/0 (socket says 172.21.15.96:36624) 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb1af9640 1 -- 172.21.15.96:0/1338579255 learned_addr learned my addr 172.21.15.96:0/1338579255 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb12f8640 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84afd80 0x55a4b84b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.424+0000 7facb12f8640 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 -- 172.21.15.96:0/1338579255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 msgr2=0x55a4b84a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 -- 172.21.15.96:0/1338579255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84afd80 msgr2=0x55a4b84b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84afd80 0x55a4b84b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 -- 172.21.15.96:0/1338579255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b8167650 con 0x55a4b84a3df0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7faca2ffd640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 0x7fac980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb1af9640 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faca8011540 tx=0x7faca8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b27b6e4ac1477e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7facb12f8640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 0x7fac980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7faca2ffd640 1 -- 172.21.15.96:0/4100237325 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faca40e60a0 con 0x7facac106ed0 2024-09-10T22:52:18.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7fac97fff640 1 -- 172.21.15.96:0/1338579255 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800f5c0 con 0x55a4b84a3df0 2024-09-10T22:52:18.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7fac97fff640 1 -- 172.21.15.96:0/1338579255 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca800fc10 con 0x55a4b84a3df0 2024-09-10T22:52:18.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.425+0000 7fac97fff640 1 -- 172.21.15.96:0/1338579255 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8019c50 con 0x55a4b84a3df0 2024-09-10T22:52:18.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- 172.21.15.96:0/1338579255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 msgr2=0x55a4b84af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faca8011540 tx=0x7faca8011a00 comp rx=0 tx=0).stop 2024-09-10T22:52:18.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb12f8640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 0x7fac980b7820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac9c031440 tx=0x7fac9c005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- 172.21.15.96:0/1338579255 shutdown_connections 2024-09-10T22:52:18.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84afd80 0x55a4b84b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84af840 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 --2- 172.21.15.96:0/1338579255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- 172.21.15.96:0/1338579255 >> 172.21.15.96:0/1338579255 conn(0x55a4b84a1300 msgr2=0x55a4b84a24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- 172.21.15.96:0/1338579255 shutdown_connections 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- 172.21.15.96:0/1338579255 wait complete. 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 -- start start 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.427+0000 7facb6132740 1 Processor -- start 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a3df0 0x55a4b84ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a4b826a700 con 0x55a4b84afd80 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a4b83a4160 con 0x55a4b84a34d0 2024-09-10T22:52:18.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a4b84b2a00 con 0x55a4b84a3df0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb12f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a3df0 0x55a4b84ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb6132740 10 client.0 _ll_register_callbacks cb 0x55a4b83bd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56208/0 (socket says 172.21.15.96:56208) 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 -- 172.21.15.96:0/3702757047 learned_addr learned my addr 172.21.15.96:0/3702757047 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb22fa640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 -- 172.21.15.96:0/3702757047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a3df0 msgr2=0x55a4b84ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a3df0 0x55a4b84ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 -- 172.21.15.96:0/3702757047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 msgr2=0x55a4b84acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.428+0000 7facb1af9640 1 -- 172.21.15.96:0/3702757047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b83d1c90 con 0x55a4b84a34d0 2024-09-10T22:52:18.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7facb1af9640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faca8009d00 tx=0x7faca8009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2234e1b4f2c9420f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7fac957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800f8d0 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faca8019950 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca800fc10 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faca800fa70 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7facb6132740 10 client.26045 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a4b84be280 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.429+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca800fc10 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faca8004750 con 0x55a4b84a34d0 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 20 client.26045 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b84bf680 0x55a4b84c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 4 client.26045 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:18.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4b841bdf0 con 0x55a4b84bf680 2024-09-10T22:52:18.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb12f8640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b84bf680 0x55a4b84c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7facb12f8640 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b84bf680 0x55a4b84c1a70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ac4608b5a4391b server_cookie=8cae2187458021de in_seq=0 out_seq=0 2024-09-10T22:52:18.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.430+0000 7fac957fa640 10 client.26045 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:18.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.431+0000 7fac957fa640 1 -- 172.21.15.96:0/3702757047 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a4b841bdf0 con 0x55a4b84bf680 2024-09-10T22:52:18.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.431+0000 7fac957fa640 10 client.26045 handle_command_reply: tid=0 2024-09-10T22:52:18.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 client.26045 shutdown 2024-09-10T22:52:18.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 msgr2=0x55a4b84af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faca8009d00 tx=0x7faca8009de0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 shutdown_connections 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b84bf680 0x55a4b84c1a70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a3df0 0x55a4b84ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 --2- 172.21.15.96:0/3702757047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84af590 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 >> 172.21.15.96:0/3702757047 conn(0x55a4b84a1300 msgr2=0x55a4b84a6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.432+0000 7fac957fa640 0 client.26045 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.433+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 shutdown_connections 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.433+0000 7facb6132740 1 -- 172.21.15.96:0/3702757047 wait complete. 2024-09-10T22:52:18.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.433+0000 7facb6132740 20 client.26045 trim_cache size 0 max 16384 2024-09-10T22:52:18.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.452+0000 7facb6132740 1 Processor -- start 2024-09-10T22:52:18.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 -- start start 2024-09-10T22:52:18.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84afd80 0x55a4b84aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b48b0 con 0x55a4b84a3df0 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b63f0 con 0x55a4b84a34d0 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb6132740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b4f80 con 0x55a4b84afd80 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb12f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb12f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36648/0 (socket says 172.21.15.96:36648) 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb12f8640 1 -- 172.21.15.96:0/2569907108 learned_addr learned my addr 172.21.15.96:0/2569907108 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb1af9640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.454+0000 7facb22fa640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84afd80 0x55a4b84aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 -- 172.21.15.96:0/2569907108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84afd80 msgr2=0x55a4b84aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84afd80 0x55a4b84aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 -- 172.21.15.96:0/2569907108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 msgr2=0x55a4b84a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 -- 172.21.15.96:0/2569907108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4b84a1d80 con 0x55a4b84a34d0 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb12f8640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7facb1af9640 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84a2a60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faca80224a0 tx=0x7faca800ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83fb907e61aa9158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7fac977fe640 1 -- 172.21.15.96:0/2569907108 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca800f8d0 con 0x55a4b84a34d0 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7fac977fe640 1 -- 172.21.15.96:0/2569907108 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8027070 con 0x55a4b84a34d0 2024-09-10T22:52:18.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.455+0000 7fac977fe640 1 -- 172.21.15.96:0/2569907108 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80198b0 con 0x55a4b84a34d0 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 -- 172.21.15.96:0/2569907108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 msgr2=0x55a4b84a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84a2a60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faca80224a0 tx=0x7faca800ffd0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 -- 172.21.15.96:0/2569907108 shutdown_connections 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84afd80 0x55a4b84aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84a3df0 0x55a4b84a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 --2- 172.21.15.96:0/2569907108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a34d0 0x55a4b84a2a60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 -- 172.21.15.96:0/2569907108 >> 172.21.15.96:0/2569907108 conn(0x55a4b84a1300 msgr2=0x55a4b84a2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 -- 172.21.15.96:0/2569907108 shutdown_connections 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.456+0000 7facb6132740 1 -- 172.21.15.96:0/2569907108 wait complete. 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 -- start start 2024-09-10T22:52:18.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 Processor -- start 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b5d20 con 0x55a4b84afd80 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b5650 con 0x55a4b84a3df0 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7facac1b2890 con 0x55a4b84a34d0 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb6132740 10 client.0 _ll_register_callbacks cb 0x7faca40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb12f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb22fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb22fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36652/0 (socket says 172.21.15.96:36652) 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb12f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56220/0 (socket says 172.21.15.96:56220) 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.457+0000 7facb22fa640 1 -- 172.21.15.96:0/1660373254 learned_addr learned my addr 172.21.15.96:0/1660373254 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:18.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb1af9640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 -- 172.21.15.96:0/1660373254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 msgr2=0x55a4b84b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 -- 172.21.15.96:0/1660373254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 msgr2=0x55a4b84a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 -- 172.21.15.96:0/1660373254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4b84aa220 con 0x55a4b84a3df0 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb22fa640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb12f8640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac9c004260 tx=0x7fac9c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b943c3f8bae1dc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7fac957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:18.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0314a0 con 0x55a4b84a3df0 2024-09-10T22:52:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac9c031640 con 0x55a4b84a3df0 2024-09-10T22:52:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.458+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca8027070 con 0x55a4b84a3df0 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.459+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac9c065050 con 0x55a4b84a3df0 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.459+0000 7facb6132740 10 client.26051 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.459+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a4b852c330 con 0x55a4b84a3df0 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca8027070 con 0x55a4b84a3df0 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac9c055070 con 0x55a4b84a3df0 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 10 client.26051 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:18.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 20 client.26051 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b852cb30 0x55a4b852ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 4 client.26051 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.460+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4b84178a0 con 0x55a4b852cb30 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.461+0000 7facb1af9640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b852cb30 0x55a4b852ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.462+0000 7fac957fa640 10 client.26051 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.462+0000 7facb1af9640 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b852cb30 0x55a4b852ef50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1e9d71d76b45cf server_cookie=d2717474e366f12c in_seq=0 out_seq=0 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7fac957fa640 1 -- 172.21.15.96:0/1660373254 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4b84178a0 con 0x55a4b852cb30 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7fac957fa640 10 client.26051 handle_command_reply: tid=0 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 client.26051 shutdown 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 msgr2=0x55a4b84a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac9c004260 tx=0x7fac9c02a410 comp rx=0 tx=0).stop 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 shutdown_connections 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a4b852cb30 0x55a4b852ef50 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a4b84a3df0 0x55a4b84a96a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4b84afd80 0x55a4b84a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 --2- 172.21.15.96:0/1660373254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4b84a34d0 0x55a4b84b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7fac957fa640 0 client.26051 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 >> 172.21.15.96:0/1660373254 conn(0x55a4b84a1300 msgr2=0x55a4b849fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 shutdown_connections 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 1 -- 172.21.15.96:0/1660373254 wait complete. 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.463+0000 7facb6132740 20 client.26051 trim_cache size 0 max 16384 2024-09-10T22:52:18.468 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:52:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 msgr2=0x7fac980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 0x7fac980b7820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac9c031440 tx=0x7fac9c005780 comp rx=0 tx=0).stop 2024-09-10T22:52:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 msgr2=0x7facac1a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:18.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 shutdown_connections 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facac106ed0 0x7facac1a51a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac980b5360 0x7fac980b7820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facac1023b0 0x7facac19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 --2- 172.21.15.96:0/4100237325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facac101a90 0x7facac19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 >> 172.21.15.96:0/4100237325 conn(0x7facac0fd700 msgr2=0x7facac1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 shutdown_connections 2024-09-10T22:52:18.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:18.464+0000 7facb3d5d640 1 -- 172.21.15.96:0/4100237325 wait complete. 2024-09-10T22:52:19.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:18 smithi096 ceph-mon[30869]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-10T22:52:19.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:18 smithi121 ceph-mon[35119]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-10T22:52:19.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:18 smithi117 ceph-mon[35081]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 MiB/s wr, 658 op/s 2024-09-10T22:52:20.362 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:52:20.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:52:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 285.0 s, 2534.2 tps, lat 39.571 ms stddev 41.854 2024-09-10T22:52:21.085 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:20 smithi096 ceph-mon[30869]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 850 op/s 2024-09-10T22:52:21.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:20 smithi121 ceph-mon[35119]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 850 op/s 2024-09-10T22:52:21.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:20 smithi117 ceph-mon[35081]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 850 op/s 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 -- 172.21.15.96:0/3255587677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee70101c90 msgr2=0x7fee70102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 --2- 172.21.15.96:0/3255587677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee70101c90 0x7fee70102070 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fee60009900 tx=0x7fee6002a410 comp rx=0 tx=0).stop 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 -- 172.21.15.96:0/3255587677 shutdown_connections 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 --2- 172.21.15.96:0/3255587677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee701076a0 0x7fee70107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 --2- 172.21.15.96:0/3255587677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee7010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 --2- 172.21.15.96:0/3255587677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee70101c90 0x7fee70102070 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.480+0000 7fee77ee3640 1 -- 172.21.15.96:0/3255587677 >> 172.21.15.96:0/3255587677 conn(0x7fee700fd700 msgr2=0x7fee700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.481+0000 7fee77ee3640 1 -- 172.21.15.96:0/3255587677 shutdown_connections 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.481+0000 7fee77ee3640 1 -- 172.21.15.96:0/3255587677 wait complete. 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 Processor -- start 2024-09-10T22:52:21.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 -- start start 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee70101c90 0x7fee7010a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee70108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee70111d30 con 0x7fee701076a0 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee70111bb0 con 0x7fee70101c90 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee77ee3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee70111eb0 con 0x7fee70108020 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee7547e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee7547e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36676/0 (socket says 172.21.15.96:36676) 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.482+0000 7fee7547e640 1 -- 172.21.15.96:0/1125452406 learned_addr learned my addr 172.21.15.96:0/1125452406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee70101c90 0x7fee7010a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee76480640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee70108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 -- 172.21.15.96:0/1125452406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 msgr2=0x7fee70108f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee70108f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 -- 172.21.15.96:0/1125452406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee70101c90 msgr2=0x7fee7010a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee70101c90 0x7fee7010a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 -- 172.21.15.96:0/1125452406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee701abcd0 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee76480640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee70108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee7547e640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00e9f0 tx=0x7fee6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee66ffd640 1 -- 172.21.15.96:0/1125452406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00cd80 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee701abea0 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee66ffd640 1 -- 172.21.15.96:0/1125452406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6c0108e0 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.483+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701ac3a0 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.484+0000 7fee66ffd640 1 -- 172.21.15.96:0/1125452406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c018980 con 0x7fee701076a0 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 Processor -- start 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 -- start start 2024-09-10T22:52:21.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1deab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559c1d52e60 con 0x5559c1de8790 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559c1d42290 con 0x5559c1ddc800 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5559c1d460a0 con 0x5559c1ddbee0 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee76480640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee76480640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51660/0 (socket says 172.21.15.96:51660) 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee75c7f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.487+0000 7fee76480640 1 -- 172.21.15.96:0/2950761038 learned_addr learned my addr 172.21.15.96:0/2950761038 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 -- 172.21.15.96:0/2950761038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 msgr2=0x5559c1de8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 -- 172.21.15.96:0/2950761038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 msgr2=0x5559c1deab80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1deab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 -- 172.21.15.96:0/2950761038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c1aa0930 con 0x5559c1ddbee0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee76480640 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee680117c0 tx=0x7fee68011c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9e762b1047e324 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee66ffd640 1 -- 172.21.15.96:0/1125452406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fee6c010430 con 0x7fee701076a0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee5bfff640 1 -- 172.21.15.96:0/2950761038 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68003c30 con 0x5559c1ddbee0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee5bfff640 1 -- 172.21.15.96:0/2950761038 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6800fe50 con 0x5559c1ddbee0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.488+0000 7fee5bfff640 1 -- 172.21.15.96:0/2950761038 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680193e0 con 0x5559c1ddbee0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee66ffd640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 0x7fee5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- 172.21.15.96:0/2950761038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 msgr2=0x5559c1ddc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee680117c0 tx=0x7fee68011c80 comp rx=0 tx=0).stop 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 0x7fee5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- 172.21.15.96:0/2950761038 shutdown_connections 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1deab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/2950761038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1ddc2c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- 172.21.15.96:0/2950761038 >> 172.21.15.96:0/2950761038 conn(0x5559c1dd9d10 msgr2=0x5559c1ddaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee66ffd640 1 -- 172.21.15.96:0/1125452406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fee6c0e5e40 con 0x7fee701076a0 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- 172.21.15.96:0/2950761038 shutdown_connections 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- 172.21.15.96:0/2950761038 wait complete. 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 -- start start 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 0x7fee5c0b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fee60005ab0 tx=0x7fee60005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.489+0000 7fee7a2b8740 1 Processor -- start 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1de6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559c1ba3890 con 0x5559c1ddc800 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559c1cdcd40 con 0x5559c1de8790 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5559c1deb440 con 0x5559c1ddbee0 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7a2b8740 10 client.0 _ll_register_callbacks cb 0x5559c1d40c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36690/0 (socket says 172.21.15.96:36690) 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 -- 172.21.15.96:0/654786570 learned_addr learned my addr 172.21.15.96:0/654786570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee76480640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1de6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 -- 172.21.15.96:0/654786570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 msgr2=0x5559c1de57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 -- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 msgr2=0x5559c1de6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1de6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.490+0000 7fee7547e640 1 -- 172.21.15.96:0/654786570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c1df4870 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7547e640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ac6e0 tx=0x7fee6c0ac610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66f8f075273fd96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0ac750 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fee6c0abbc0 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6800fe50 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee6c0b1180 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 10 client.16491 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559c1df7220 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6800fe50 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fee6c0b2310 con 0x5559c1ddc800 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 10 client.16491 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:21.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 20 client.16491 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1df86e0 0x5559c1dfaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 4 client.16491 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.491+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c1d54db0 con 0x5559c1df86e0 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.492+0000 7fee75c7f640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1df86e0 0x5559c1dfaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.492+0000 7fee75c7f640 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1df86e0 0x5559c1dfaad0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8601f3d9bc5b1e5f server_cookie=fea6c520f318b5e2 in_seq=0 out_seq=0 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.493+0000 7fee597fa640 10 client.16491 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.493+0000 7fee597fa640 1 -- 172.21.15.96:0/654786570 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559c1d54db0 con 0x5559c1df86e0 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.493+0000 7fee597fa640 10 client.16491 handle_command_reply: tid=0 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.493+0000 7fee7a2b8740 1 client.16491 shutdown 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 msgr2=0x5559c1de5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ac6e0 tx=0x7fee6c0ac610 comp rx=0 tx=0).stop 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 shutdown_connections 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1df86e0 0x5559c1dfaad0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1de6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddc800 0x5559c1de5cf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee597fa640 0 client.16491 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a", 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:52:21.550 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/654786570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 >> 172.21.15.96:0/654786570 conn(0x5559c1dd9d10 msgr2=0x5559c1dd8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 shutdown_connections 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 1 -- 172.21.15.96:0/654786570 wait complete. 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.494+0000 7fee7a2b8740 20 client.16491 trim_cache size 0 max 16384 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 Processor -- start 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 -- start start 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b49f0 con 0x5559c1de8790 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b6530 con 0x5559c1ddc800 2024-09-10T22:52:21.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7a2b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b50c0 con 0x5559c1ddbee0 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee76480640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee76480640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36696/0 (socket says 172.21.15.96:36696) 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee76480640 1 -- 172.21.15.96:0/1321467316 learned_addr learned my addr 172.21.15.96:0/1321467316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.515+0000 7fee7547e640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee76480640 1 -- 172.21.15.96:0/1321467316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 msgr2=0x5559c1de29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee76480640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee76480640 1 -- 172.21.15.96:0/1321467316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 msgr2=0x5559c1de2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee76480640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee76480640 1 -- 172.21.15.96:0/1321467316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c1de6a70 con 0x5559c1de8790 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.516+0000 7fee7547e640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:21.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.517+0000 7fee76480640 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fee6800f790 tx=0x7fee68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c165f5cf14475ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.518+0000 7fee5b7fe640 1 -- 172.21.15.96:0/1321467316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68004050 con 0x5559c1de8790 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.518+0000 7fee5b7fe640 1 -- 172.21.15.96:0/1321467316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee680041f0 con 0x5559c1de8790 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.518+0000 7fee5b7fe640 1 -- 172.21.15.96:0/1321467316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680052b0 con 0x5559c1de8790 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1321467316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 msgr2=0x5559c1e003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fee6800f790 tx=0x7fee68009f90 comp rx=0 tx=0).stop 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1321467316 shutdown_connections 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1ddc800 0x5559c1de2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1de8790 0x5559c1e003d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1321467316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddbee0 0x5559c1de29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1321467316 >> 172.21.15.96:0/1321467316 conn(0x5559c1dd9d10 msgr2=0x5559c1de7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1321467316 shutdown_connections 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1321467316 wait complete. 2024-09-10T22:52:21.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 -- start start 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.519+0000 7fee7a2b8740 1 Processor -- start 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 0x5559c1df0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 0x5559c1def250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee75c7f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 0x5559c1df0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee75c7f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 0x5559c1df0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36706/0 (socket says 172.21.15.96:36706) 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee75c7f640 1 -- 172.21.15.96:0/1117405690 learned_addr learned my addr 172.21.15.96:0/1117405690 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1def790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b5e60 con 0x5559c1ddbee0 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b5790 con 0x5559c1de8790 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee701b29d0 con 0x5559c1ddc800 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7a2b8740 10 client.0 _ll_register_callbacks cb 0x7fee6c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.520+0000 7fee7547e640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 0x5559c1def250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee76480640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1def790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 -- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 msgr2=0x5559c1def790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1def790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 -- 172.21.15.96:0/1117405690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 msgr2=0x5559c1df0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 0x5559c1df0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c1defed0 con 0x5559c1ddc800 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7547e640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 0x5559c1def250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ac450 tx=0x7fee6c0107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8391b8531974137e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee597fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0ac750 con 0x5559c1ddc800 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fee6c018b20 con 0x5559c1ddc800 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee680041f0 con 0x5559c1ddc800 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee6c0b5c30 con 0x5559c1ddc800 2024-09-10T22:52:21.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7a2b8740 10 client.26104 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.521+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559c1e66560 con 0x5559c1ddc800 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee6c0a6e50 con 0x5559c1ddc800 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fee6c0c7260 con 0x5559c1ddc800 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 10 client.26104 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 20 client.26104 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1e66bb0 0x5559c1e68fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 4 client.26104 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fee680041f0 con 0x5559c1e66bb0 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.522+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1e66bb0 0x5559c1e68fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.523+0000 7fee597fa640 10 client.26104 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.523+0000 7fee75c7f640 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1e66bb0 0x5559c1e68fd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb6511fce393e84 server_cookie=86c2f9ca3a30e82a in_seq=0 out_seq=0 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.524+0000 7fee597fa640 1 -- 172.21.15.96:0/1117405690 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fee680041f0 con 0x5559c1e66bb0 2024-09-10T22:52:21.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.524+0000 7fee597fa640 10 client.26104 handle_command_reply: tid=0 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.524+0000 7fee7a2b8740 1 client.26104 shutdown 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.524+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 msgr2=0x5559c1def250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.524+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 0x5559c1def250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0ac450 tx=0x7fee6c0107d0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 shutdown_connections 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5559c1e66bb0 0x5559c1e68fd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559c1de8790 0x5559c1def790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559c1ddc800 0x5559c1def250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 --2- 172.21.15.96:0/1117405690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559c1ddbee0 0x5559c1df0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 >> 172.21.15.96:0/1117405690 conn(0x5559c1dd9d10 msgr2=0x5559c1ddef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee597fa640 0 client.26104 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 shutdown_connections 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 1 -- 172.21.15.96:0/1117405690 wait complete. 2024-09-10T22:52:21.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee7a2b8740 20 client.26104 trim_cache size 0 max 16384 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 msgr2=0x7fee5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.525+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 0x7fee5c0b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fee60005ab0 tx=0x7fee60005a20 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 msgr2=0x7fee70108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00e9f0 tx=0x7fee6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 shutdown_connections 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fee5c0b52e0 0x7fee5c0b77a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee70108020 0x7fee70108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee701076a0 0x7fee70108910 secure :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00e9f0 tx=0x7fee6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 --2- 172.21.15.96:0/1125452406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee70101c90 0x7fee7010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.526+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 >> 172.21.15.96:0/1125452406 conn(0x7fee700fd700 msgr2=0x7fee7006a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.528+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 shutdown_connections 2024-09-10T22:52:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:21.528+0000 7fee77ee3640 1 -- 172.21.15.96:0/1125452406 wait complete. 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 728 op/s 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:52:23.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:22 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 728 op/s 2024-09-10T22:52:23.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:52:23.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:52:23.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:52:23.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:22 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.0 MiB/s wr, 728 op/s 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:52:23.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:22 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:23.454 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:24.243 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:23 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:24.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:23 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:24.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:23 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.360+0000 7fbb804f8640 1 -- 172.21.15.96:0/127735514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78102de0 msgr2=0x7fbb781031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.360+0000 7fbb804f8640 1 --2- 172.21.15.96:0/127735514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78102de0 0x7fbb781031c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009ca0 tx=0x7fbb7402a8f0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.361+0000 7fbb804f8640 1 -- 172.21.15.96:0/127735514 shutdown_connections 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.361+0000 7fbb804f8640 1 --2- 172.21.15.96:0/127735514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb78109170 0x7fbb7810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.361+0000 7fbb804f8640 1 --2- 172.21.15.96:0/127735514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb781087f0 0x7fbb78108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.361+0000 7fbb804f8640 1 --2- 172.21.15.96:0/127735514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78102de0 0x7fbb781031c0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009ca0 tx=0x7fbb7402a8f0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.361+0000 7fbb804f8640 1 -- 172.21.15.96:0/127735514 >> 172.21.15.96:0/127735514 conn(0x7fbb780fe830 msgr2=0x7fbb78100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.363+0000 7fbb804f8640 1 -- 172.21.15.96:0/127735514 shutdown_connections 2024-09-10T22:52:24.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.367+0000 7fbb804f8640 1 -- 172.21.15.96:0/127735514 wait complete. 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.367+0000 7fbb804f8640 1 Processor -- start 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.367+0000 7fbb804f8640 1 -- start start 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 0x7fbb78076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 0x7fbb780754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78075a30 0x7fbb78075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78112f00 con 0x7fbb78109170 2024-09-10T22:52:24.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78112d80 con 0x7fbb78075a30 2024-09-10T22:52:24.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.368+0000 7fbb804f8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78113080 con 0x7fbb781087f0 2024-09-10T22:52:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.369+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 0x7fbb78076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.370+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 0x7fbb78076f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51692/0 (socket says 172.21.15.96:51692) 2024-09-10T22:52:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.370+0000 7fbb7e294640 1 -- 172.21.15.96:0/1359880426 learned_addr learned my addr 172.21.15.96:0/1359880426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:24.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.370+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78075a30 0x7fbb78075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.371+0000 7fbb7da93640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 0x7fbb780754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.372+0000 7fbb7da93640 1 -- 172.21.15.96:0/1359880426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 msgr2=0x7fbb78076f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.372+0000 7fbb7da93640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 0x7fbb78076f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.372+0000 7fbb7da93640 1 -- 172.21.15.96:0/1359880426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78075a30 msgr2=0x7fbb78075eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.372+0000 7fbb7da93640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78075a30 0x7fbb78075eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.372+0000 7fbb7da93640 1 -- 172.21.15.96:0/1359880426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb781ac250 con 0x7fbb78109170 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.375+0000 7fbb7da93640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 0x7fbb780754a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009fd0 tx=0x7fbb7000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.376+0000 7fbb6f7fe640 1 -- 172.21.15.96:0/1359880426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70015020 con 0x7fbb78109170 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.376+0000 7fbb6f7fe640 1 -- 172.21.15.96:0/1359880426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb70011ce0 con 0x7fbb78109170 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.376+0000 7fbb6f7fe640 1 -- 172.21.15.96:0/1359880426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70010e40 con 0x7fbb78109170 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.377+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb781ac540 con 0x7fbb78109170 2024-09-10T22:52:24.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.377+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb781aca80 con 0x7fbb78109170 2024-09-10T22:52:24.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.381+0000 7fbb828cd740 1 Processor -- start 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.385+0000 7fbb828cd740 1 -- start start 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b4bda0 0x563ef2b577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ef2ac1e50 con 0x563ef2b57d30 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ef2ab1280 con 0x563ef2b4b480 2024-09-10T22:52:24.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb828cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ef2ab5090 con 0x563ef2b4bda0 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36730/0 (socket says 172.21.15.96:36730) 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb7da93640 1 -- 172.21.15.96:0/3931337748 learned_addr learned my addr 172.21.15.96:0/3931337748 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.386+0000 7fbb7e294640 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b4bda0 0x563ef2b577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 -- 172.21.15.96:0/3931337748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b4bda0 msgr2=0x563ef2b577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b4bda0 0x563ef2b577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 -- 172.21.15.96:0/3931337748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 msgr2=0x563ef2b4b860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b4b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 -- 172.21.15.96:0/3931337748 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef280f650 con 0x563ef2b57d30 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb7da93640 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ed2f0 tx=0x7fbb700ed7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3709a9a25a99ec3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.387+0000 7fbb6f7fe640 1 -- 172.21.15.96:0/1359880426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbb70022060 con 0x7fbb78109170 2024-09-10T22:52:24.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.388+0000 7fbb6cff9640 1 -- 172.21.15.96:0/3931337748 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700e5560 con 0x563ef2b57d30 2024-09-10T22:52:24.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.391+0000 7fbb6cff9640 1 -- 172.21.15.96:0/3931337748 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb700f2030 con 0x563ef2b57d30 2024-09-10T22:52:24.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.392+0000 7fbb828cd740 1 -- 172.21.15.96:0/3931337748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 msgr2=0x563ef2b5a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.392+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ed2f0 tx=0x7fbb700ed7b0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.398+0000 7fbb6f7fe640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 0x7fbb640b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.399+0000 7fbb6f7fe640 1 -- 172.21.15.96:0/1359880426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbb700e5130 con 0x7fbb78109170 2024-09-10T22:52:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.401+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 0x7fbb640b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.402+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 0x7fbb640b7970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b6d0 tx=0x7fbb68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.405+0000 7fbb828cd740 1 -- 172.21.15.96:0/3931337748 shutdown_connections 2024-09-10T22:52:24.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.405+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b57d30 0x563ef2b5a120 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.405+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b4bda0 0x563ef2b577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.405+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3931337748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.405+0000 7fbb828cd740 1 -- 172.21.15.96:0/3931337748 >> 172.21.15.96:0/3931337748 conn(0x563ef2b492b0 msgr2=0x563ef2b4a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.409+0000 7fbb828cd740 1 -- 172.21.15.96:0/3931337748 shutdown_connections 2024-09-10T22:52:24.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.410+0000 7fbb828cd740 1 -- 172.21.15.96:0/3931337748 wait complete. 2024-09-10T22:52:24.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.410+0000 7fbb828cd740 1 -- start start 2024-09-10T22:52:24.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.410+0000 7fbb828cd740 1 Processor -- start 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 0x563ef2b57560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 0x563ef2b549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 0x563ef2b549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 0x563ef2b549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56276/0 (socket says 172.21.15.96:56276) 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 0x563ef2b57560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 0x563ef2b57560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36742/0 (socket says 172.21.15.96:36742) 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.411+0000 7fbb7e294640 1 -- 172.21.15.96:0/1385881128 learned_addr learned my addr 172.21.15.96:0/1385881128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.412+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.412+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ef2912700 con 0x563ef2b4b480 2024-09-10T22:52:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.412+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ef2a4c110 con 0x563ef2b4bda0 2024-09-10T22:52:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.412+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.412+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ef2b6b840 con 0x563ef2b57d30 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb828cd740 10 client.0 _ll_register_callbacks cb 0x563ef2a65990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb7ea95640 1 -- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 msgr2=0x563ef2b549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 0x563ef2b549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb7ea95640 1 -- 172.21.15.96:0/1385881128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 msgr2=0x563ef2b57560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 0x563ef2b57560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.413+0000 7fbb7ea95640 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef2a79c40 con 0x563ef2b57d30 2024-09-10T22:52:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.414+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b54ef0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc41bddc0f2b43f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68022020 con 0x563ef2b57d30 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb700f2030 con 0x563ef2b57d30 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbb68014960 con 0x563ef2b57d30 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb6801a040 con 0x563ef2b57d30 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb828cd740 10 client.26113 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.417+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ef2b77240 con 0x563ef2b57d30 2024-09-10T22:52:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb700f2030 con 0x563ef2b57d30 2024-09-10T22:52:24.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbb6801d030 con 0x563ef2b57d30 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 10 client.26113 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 10 client.26113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 20 client.26113 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2b78750 0x563ef2b7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 4 client.26113 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.418+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef2ac3da0 con 0x563ef2b78750 2024-09-10T22:52:24.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.419+0000 7fbb7e294640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2b78750 0x563ef2b7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.420+0000 7fbb7e294640 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2b78750 0x563ef2b7ab90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2587f8ad717c6 server_cookie=2d5ed47ca4271ba in_seq=0 out_seq=0 2024-09-10T22:52:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.420+0000 7fbb61ffb640 10 client.26113 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.420+0000 7fbb61ffb640 1 -- 172.21.15.96:0/1385881128 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ef2ac3da0 con 0x563ef2b78750 2024-09-10T22:52:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.420+0000 7fbb61ffb640 10 client.26113 handle_command_reply: tid=0 2024-09-10T22:52:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 client.26113 shutdown 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 msgr2=0x563ef2b54ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b54ef0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68004770 comp rx=0 tx=0).stop 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 shutdown_connections 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2b78750 0x563ef2b7ab90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4bda0 0x563ef2b549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b54ef0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 --2- 172.21.15.96:0/1385881128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4b480 0x563ef2b57560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 >> 172.21.15.96:0/1385881128 conn(0x563ef2b492b0 msgr2=0x563ef2b4ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.421+0000 7fbb61ffb640 0 client.26113 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.422+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 shutdown_connections 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.422+0000 7fbb828cd740 1 -- 172.21.15.96:0/1385881128 wait complete. 2024-09-10T22:52:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.422+0000 7fbb828cd740 20 client.26113 trim_cache size 0 max 16384 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.446+0000 7fbb828cd740 1 Processor -- start 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 -- start start 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b48bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78074e80 con 0x563ef2b4bda0 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781ae3f0 con 0x563ef2b4b480 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb828cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781acf80 con 0x563ef2b57d30 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36752/0 (socket says 172.21.15.96:36752) 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 -- 172.21.15.96:0/2103761148 learned_addr learned my addr 172.21.15.96:0/2103761148 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7e294640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b48bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 -- 172.21.15.96:0/2103761148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 msgr2=0x563ef2b53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 -- 172.21.15.96:0/2103761148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 msgr2=0x563ef2b48bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b48bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7da93640 1 -- 172.21.15.96:0/2103761148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef2b53910 con 0x563ef2b4bda0 2024-09-10T22:52:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.447+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb7da93640 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbb700f65e0 tx=0x7fbb700f6610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641f8ee91744883 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb63fff640 1 -- 172.21.15.96:0/2103761148 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700a7790 con 0x563ef2b4bda0 2024-09-10T22:52:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb63fff640 1 -- 172.21.15.96:0/2103761148 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb700f2070 con 0x563ef2b4bda0 2024-09-10T22:52:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb63fff640 1 -- 172.21.15.96:0/2103761148 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700f63d0 con 0x563ef2b4bda0 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb828cd740 1 -- 172.21.15.96:0/2103761148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 msgr2=0x563ef2b54e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.448+0000 7fbb828cd740 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbb700f65e0 tx=0x7fbb700f6610 comp rx=0 tx=0).stop 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 -- 172.21.15.96:0/2103761148 shutdown_connections 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2b53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b54e70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- 172.21.15.96:0/2103761148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b48bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 -- 172.21.15.96:0/2103761148 >> 172.21.15.96:0/2103761148 conn(0x563ef2b492b0 msgr2=0x563ef2b54760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 -- 172.21.15.96:0/2103761148 shutdown_connections 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 -- 172.21.15.96:0/2103761148 wait complete. 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 -- start start 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 Processor -- start 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.449+0000 7fbb828cd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2bde630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb828cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781add20 con 0x563ef2b4bda0 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb828cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb781ad650 con 0x563ef2b4b480 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb828cd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78072e80 con 0x563ef2b57d30 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb828cd740 10 client.0 _ll_register_callbacks cb 0x7fbb6800b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7da93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56292/0 (socket says 172.21.15.96:56292) 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 -- 172.21.15.96:0/3299119750 learned_addr learned my addr 172.21.15.96:0/3299119750 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7ea95640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2bde630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 -- 172.21.15.96:0/3299119750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 msgr2=0x563ef2bde630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2bde630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 -- 172.21.15.96:0/3299119750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 msgr2=0x563ef2b6d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b6d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.450+0000 7fbb7e294640 1 -- 172.21.15.96:0/3299119750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef2bded70 con 0x563ef2b4b480 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb7e294640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563ef2b7b3d0 tx=0x7fbb740314f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6419c83421150a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74003970 con 0x563ef2b4b480 2024-09-10T22:52:24.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbb74003b10 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb700f2070 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb74003d10 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb828cd740 10 client.26084 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ef2be6110 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.451+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb700f2070 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbb740308c0 con 0x563ef2b4b480 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 10 client.26084 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 20 client.26084 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2be68c0 0x563ef2be8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 4 client.26084 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ef2abf850 con 0x563ef2be68c0 2024-09-10T22:52:24.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb7da93640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2be68c0 0x563ef2be8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb7da93640 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2be68c0 0x563ef2be8ce0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433fe9e4a52ad521 server_cookie=698bb69080d06bc7 in_seq=0 out_seq=0 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb61ffb640 10 client.26084 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.452+0000 7fbb61ffb640 1 -- 172.21.15.96:0/3299119750 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x563ef2abf850 con 0x563ef2be68c0 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb61ffb640 10 client.26084 handle_command_reply: tid=0 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 client.26084 shutdown 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 msgr2=0x563ef2b6d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563ef2b7b3d0 tx=0x7fbb740314f0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 shutdown_connections 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ef2be68c0 0x563ef2be8ce0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb61ffb640 0 client.26084 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ef2b57d30 0x563ef2bde630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ef2b4bda0 0x563ef2b6d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 --2- 172.21.15.96:0/3299119750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ef2b4b480 0x563ef2b6d430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 >> 172.21.15.96:0/3299119750 conn(0x563ef2b492b0 msgr2=0x563ef2b54760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 shutdown_connections 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 1 -- 172.21.15.96:0/3299119750 wait complete. 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.453+0000 7fbb828cd740 20 client.26084 trim_cache size 0 max 16384 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a", 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:52:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:52:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:24.466 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 msgr2=0x7fbb640b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 0x7fbb640b7970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b6d0 tx=0x7fbb68008040 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 msgr2=0x7fbb780754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 0x7fbb780754a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009fd0 tx=0x7fbb7000b870 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 shutdown_connections 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78075a30 0x7fbb78075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbb640b54b0 0x7fbb640b7970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb78109170 0x7fbb780754a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 --2- 172.21.15.96:0/1359880426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb781087f0 0x7fbb78076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 >> 172.21.15.96:0/1359880426 conn(0x7fbb780fe830 msgr2=0x7fbb78100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:24.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 shutdown_connections 2024-09-10T22:52:24.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:24.454+0000 7fbb804f8640 1 -- 172.21.15.96:0/1359880426 wait complete. 2024-09-10T22:52:25.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:24 smithi096 ceph-mon[30869]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.2 MiB/s wr, 751 op/s 2024-09-10T22:52:25.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:24 smithi121 ceph-mon[35119]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.2 MiB/s wr, 751 op/s 2024-09-10T22:52:25.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:24 smithi117 ceph-mon[35081]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.2 MiB/s wr, 751 op/s 2024-09-10T22:52:25.388 INFO:tasks.cephfs.filesystem:scrub status for tag:c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a - {'path': '/', 'tag': 'c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a', 'options': 'recursive,force'} 2024-09-10T22:52:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 290.0 s, 2445.6 tps, lat 40.731 ms stddev 43.309 2024-09-10T22:52:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/3812436438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b41066b0 msgr2=0x7ff0b410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/3812436438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b41066b0 0x7ff0b410d030 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009890 tx=0x7ff0ac02a380 comp rx=0 tx=0).stop 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/3812436438 shutdown_connections 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/3812436438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b41066b0 0x7ff0b410d030 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/3812436438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4105cf0 0x7ff0b4106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/3812436438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b4104ba0 0x7ff0b4104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/3812436438 >> 172.21.15.96:0/3812436438 conn(0x7ff0b4100440 msgr2=0x7ff0b4102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/3812436438 shutdown_connections 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.159+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/3812436438 wait complete. 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 Processor -- start 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 -- start start 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 0x7ff0b419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b41066b0 0x7ff0b419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41107f0 con 0x7ff0b4105cf0 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4110670 con 0x7ff0b41066b0 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0bc4a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4110970 con 0x7ff0b4104ba0 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 0x7ff0b419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:60516/0 (socket says 172.21.15.96:60516) 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 0x7ff0b419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56988/0 (socket says 172.21.15.96:56988) 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0ba245640 1 -- 172.21.15.96:0/82103112 learned_addr learned my addr 172.21.15.96:0/82103112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.160+0000 7ff0baa46640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b41066b0 0x7ff0b419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 -- 172.21.15.96:0/82103112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b41066b0 msgr2=0x7ff0b419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b41066b0 0x7ff0b419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 -- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 msgr2=0x7ff0b419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 0x7ff0b419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 -- 172.21.15.96:0/82103112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b419cab0 con 0x7ff0b4104ba0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0ba245640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ebc0 tx=0x7ff0a400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0b37fe640 1 -- 172.21.15.96:0/82103112 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4010630 con 0x7ff0b4104ba0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0b37fe640 1 -- 172.21.15.96:0/82103112 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a4010c80 con 0x7ff0b4104ba0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0b37fe640 1 -- 172.21.15.96:0/82103112 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4018b60 con 0x7ff0b4104ba0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b419cda0 con 0x7ff0b4104ba0 2024-09-10T22:52:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.161+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41a3470 con 0x7ff0b4104ba0 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.165+0000 7ff0bf275740 1 Processor -- start 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 -- start start 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd18bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd18e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560bcd0fb6f0 con 0x560bcd189860 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560bcd0eab20 con 0x560bcd18c190 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0bf275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560bcd0ee930 con 0x560bcd18ebd0 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0baa46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd18bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0baa46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd18bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56990/0 (socket says 172.21.15.96:56990) 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:60522/0 (socket says 172.21.15.96:60522) 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.166+0000 7ff0baa46640 1 -- 172.21.15.96:0/66893733 learned_addr learned my addr 172.21.15.96:0/66893733 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b9a44640 1 -- 172.21.15.96:0/66893733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 msgr2=0x560bcd18e580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd18e580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b9a44640 1 -- 172.21.15.96:0/66893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 msgr2=0x560bcd18bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd18bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b9a44640 1 -- 172.21.15.96:0/66893733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd0b0530 con 0x560bcd18ebd0 2024-09-10T22:52:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.167+0000 7ff0b37fe640 1 -- 172.21.15.96:0/82103112 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff0a40107d0 con 0x7ff0b4104ba0 2024-09-10T22:52:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.168+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8005fd0 tx=0x7ff0a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4caf460bdbb5fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.168+0000 7ff0b37fe640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 0x7ff09c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.168+0000 7ff0b0bfe640 1 -- 172.21.15.96:0/66893733 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8004020 con 0x560bcd18ebd0 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.168+0000 7ff0b0bfe640 1 -- 172.21.15.96:0/66893733 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a80041c0 con 0x560bcd18ebd0 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.168+0000 7ff0b0bfe640 1 -- 172.21.15.96:0/66893733 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80197a0 con 0x560bcd18ebd0 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.169+0000 7ff0b37fe640 1 -- 172.21.15.96:0/82103112 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff0a40e6150 con 0x7ff0b4104ba0 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.169+0000 7ff0baa46640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 0x7ff09c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.169+0000 7ff0bf275740 1 -- 172.21.15.96:0/66893733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 msgr2=0x560bcd191090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.169+0000 7ff0bf275740 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8005fd0 tx=0x7ff0a8011810 comp rx=0 tx=0).stop 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- 172.21.15.96:0/66893733 shutdown_connections 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd18e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18ebd0 0x560bcd191090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- 172.21.15.96:0/66893733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd18bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- 172.21.15.96:0/66893733 >> 172.21.15.96:0/66893733 conn(0x560bcd1849b0 msgr2=0x560bcd185740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0baa46640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 0x7ff09c0b78a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0095f0 tx=0x7ff0ac02c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- 172.21.15.96:0/66893733 shutdown_connections 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- 172.21.15.96:0/66893733 wait complete. 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- start start 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 Processor -- start 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd192f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd193620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560bcd022ab0 con 0x560bcd18ebd0 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560bcd090730 con 0x560bcd18c190 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560bcd191bb0 con 0x560bcd189860 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.170+0000 7ff0bf275740 10 client.0 _ll_register_callbacks cb 0x560bcd0e46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd192f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56994/0 (socket says 172.21.15.96:56994) 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 -- 172.21.15.96:0/4076497166 learned_addr learned my addr 172.21.15.96:0/4076497166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd193620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 -- 172.21.15.96:0/4076497166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 msgr2=0x560bcd192f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd192f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 -- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 msgr2=0x560bcd193620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd193620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 -- 172.21.15.96:0/4076497166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcce4c790 con 0x560bcd18ebd0 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd193620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0baa46640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac035040 tx=0x7ff0ac02a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe93dc3ec0a98b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac04d020 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff0ac0035f0 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a80041c0 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac037040 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0bf275740 10 client.16524 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.171+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560bccec5620 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a80041c0 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff0ac03e900 con 0x560bcd18ebd0 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 10 client.16524 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 20 client.16524 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd19e880 0x560bcd1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 4 client.16524 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcd0fd640 con 0x560bcd19e880 2024-09-10T22:52:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.172+0000 7ff0ba245640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd19e880 0x560bcd1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.173+0000 7ff099ffb640 10 client.16524 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.173+0000 7ff0ba245640 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd19e880 0x560bcd1a0c70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1c4358f4a57812 server_cookie=88edc3cb3fedd1ad in_seq=0 out_seq=0 2024-09-10T22:52:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff099ffb640 1 -- 172.21.15.96:0/4076497166 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560bcd0fd640 con 0x560bcd19e880 2024-09-10T22:52:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff099ffb640 10 client.16524 handle_command_reply: tid=0 2024-09-10T22:52:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 client.16524 shutdown 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 msgr2=0x560bcd1941f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac035040 tx=0x7ff0ac02a380 comp rx=0 tx=0).stop 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 shutdown_connections 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd19e880 0x560bcd1a0c70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd193620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd1941f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 --2- 172.21.15.96:0/4076497166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd192f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 >> 172.21.15.96:0/4076497166 conn(0x560bcd1849b0 msgr2=0x560bcd183bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff099ffb640 0 client.16524 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 shutdown_connections 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 1 -- 172.21.15.96:0/4076497166 wait complete. 2024-09-10T22:52:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.174+0000 7ff0bf275740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 Processor -- start 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 -- start start 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 0x560bcd1921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18ebd0 0x560bcd192700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41ac050 con 0x560bcd189860 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41adb90 con 0x560bcd18ebd0 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.194+0000 7ff0bf275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41ac720 con 0x560bcd18c190 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 0x560bcd1921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57000/0 (socket says 172.21.15.96:57000) 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 -- 172.21.15.96:0/2691511265 learned_addr learned my addr 172.21.15.96:0/2691511265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0b9a44640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 0x560bcd1921c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:60540/0 (socket says 172.21.15.96:60540) 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0baa46640 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18ebd0 0x560bcd192700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 -- 172.21.15.96:0/2691511265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 msgr2=0x560bcd1921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 0x560bcd1921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 -- 172.21.15.96:0/2691511265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18ebd0 msgr2=0x560bcd192700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18ebd0 0x560bcd192700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 -- 172.21.15.96:0/2691511265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd193990 con 0x560bcd189860 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ac6c0 tx=0x7ff0a40099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74fe596df987277 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff09bfff640 1 -- 172.21.15.96:0/2691511265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40b11e0 con 0x560bcd189860 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff09bfff640 1 -- 172.21.15.96:0/2691511265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a4018420 con 0x560bcd189860 2024-09-10T22:52:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.195+0000 7ff09bfff640 1 -- 172.21.15.96:0/2691511265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a40a8310 con 0x560bcd189860 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 -- 172.21.15.96:0/2691511265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 msgr2=0x560bcd1843e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ac6c0 tx=0x7ff0a40099b0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 -- 172.21.15.96:0/2691511265 shutdown_connections 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18ebd0 0x560bcd192700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd18c190 0x560bcd1921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2691511265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd189860 0x560bcd1843e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 -- 172.21.15.96:0/2691511265 >> 172.21.15.96:0/2691511265 conn(0x560bcd1849b0 msgr2=0x560bcd18efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 -- 172.21.15.96:0/2691511265 shutdown_connections 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.196+0000 7ff0bf275740 1 -- 172.21.15.96:0/2691511265 wait complete. 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 -- start start 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 Processor -- start 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd199130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41ad4c0 con 0x560bcd18ebd0 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41acdf0 con 0x560bcd18c190 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b41aa050 con 0x560bcd189860 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0bf275740 10 client.0 _ll_register_callbacks cb 0x7ff0a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0baa46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0ba245640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:60556/0 (socket says 172.21.15.96:60556) 2024-09-10T22:52:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0ba245640 1 -- 172.21.15.96:0/2154810751 learned_addr learned my addr 172.21.15.96:0/2154810751 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.197+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 -- 172.21.15.96:0/2154810751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 msgr2=0x560bcd197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 -- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 msgr2=0x560bcd1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 -- 172.21.15.96:0/2154810751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd199960 con 0x560bcd18ebd0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0b9a44640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0baa46640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd199130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac03eda0 tx=0x7ff0ac030b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2190961c309c5dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac04d050 con 0x560bcd18ebd0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff0ac032710 con 0x560bcd18ebd0 2024-09-10T22:52:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bcd0f90f0 con 0x560bcd18ebd0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac005850 con 0x560bcd18ebd0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0bf275740 10 client.16530 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560bcd19ad40 con 0x560bcd18ebd0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560bcd0f90f0 con 0x560bcd18ebd0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff0ac064030 con 0x560bcd18ebd0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0bf275740 10 client.16530 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.198+0000 7ff0bf275740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0bf275740 20 client.16530 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd20bc40 0x560bcd20e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0bf275740 4 client.16530 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bcd0f6520 con 0x560bcd20bc40 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd20bc40 0x560bcd20e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff099ffb640 10 client.16530 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.199+0000 7ff0ba245640 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd20bc40 0x560bcd20e060 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03b0845e64f9c51 server_cookie=e12e07b4a54350ca in_seq=0 out_seq=0 2024-09-10T22:52:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff099ffb640 1 -- 172.21.15.96:0/2154810751 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bcd0f6520 con 0x560bcd20bc40 2024-09-10T22:52:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff099ffb640 10 client.16530 handle_command_reply: tid=0 2024-09-10T22:52:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 client.16530 shutdown 2024-09-10T22:52:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 msgr2=0x560bcd199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd199130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac03eda0 tx=0x7ff0ac030b70 comp rx=0 tx=0).stop 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 shutdown_connections 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560bcd20bc40 0x560bcd20e060 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560bcd18c190 0x560bcd1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff099ffb640 0 client.16530 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bcd18ebd0 0x560bcd199130 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 --2- 172.21.15.96:0/2154810751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560bcd189860 0x560bcd197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 >> 172.21.15.96:0/2154810751 conn(0x560bcd1849b0 msgr2=0x560bcd1921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.215+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 shutdown_connections 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bf275740 1 -- 172.21.15.96:0/2154810751 wait complete. 2024-09-10T22:52:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bf275740 20 client.16530 trim_cache size 0 max 16384 2024-09-10T22:52:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:52:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:52:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:52:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 msgr2=0x7ff09c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 0x7ff09c0b78a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0095f0 tx=0x7ff0ac02c040 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 msgr2=0x7ff0b419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ebc0 tx=0x7ff0a400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 shutdown_connections 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b41066b0 0x7ff0b419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff09c0b53e0 0x7ff09c0b78a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0b4105cf0 0x7ff0b419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 --2- 172.21.15.96:0/82103112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0b4104ba0 0x7ff0b419af70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 >> 172.21.15.96:0/82103112 conn(0x7ff0b4100440 msgr2=0x7ff0b4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 shutdown_connections 2024-09-10T22:52:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:26.216+0000 7ff0bc4a9640 1 -- 172.21.15.96:0/82103112 wait complete. 2024-09-10T22:52:27.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:26 smithi096 ceph-mon[30869]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-10T22:52:27.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:26 smithi121 ceph-mon[35119]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-10T22:52:27.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:26 smithi117 ceph-mon[35081]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-10T22:52:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:28 smithi096 ceph-mon[30869]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.1 MiB/s wr, 703 op/s 2024-09-10T22:52:29.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:28 smithi121 ceph-mon[35119]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.1 MiB/s wr, 703 op/s 2024-09-10T22:52:29.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:28 smithi117 ceph-mon[35081]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.1 MiB/s wr, 703 op/s 2024-09-10T22:52:30.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:29 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:52:30.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:29 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:52:30.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:29 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:52:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 295.0 s, 2664.2 tps, lat 37.588 ms stddev 41.695 2024-09-10T22:52:31.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:30 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:52:31.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:30 smithi096 ceph-mon[30869]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.3 MiB/s wr, 921 op/s 2024-09-10T22:52:31.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:30 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:52:31.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:30 smithi121 ceph-mon[35119]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.3 MiB/s wr, 921 op/s 2024-09-10T22:52:31.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:30 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:52:31.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:30 smithi117 ceph-mon[35081]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.3 MiB/s wr, 921 op/s 2024-09-10T22:52:33.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:32 smithi096 ceph-mon[30869]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-09-10T22:52:33.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:32 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:33.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:32 smithi121 ceph-mon[35119]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-09-10T22:52:33.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:32 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:33.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:32 smithi117 ceph-mon[35081]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-09-10T22:52:33.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:32 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:52:34.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:34.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:34.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:33 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:52:34.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:34.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:34.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:33 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:52:34.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:52:34.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:52:34.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:33 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:52:35.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:34 smithi096 ceph-mon[30869]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 811 op/s 2024-09-10T22:52:35.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:34 smithi121 ceph-mon[35119]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 811 op/s 2024-09-10T22:52:35.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:34 smithi117 ceph-mon[35081]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 811 op/s 2024-09-10T22:52:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 300.0 s, 2944.2 tps, lat 34.091 ms stddev 34.708 2024-09-10T22:52:37.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:36 smithi096 ceph-mon[30869]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-09-10T22:52:37.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:36 smithi121 ceph-mon[35119]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-09-10T22:52:37.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:36 smithi117 ceph-mon[35081]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-09-10T22:52:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:38 smithi096 ceph-mon[30869]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-09-10T22:52:39.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:38 smithi121 ceph-mon[35119]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-09-10T22:52:39.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:38 smithi117 ceph-mon[35081]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 -- 172.21.15.96:0/677065899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c108680 msgr2=0x7f427c10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 --2- 172.21.15.96:0/677065899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c108680 0x7f427c10ab40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f427402a8d0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 -- 172.21.15.96:0/677065899 shutdown_connections 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 --2- 172.21.15.96:0/677065899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c105ad0 0x7f427c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 --2- 172.21.15.96:0/677065899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c108680 0x7f427c10ab40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 --2- 172.21.15.96:0/677065899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f427c06bc90 0x7f427c105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 -- 172.21.15.96:0/677065899 >> 172.21.15.96:0/677065899 conn(0x7f427c0fd130 msgr2=0x7f427c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 -- 172.21.15.96:0/677065899 shutdown_connections 2024-09-10T22:52:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.340+0000 7f42816fe640 1 -- 172.21.15.96:0/677065899 wait complete. 2024-09-10T22:52:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.341+0000 7f42816fe640 1 Processor -- start 2024-09-10T22:52:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.341+0000 7f42816fe640 1 -- start start 2024-09-10T22:52:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.342+0000 7f42816fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f42816fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f427c105ad0 0x7f427c10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f42816fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 0x7f427c10f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f42816fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f427c101a80 con 0x7f427c105ad0 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f42816fe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f427c101900 con 0x7f427c108680 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f42816fe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c101c00 con 0x7f427c06bc90 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36298/0 (socket says 172.21.15.96:36298) 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 -- 172.21.15.96:0/965307398 learned_addr learned my addr 172.21.15.96:0/965307398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427b7fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 0x7f427c10f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 -- 172.21.15.96:0/965307398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 msgr2=0x7f427c10f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 0x7f427c10f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 -- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f427c105ad0 msgr2=0x7f427c10e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f427c105ad0 0x7f427c10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 -- 172.21.15.96:0/965307398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c10fe50 con 0x7f427c06bc90 2024-09-10T22:52:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427b7fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 0x7f427c10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.343+0000 7f427affd640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d950 tx=0x7f426c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.344+0000 7f4263fff640 1 -- 172.21.15.96:0/965307398 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c017070 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.344+0000 7f4263fff640 1 -- 172.21.15.96:0/965307398 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426c009c30 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.344+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c110140 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.344+0000 7f4263fff640 1 -- 172.21.15.96:0/965307398 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c00f620 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.344+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c110860 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.346+0000 7f4263fff640 1 -- 172.21.15.96:0/965307398 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f426c010460 con 0x7f427c06bc90 2024-09-10T22:52:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.346+0000 7f42844d0740 1 Processor -- start 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.346+0000 7f42844d0740 1 -- start start 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f42844d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f42844d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab7f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f42844d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f427affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab7f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f42844d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560eab75e6f0 con 0x560eab7ef190 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f42844d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560eab74db20 con 0x560eab7f1bd0 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f427a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f42844d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560eab751930 con 0x560eab7ec860 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.347+0000 7f427affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab7f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35390/0 (socket says 172.21.15.96:35390) 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:47656/0 (socket says 172.21.15.96:47656) 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427affd640 1 -- 172.21.15.96:0/3809007862 learned_addr learned my addr 172.21.15.96:0/3809007862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f4263fff640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 0x7f42640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 -- 172.21.15.96:0/3809007862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 msgr2=0x560eab7eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 -- 172.21.15.96:0/3809007862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 msgr2=0x560eab7f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab7f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 -- 172.21.15.96:0/3809007862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eab713530 con 0x560eab7f1bd0 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427b7fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 0x7f42640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f4263fff640 1 -- 172.21.15.96:0/965307398 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f426c0e5880 con 0x7f427c06bc90 2024-09-10T22:52:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.348+0000 7f427a7fc640 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4270009720 tx=0x7f4270009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc3d411702300fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42617fa640 1 -- 172.21.15.96:0/3809007862 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270015020 con 0x560eab7f1bd0 2024-09-10T22:52:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42617fa640 1 -- 172.21.15.96:0/3809007862 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4270009e30 con 0x560eab7f1bd0 2024-09-10T22:52:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42617fa640 1 -- 172.21.15.96:0/3809007862 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270010380 con 0x560eab7f1bd0 2024-09-10T22:52:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f427b7fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 0x7f42640b77a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4274031040 tx=0x7f42740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 -- 172.21.15.96:0/3809007862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 msgr2=0x560eab7f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4270009720 tx=0x7f4270009be0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 -- 172.21.15.96:0/3809007862 shutdown_connections 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab7f4090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab7f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 --2- 172.21.15.96:0/3809007862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 -- 172.21.15.96:0/3809007862 >> 172.21.15.96:0/3809007862 conn(0x560eab7e79b0 msgr2=0x560eab7e8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 -- 172.21.15.96:0/3809007862 shutdown_connections 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.349+0000 7f42844d0740 1 -- 172.21.15.96:0/3809007862 wait complete. 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 -- start start 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 Processor -- start 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab806eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab807550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560eab685ab0 con 0x560eab7f1bd0 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560eab6f3730 con 0x560eab7ef190 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560eab805bc0 con 0x560eab7ec860 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f42844d0740 10 client.0 _ll_register_callbacks cb 0x560eab7476b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab807550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35406/0 (socket says 172.21.15.96:35406) 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 -- 172.21.15.96:0/1546702569 learned_addr learned my addr 172.21.15.96:0/1546702569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427affd640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab806eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 -- 172.21.15.96:0/1546702569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 msgr2=0x560eab806eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab806eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 -- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 msgr2=0x560eab807550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab807550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427b7fe640 1 -- 172.21.15.96:0/1546702569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eab4af790 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.350+0000 7f427a7fc640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab807550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f427b7fe640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4274035000 tx=0x7f427402a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60940cd0ad5c1114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274003d80 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4274004050 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4270009e30 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f427402f3b0 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 10 client.16539 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560eab528620 con 0x560eab7f1bd0 2024-09-10T22:52:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4270009e30 con 0x560eab7f1bd0 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f427402fa30 con 0x560eab7f1bd0 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 10 client.16539 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.351+0000 7f42844d0740 20 client.16539 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.352+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab814ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.352+0000 7f42844d0740 4 client.16539 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.352+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eab760640 con 0x560eab812820 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.352+0000 7f427affd640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab814ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.353+0000 7f427affd640 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab814ca0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758623059cb3b0f server_cookie=bd512a53c22207aa in_seq=0 out_seq=0 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.353+0000 7f425eff5640 10 client.16539 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f425eff5640 1 -- 172.21.15.96:0/1546702569 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560eab760640 con 0x560eab812820 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f425eff5640 10 client.16539 handle_command_reply: tid=0 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 client.16539 shutdown 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 msgr2=0x560eab808120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4274035000 tx=0x7f427402a8d0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 shutdown_connections 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab814ca0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab807550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f425eff5640 0 client.16539 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab808120 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 --2- 172.21.15.96:0/1546702569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab806eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 >> 172.21.15.96:0/1546702569 conn(0x560eab7e79b0 msgr2=0x560eab7e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 shutdown_connections 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 1 -- 172.21.15.96:0/1546702569 wait complete. 2024-09-10T22:52:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.354+0000 7f42844d0740 20 client.16539 trim_cache size 0 max 16384 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 Processor -- start 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 -- start start 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab7f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f427c073be0 con 0x560eab7f1bd0 2024-09-10T22:52:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1a9720 con 0x560eab7ef190 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.378+0000 7f42844d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c0742b0 con 0x560eab7ec860 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35420/0 (socket says 172.21.15.96:35420) 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 -- 172.21.15.96:0/3602343836 learned_addr learned my addr 172.21.15.96:0/3602343836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427affd640 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427a7fc640 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab7f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 -- 172.21.15.96:0/3602343836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 msgr2=0x560eab7f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 -- 172.21.15.96:0/3602343836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 msgr2=0x560eab7f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab7f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 -- 172.21.15.96:0/3602343836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eab7eefc0 con 0x560eab7f1bd0 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.379+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f427402fe20 tx=0x7f4274031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cad41a632aef5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.380+0000 7f4260ff9640 1 -- 172.21.15.96:0/3602343836 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274047630 con 0x560eab7f1bd0 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.380+0000 7f4260ff9640 1 -- 172.21.15.96:0/3602343836 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427404c030 con 0x560eab7f1bd0 2024-09-10T22:52:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.380+0000 7f4260ff9640 1 -- 172.21.15.96:0/3602343836 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274047db0 con 0x560eab7f1bd0 2024-09-10T22:52:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- 172.21.15.96:0/3602343836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 msgr2=0x560eab805f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f427402fe20 tx=0x7f4274031930 comp rx=0 tx=0).stop 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- 172.21.15.96:0/3602343836 shutdown_connections 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7ef190 0x560eab7f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7f1bd0 0x560eab805f60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- 172.21.15.96:0/3602343836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab7f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- 172.21.15.96:0/3602343836 >> 172.21.15.96:0/3602343836 conn(0x560eab7e79b0 msgr2=0x560eab7e2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- 172.21.15.96:0/3602343836 shutdown_connections 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- 172.21.15.96:0/3602343836 wait complete. 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 -- start start 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 Processor -- start 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab80e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.381+0000 7f42844d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab80d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f42844d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab80db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f427affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab80e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f427affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab80e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36320/0 (socket says 172.21.15.96:36320) 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f427affd640 1 -- 172.21.15.96:0/3417196179 learned_addr learned my addr 172.21.15.96:0/3417196179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab80db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f427a7fc640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab80d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f427c075050 con 0x560eab7ef190 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f427c074980 con 0x560eab7f1bd0 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c071bc0 con 0x560eab7ec860 2024-09-10T22:52:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.382+0000 7f42844d0740 10 client.0 _ll_register_callbacks cb 0x7f4274009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 -- 172.21.15.96:0/3417196179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 msgr2=0x560eab80e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab80e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 -- 172.21.15.96:0/3417196179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 msgr2=0x560eab80d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab80d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eab7f18f0 con 0x560eab7f1bd0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f427b7fe640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab80db90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f4274047d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9287c999576c34b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f425eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427402fa30 con 0x560eab7f1bd0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f427402fbd0 con 0x560eab7f1bd0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.383+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427404c030 con 0x560eab7f1bd0 2024-09-10T22:52:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f427403d420 con 0x560eab7f1bd0 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 10 client.26114 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560eab80d050 con 0x560eab7f1bd0 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427404c030 con 0x560eab7f1bd0 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4274072420 con 0x560eab7f1bd0 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 10 client.26114 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 20 client.26114 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab881ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 4 client.26114 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560eab75c0f0 con 0x560eab812820 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.384+0000 7f427affd640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab881ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.385+0000 7f425eff5640 10 client.26114 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.385+0000 7f427affd640 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab881ea0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2f70a9785dfc11 server_cookie=96348e06cfe108a4 in_seq=0 out_seq=0 2024-09-10T22:52:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f425eff5640 1 -- 172.21.15.96:0/3417196179 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560eab75c0f0 con 0x560eab812820 2024-09-10T22:52:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f425eff5640 10 client.26114 handle_command_reply: tid=0 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 client.26114 shutdown 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 msgr2=0x560eab80db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab80db90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4274009ca0 tx=0x7f4274047d40 comp rx=0 tx=0).stop 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 shutdown_connections 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560eab812820 0x560eab881ea0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560eab7f1bd0 0x560eab80db90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560eab7ef190 0x560eab80d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f425eff5640 0 client.26114 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 --2- 172.21.15.96:0/3417196179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560eab7ec860 0x560eab80e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.406+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 >> 172.21.15.96:0/3417196179 conn(0x560eab7e79b0 msgr2=0x560eab7ebd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.407+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 shutdown_connections 2024-09-10T22:52:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.407+0000 7f42844d0740 1 -- 172.21.15.96:0/3417196179 wait complete. 2024-09-10T22:52:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.407+0000 7f42844d0740 20 client.26114 trim_cache size 0 max 16384 2024-09-10T22:52:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:52:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:52:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:52:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 msgr2=0x7f42640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 0x7f42640b77a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4274031040 tx=0x7f42740095c0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 msgr2=0x7f427c10e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d950 tx=0x7f426c00de10 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 shutdown_connections 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f427c108680 0x7f427c10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42640b52e0 0x7f42640b77a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f427c105ad0 0x7f427c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 --2- 172.21.15.96:0/965307398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c06bc90 0x7f427c10e310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.408+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 >> 172.21.15.96:0/965307398 conn(0x7f427c0fd130 msgr2=0x7f427c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.409+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 shutdown_connections 2024-09-10T22:52:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:39.409+0000 7f42816fe640 1 -- 172.21.15.96:0/965307398 wait complete. 2024-09-10T22:52:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 305.0 s, 3314.2 tps, lat 30.084 ms stddev 33.238 2024-09-10T22:52:41.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:40 smithi096 ceph-mon[30869]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 853 op/s 2024-09-10T22:52:41.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:40 smithi121 ceph-mon[35119]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 853 op/s 2024-09-10T22:52:41.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:40 smithi117 ceph-mon[35081]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 853 op/s 2024-09-10T22:52:43.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:42 smithi096 ceph-mon[30869]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-09-10T22:52:43.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:42 smithi121 ceph-mon[35119]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-09-10T22:52:43.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:42 smithi117 ceph-mon[35081]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-09-10T22:52:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:44 smithi096 ceph-mon[30869]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 713 op/s 2024-09-10T22:52:45.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:44 smithi121 ceph-mon[35119]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 713 op/s 2024-09-10T22:52:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:44 smithi117 ceph-mon[35081]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.6 MiB/s wr, 713 op/s 2024-09-10T22:52:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 310.0 s, 3030.7 tps, lat 33.033 ms stddev 35.273 2024-09-10T22:52:47.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:46 smithi096 ceph-mon[30869]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-09-10T22:52:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:46 smithi121 ceph-mon[35119]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-09-10T22:52:47.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:46 smithi117 ceph-mon[35081]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-09-10T22:52:49.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:48 smithi096 ceph-mon[30869]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-09-10T22:52:49.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:48 smithi121 ceph-mon[35119]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-09-10T22:52:49.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:48 smithi117 ceph-mon[35081]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-09-10T22:52:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 315.0 s, 2965.8 tps, lat 33.603 ms stddev 38.758 2024-09-10T22:52:51.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:50 smithi096 ceph-mon[30869]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 809 op/s 2024-09-10T22:52:51.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:50 smithi121 ceph-mon[35119]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 809 op/s 2024-09-10T22:52:51.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:50 smithi117 ceph-mon[35081]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.1 MiB/s wr, 809 op/s 2024-09-10T22:52:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 -- 172.21.15.96:0/3044481755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9124106290 msgr2=0x7f9124106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 --2- 172.21.15.96:0/3044481755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9124106290 0x7f9124106740 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f911c009ca0 tx=0x7f911c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 -- 172.21.15.96:0/3044481755 shutdown_connections 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 --2- 172.21.15.96:0/3044481755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9124106290 0x7f9124106740 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 --2- 172.21.15.96:0/3044481755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91241036e0 0x7f9124105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 --2- 172.21.15.96:0/3044481755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f9124103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.533+0000 7f9129fa1640 1 -- 172.21.15.96:0/3044481755 >> 172.21.15.96:0/3044481755 conn(0x7f91240fcf30 msgr2=0x7f91240ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.534+0000 7f9129fa1640 1 -- 172.21.15.96:0/3044481755 shutdown_connections 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.534+0000 7f9129fa1640 1 -- 172.21.15.96:0/3044481755 wait complete. 2024-09-10T22:52:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.534+0000 7f9129fa1640 1 Processor -- start 2024-09-10T22:52:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.534+0000 7f9129fa1640 1 -- start start 2024-09-10T22:52:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 0x7f912419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 0x7f91241a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91241149f0 con 0x7f91241036e0 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9124114870 con 0x7f912406b470 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9129fa1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9124114b70 con 0x7f9124106290 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9128f9f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9128f9f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33460/0 (socket says 172.21.15.96:33460) 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9128f9f640 1 -- 172.21.15.96:0/2345087950 learned_addr learned my addr 172.21.15.96:0/2345087950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f91297a0640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 0x7f91241a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9123fff640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 0x7f912419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9128f9f640 1 -- 172.21.15.96:0/2345087950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 msgr2=0x7f91241a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.535+0000 7f9128f9f640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 0x7f91241a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9128f9f640 1 -- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 msgr2=0x7f912419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9128f9f640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 0x7f912419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9128f9f640 1 -- 172.21.15.96:0/2345087950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91241a0c60 con 0x7f912406b470 2024-09-10T22:52:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9123fff640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 0x7f912419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:52:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f91297a0640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 0x7f91241a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9128f9f640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f911800e9f0 tx=0x7f911800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9121ffb640 1 -- 172.21.15.96:0/2345087950 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911800cd80 con 0x7f912406b470 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9121ffb640 1 -- 172.21.15.96:0/2345087950 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91180108e0 con 0x7f912406b470 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.536+0000 7f9121ffb640 1 -- 172.21.15.96:0/2345087950 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118018980 con 0x7f912406b470 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.537+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91241a0f50 con 0x7f912406b470 2024-09-10T22:52:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.537+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9124107940 con 0x7f912406b470 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.541+0000 7f912dfd6740 1 Processor -- start 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.541+0000 7f912dfd6740 1 -- start start 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb92f6f0 con 0x55f9eb9bda60 2024-09-10T22:52:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb91eb20 con 0x55f9eb9c2dd0 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f912dfd6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb922930 con 0x55f9eb9c0390 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f9123fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f9123fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33462/0 (socket says 172.21.15.96:33462) 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f9123fff640 1 -- 172.21.15.96:0/2955915712 learned_addr learned my addr 172.21.15.96:0/2955915712 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.542+0000 7f91297a0640 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 -- 172.21.15.96:0/2955915712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 msgr2=0x55f9eb9c2780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 -- 172.21.15.96:0/2955915712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 msgr2=0x55f9eb9c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 -- 172.21.15.96:0/2955915712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9eb8e4530 con 0x55f9eb9bda60 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f9121ffb640 1 -- 172.21.15.96:0/2345087950 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9118010430 con 0x7f912406b470 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f91297a0640 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9bfe50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f911c005e30 tx=0x7f911c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8f9ba50a2ac65e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f910effd640 1 -- 172.21.15.96:0/2955915712 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c031650 con 0x55f9eb9bda60 2024-09-10T22:52:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f910effd640 1 -- 172.21.15.96:0/2955915712 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f911c036040 con 0x55f9eb9bda60 2024-09-10T22:52:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.543+0000 7f910effd640 1 -- 172.21.15.96:0/2955915712 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c0045d0 con 0x55f9eb9bda60 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.544+0000 7f912dfd6740 1 -- 172.21.15.96:0/2955915712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 msgr2=0x55f9eb9bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.544+0000 7f912dfd6740 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9bfe50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f911c005e30 tx=0x7f911c009380 comp rx=0 tx=0).stop 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.544+0000 7f9121ffb640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 0x7f91140b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.544+0000 7f912dfd6740 1 -- 172.21.15.96:0/2955915712 shutdown_connections 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.544+0000 7f912dfd6740 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 --2- 172.21.15.96:0/2955915712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9bfe50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 -- 172.21.15.96:0/2955915712 >> 172.21.15.96:0/2955915712 conn(0x55f9eb9b8bb0 msgr2=0x55f9eb9b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 -- 172.21.15.96:0/2955915712 shutdown_connections 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f9128f9f640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 0x7f91140b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 -- 172.21.15.96:0/2955915712 wait complete. 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f9121ffb640 1 -- 172.21.15.96:0/2345087950 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f91180e5e40 con 0x7f912406b470 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 -- start start 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f9128f9f640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 0x7f91140b78f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9118058330 tx=0x7f9118070f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.545+0000 7f912dfd6740 1 Processor -- start 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb856ab0 con 0x55f9eb9bda60 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb8c4730 con 0x55f9eb9c2dd0 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f91297a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f9eb9c5bd0 con 0x55f9eb9c0390 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f91297a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53758/0 (socket says 172.21.15.96:53758) 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f91297a0640 1 -- 172.21.15.96:0/791662408 learned_addr learned my addr 172.21.15.96:0/791662408 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f9123fff640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f91297a0640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.546+0000 7f912dfd6740 10 client.0 _ll_register_callbacks cb 0x55f9eb9186b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.547+0000 7f91297a0640 1 -- 172.21.15.96:0/791662408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 msgr2=0x55f9eb9c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.547+0000 7f91297a0640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.547+0000 7f91297a0640 1 -- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 msgr2=0x55f9eb9c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.547+0000 7f91297a0640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.547+0000 7f91297a0640 1 -- 172.21.15.96:0/791662408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9eb680790 con 0x55f9eb9bda60 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f91297a0640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f911c02f470 tx=0x7f911c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d526786e4d2ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f9103fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:52:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c05a060 con 0x55f9eb9bda60 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f911c003c30 con 0x55f9eb9bda60 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911c036040 con 0x55f9eb9bda60 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f911c003e30 con 0x55f9eb9bda60 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f912dfd6740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.548+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9eb6f9620 con 0x55f9eb9bda60 2024-09-10T22:52:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f911c036040 con 0x55f9eb9bda60 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f911c044070 con 0x55f9eb9bda60 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 20 client.16557 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eb9d2500 0x55f9eb9d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 4 client.16557 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9eb931640 con 0x55f9eb9d2500 2024-09-10T22:52:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.549+0000 7f9123fff640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eb9d2500 0x55f9eb9d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.550+0000 7f9103fff640 10 client.16557 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.550+0000 7f9123fff640 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eb9d2500 0x55f9eb9d48f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe79d8e5e973ee9 server_cookie=cfbe96729417bbe in_seq=0 out_seq=0 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.551+0000 7f9103fff640 1 -- 172.21.15.96:0/791662408 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9eb931640 con 0x55f9eb9d2500 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.551+0000 7f9103fff640 10 client.16557 handle_command_reply: tid=0 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.551+0000 7f912dfd6740 1 client.16557 shutdown 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.552+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 msgr2=0x55f9eb9c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.552+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f911c02f470 tx=0x7f911c0093f0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.552+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 shutdown_connections 2024-09-10T22:52:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eb9d2500 0x55f9eb9d48f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 --2- 172.21.15.96:0/791662408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c6f70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 >> 172.21.15.96:0/791662408 conn(0x55f9eb9b8bb0 msgr2=0x55f9eb9b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f9103fff640 0 client.16557 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 shutdown_connections 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 1 -- 172.21.15.96:0/791662408 wait complete. 2024-09-10T22:52:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.553+0000 7f912dfd6740 20 client.16557 trim_cache size 0 max 16384 2024-09-10T22:52:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 Processor -- start 2024-09-10T22:52:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 -- start start 2024-09-10T22:52:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91241a7990 con 0x55f9eb9c2dd0 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91241a94d0 con 0x55f9eb9bda60 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f912dfd6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91241a8060 con 0x55f9eb9c0390 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.576+0000 7f91297a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f9123fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33492/0 (socket says 172.21.15.96:33492) 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 -- 172.21.15.96:0/190072159 learned_addr learned my addr 172.21.15.96:0/190072159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 -- 172.21.15.96:0/190072159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 msgr2=0x55f9eb9c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 -- 172.21.15.96:0/190072159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 msgr2=0x55f9eb9bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f91297a0640 1 -- 172.21.15.96:0/190072159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9eb9c01c0 con 0x55f9eb9bda60 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.577+0000 7f9123fff640 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.578+0000 7f91297a0640 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c02f530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8a9be799db3321 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.578+0000 7f910e7fc640 1 -- 172.21.15.96:0/190072159 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c003c30 con 0x55f9eb9bda60 2024-09-10T22:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.579+0000 7f910e7fc640 1 -- 172.21.15.96:0/190072159 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f911c036070 con 0x55f9eb9bda60 2024-09-10T22:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.579+0000 7f910e7fc640 1 -- 172.21.15.96:0/190072159 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c02f8c0 con 0x55f9eb9bda60 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- 172.21.15.96:0/190072159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 msgr2=0x55f9eb9c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c02f530 comp rx=0 tx=0).stop 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- 172.21.15.96:0/190072159 shutdown_connections 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 --2- 172.21.15.96:0/190072159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9c31b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- 172.21.15.96:0/190072159 >> 172.21.15.96:0/190072159 conn(0x55f9eb9b8bb0 msgr2=0x55f9eb9b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- 172.21.15.96:0/190072159 shutdown_connections 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- 172.21.15.96:0/190072159 wait complete. 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 -- start start 2024-09-10T22:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.580+0000 7f912dfd6740 1 Processor -- start 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f91241a8e00 con 0x55f9eb9c2dd0 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91241a8730 con 0x55f9eb9bda60 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f912410e490 con 0x55f9eb9c0390 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f912dfd6740 10 client.0 _ll_register_callbacks cb 0x7f91100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f91297a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f91297a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33502/0 (socket says 172.21.15.96:33502) 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f91297a0640 1 -- 172.21.15.96:0/1304426141 learned_addr learned my addr 172.21.15.96:0/1304426141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f9123fff640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.581+0000 7f91297a0640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 -- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 msgr2=0x55f9eb9ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 -- 172.21.15.96:0/1304426141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 msgr2=0x55f9eb9cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 -- 172.21.15.96:0/1304426141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9eb9ce3a0 con 0x55f9eb9c0390 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f91297a0640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9123fff640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9cd680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f9eb9b95a0 tx=0x7f91100055d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ce73fd0b0ac1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911004b060 con 0x55f9eb9c0390 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9110033730 con 0x55f9eb9c0390 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911c036070 con 0x55f9eb9c0390 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9110033930 con 0x55f9eb9c0390 2024-09-10T22:52:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f912dfd6740 10 client.26170 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9eba50480 con 0x55f9eb9c0390 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f911c036070 con 0x55f9eb9c0390 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.582+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9110050070 con 0x55f9eb9c0390 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 10 client.26170 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 10 client.26170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 20 client.26170 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eba50e10 0x55f9eba53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 4 client.26170 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9eb92d0f0 con 0x55f9eba50e10 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f91297a0640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eba50e10 0x55f9eba53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f9103fff640 10 client.26170 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.583+0000 7f91297a0640 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eba50e10 0x55f9eba53230 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e1bf912449df24 server_cookie=b52673740a348289 in_seq=0 out_seq=0 2024-09-10T22:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.594+0000 7f9103fff640 1 -- 172.21.15.96:0/1304426141 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9eb92d0f0 con 0x55f9eba50e10 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f9103fff640 10 client.26170 handle_command_reply: tid=0 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 client.26170 shutdown 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 msgr2=0x55f9eb9cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9cd680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f9eb9b95a0 tx=0x7f91100055d0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 shutdown_connections 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9eba50e10 0x55f9eba53230 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9eb9c2dd0 0x55f9eb9cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9eb9c0390 0x55f9eb9cd680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 --2- 172.21.15.96:0/1304426141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9eb9bda60 0x55f9eb9ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 >> 172.21.15.96:0/1304426141 conn(0x55f9eb9b8bb0 msgr2=0x55f9eb9c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f9103fff640 0 client.26170 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 shutdown_connections 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 1 -- 172.21.15.96:0/1304426141 wait complete. 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.595+0000 7f912dfd6740 20 client.26170 trim_cache size 0 max 16384 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 msgr2=0x7f91140b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 0x7f91140b78f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9118058330 tx=0x7f9118070f60 comp rx=0 tx=0).stop 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 msgr2=0x7f912419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f911800e9f0 tx=0x7f911800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 shutdown_connections 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f91140b5430 0x7f91140b78f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9124106290 0x7f91241a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91241036e0 0x7f912419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 --2- 172.21.15.96:0/2345087950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912406b470 0x7f912419f120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 >> 172.21.15.96:0/2345087950 conn(0x7f91240fcf30 msgr2=0x7f91241019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 shutdown_connections 2024-09-10T22:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:52:52.596+0000 7f9129fa1640 1 -- 172.21.15.96:0/2345087950 wait complete. 2024-09-10T22:52:53.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:52 smithi096 ceph-mon[30869]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-09-10T22:52:53.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:52 smithi121 ceph-mon[35119]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-09-10T22:52:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:52 smithi117 ceph-mon[35081]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-09-10T22:52:55.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:54 smithi096 ceph-mon[30869]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 744 op/s 2024-09-10T22:52:55.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:54 smithi121 ceph-mon[35119]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 744 op/s 2024-09-10T22:52:55.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:54 smithi117 ceph-mon[35081]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 MiB/s wr, 744 op/s 2024-09-10T22:52:55.389 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:52:55.809 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 320.0 s, 2850.2 tps, lat 34.886 ms stddev 40.699 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 -- 172.21.15.96:0/1827281010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc31810aa70 msgr2=0x7fc3181178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc315ffb640 1 -- 172.21.15.96:0/1827281010 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308031600 con 0x7fc31810aa70 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/1827281010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc31810aa70 0x7fc3181178c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc308009860 tx=0x7fc30802a410 comp rx=0 tx=0).stop 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 -- 172.21.15.96:0/1827281010 shutdown_connections 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/1827281010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318117e00 0x7fc31811a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/1827281010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc31810aa70 0x7fc3181178c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/1827281010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc31810a150 0x7fc31810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 -- 172.21.15.96:0/1827281010 >> 172.21.15.96:0/1827281010 conn(0x7fc31806e610 msgr2=0x7fc318070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 -- 172.21.15.96:0/1827281010 shutdown_connections 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.412+0000 7fc31d4b2640 1 -- 172.21.15.96:0/1827281010 wait complete. 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 Processor -- start 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 -- start start 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31810a150 0x7fc318117630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc31810aa70 0x7fc318114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc31811d7f0 con 0x7fc31810aa70 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc31811d670 con 0x7fc31810a150 2024-09-10T22:52:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.413+0000 7fc31d4b2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc31811d970 con 0x7fc318117e00 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46706/0 (socket says 172.21.15.96:46706) 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 -- 172.21.15.96:0/2078411774 learned_addr learned my addr 172.21.15.96:0/2078411774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31810a150 0x7fc318117630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 -- 172.21.15.96:0/2078411774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31810a150 msgr2=0x7fc318117630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31810a150 0x7fc318117630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 -- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc31810aa70 msgr2=0x7fc318114880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc31810aa70 0x7fc318114880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.414+0000 7fc3177fe640 1 -- 172.21.15.96:0/2078411774 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318115680 con 0x7fc318117e00 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc3177fe640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e940 tx=0x7fc30c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc2fffff640 1 -- 172.21.15.96:0/2078411774 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00ccc0 con 0x7fc318117e00 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc2fffff640 1 -- 172.21.15.96:0/2078411774 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c0054d0 con 0x7fc318117e00 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc2fffff640 1 -- 172.21.15.96:0/2078411774 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c010430 con 0x7fc318117e00 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3181b4650 con 0x7fc318117e00 2024-09-10T22:52:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.415+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3181b4b10 con 0x7fc318117e00 2024-09-10T22:52:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.419+0000 7fc31f887740 1 Processor -- start 2024-09-10T22:52:56.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.421+0000 7fc2fffff640 1 -- 172.21.15.96:0/2078411774 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc30c004f80 con 0x7fc318117e00 2024-09-10T22:52:56.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.421+0000 7fc31f887740 1 -- start start 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f0a0 0x559a3969f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a39615e50 con 0x559a396ab950 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a39605280 con 0x559a3969f9c0 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc31f887740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a39609090 con 0x559a3969f0a0 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60222/0 (socket says 172.21.15.96:60222) 2024-09-10T22:52:56.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc316ffd640 1 -- 172.21.15.96:0/91939270 learned_addr learned my addr 172.21.15.96:0/91939270 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:56.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.422+0000 7fc3167fc640 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f0a0 0x559a3969f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc2fffff640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 0x7fc3000b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc3167fc640 1 -- 172.21.15.96:0/91939270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f0a0 msgr2=0x559a3969f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc3167fc640 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f0a0 0x559a3969f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc3167fc640 1 -- 172.21.15.96:0/91939270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 msgr2=0x559a396ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc3167fc640 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc2fffff640 1 -- 172.21.15.96:0/2078411774 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc30c0e5f30 con 0x7fc318117e00 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc3167fc640 1 -- 172.21.15.96:0/91939270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a39363650 con 0x559a396ab950 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.423+0000 7fc316ffd640 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.424+0000 7fc3167fc640 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396add40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc308009860 tx=0x7fc30802a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23999f7a67042f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.424+0000 7fc3177fe640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 0x7fc3000b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.425+0000 7fc3177fe640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 0x7fc3000b7850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0a9960 tx=0x7fc30c0094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.426+0000 7fc2fd7fa640 1 -- 172.21.15.96:0/91939270 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308030ad0 con 0x559a396ab950 2024-09-10T22:52:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.426+0000 7fc2fd7fa640 1 -- 172.21.15.96:0/91939270 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308030c70 con 0x559a396ab950 2024-09-10T22:52:56.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.426+0000 7fc2fd7fa640 1 -- 172.21.15.96:0/91939270 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308003ac0 con 0x559a396ab950 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.427+0000 7fc31f887740 1 -- 172.21.15.96:0/91939270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 msgr2=0x559a396add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.427+0000 7fc31f887740 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396add40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc308009860 tx=0x7fc30802a910 comp rx=0 tx=0).stop 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 -- 172.21.15.96:0/91939270 shutdown_connections 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f9c0 0x559a396ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396add40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 --2- 172.21.15.96:0/91939270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f0a0 0x559a3969f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 -- 172.21.15.96:0/91939270 >> 172.21.15.96:0/91939270 conn(0x559a3969ced0 msgr2=0x559a3969e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 -- 172.21.15.96:0/91939270 shutdown_connections 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.428+0000 7fc31f887740 1 -- 172.21.15.96:0/91939270 wait complete. 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 -- start start 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 Processor -- start 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a39466700 con 0x559a396ab950 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a395a0110 con 0x559a3969f0a0 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a396ae5c0 con 0x559a3969f9c0 2024-09-10T22:52:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.429+0000 7fc31f887740 10 client.0 _ll_register_callbacks cb 0x559a395b9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60228/0 (socket says 172.21.15.96:60228) 2024-09-10T22:52:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 -- 172.21.15.96:0/2522590916 learned_addr learned my addr 172.21.15.96:0/2522590916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:56.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 -- 172.21.15.96:0/2522590916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 msgr2=0x559a396a8670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396a8670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 -- 172.21.15.96:0/2522590916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 msgr2=0x559a396a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.430+0000 7fc3167fc640 1 -- 172.21.15.96:0/2522590916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a395cdc40 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc3167fc640 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc3080095a0 tx=0x7fc308005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da80d900984dade9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080035f0 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc308030c70 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc308003840 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc308003a40 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc31f887740 10 client.26144 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a396b9f50 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.431+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308030c70 con 0x559a3969f0a0 2024-09-10T22:52:56.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc308054b80 con 0x559a3969f0a0 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 10 client.26144 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 20 client.26144 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a396bb2a0 0x559a396bd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 4 client.26144 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:52:56.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a39617da0 con 0x559a396bb2a0 2024-09-10T22:52:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.432+0000 7fc316ffd640 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a396bb2a0 0x559a396bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.433+0000 7fc2faff5640 10 client.26144 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.433+0000 7fc316ffd640 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a396bb2a0 0x559a396bd690 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f53eef961de9f server_cookie=b6d548e57df74b92 in_seq=0 out_seq=0 2024-09-10T22:52:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.434+0000 7fc2faff5640 1 -- 172.21.15.96:0/2522590916 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a39617da0 con 0x559a396bb2a0 2024-09-10T22:52:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.434+0000 7fc2faff5640 10 client.26144 handle_command_reply: tid=0 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.434+0000 7fc31f887740 1 client.26144 shutdown 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 msgr2=0x559a396ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc3080095a0 tx=0x7fc308005f10 comp rx=0 tx=0).stop 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 shutdown_connections 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a396bb2a0 0x559a396bd690 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a396ab950 0x559a396a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 --2- 172.21.15.96:0/2522590916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396ab220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 >> 172.21.15.96:0/2522590916 conn(0x559a3969ced0 msgr2=0x559a396a2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc2faff5640 0 client.26144 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:56.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.435+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 shutdown_connections 2024-09-10T22:52:56.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.436+0000 7fc31f887740 1 -- 172.21.15.96:0/2522590916 wait complete. 2024-09-10T22:52:56.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.436+0000 7fc31f887740 20 client.26144 trim_cache size 0 max 16384 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.466+0000 7fc31f887740 1 Processor -- start 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.466+0000 7fc31f887740 1 -- start start 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.466+0000 7fc31f887740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 0x559a3969c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.467+0000 7fc31f887740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.467+0000 7fc31f887740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 0x559a396ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.467+0000 7fc31f887740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181bd150 con 0x559a3969f0a0 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.467+0000 7fc31f887740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181bec90 con 0x559a396ab950 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.467+0000 7fc31f887740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181bd820 con 0x559a3969f9c0 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.468+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 0x559a396ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.468+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 0x559a396ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60236/0 (socket says 172.21.15.96:60236) 2024-09-10T22:52:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.468+0000 7fc3167fc640 1 -- 172.21.15.96:0/1205182318 learned_addr learned my addr 172.21.15.96:0/1205182318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.468+0000 7fc3167fc640 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 0x559a3969c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.470+0000 7fc316ffd640 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.472+0000 7fc3167fc640 1 -- 172.21.15.96:0/1205182318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 msgr2=0x559a396acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.472+0000 7fc3167fc640 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.472+0000 7fc3167fc640 1 -- 172.21.15.96:0/1205182318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 msgr2=0x559a396ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.472+0000 7fc3167fc640 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 0x559a396ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.472+0000 7fc3167fc640 1 -- 172.21.15.96:0/1205182318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a396a6b70 con 0x559a3969f0a0 2024-09-10T22:52:56.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.479+0000 7fc3167fc640 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 0x559a3969c710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc30803d4d0 tx=0x7fc3080309a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b099a971afe355 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.480+0000 7fc2fcff9640 1 -- 172.21.15.96:0/1205182318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308003be0 con 0x559a3969f0a0 2024-09-10T22:52:56.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.480+0000 7fc2fcff9640 1 -- 172.21.15.96:0/1205182318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30804f030 con 0x559a3969f0a0 2024-09-10T22:52:56.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.481+0000 7fc31f887740 1 -- 172.21.15.96:0/1205182318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 msgr2=0x559a3969c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.481+0000 7fc31f887740 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 0x559a3969c710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc30803d4d0 tx=0x7fc3080309a0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.484+0000 7fc31f887740 1 -- 172.21.15.96:0/1205182318 shutdown_connections 2024-09-10T22:52:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.484+0000 7fc31f887740 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a396ab950 0x559a396ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.484+0000 7fc31f887740 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a3969f9c0 0x559a396acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.484+0000 7fc31f887740 1 --2- 172.21.15.96:0/1205182318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f0a0 0x559a3969c710 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.484+0000 7fc31f887740 1 -- 172.21.15.96:0/1205182318 >> 172.21.15.96:0/1205182318 conn(0x559a3969ced0 msgr2=0x559a396aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.488+0000 7fc31f887740 1 -- 172.21.15.96:0/1205182318 shutdown_connections 2024-09-10T22:52:56.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.488+0000 7fc31f887740 1 -- 172.21.15.96:0/1205182318 wait complete. 2024-09-10T22:52:56.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.488+0000 7fc31f887740 1 -- start start 2024-09-10T22:52:56.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.488+0000 7fc31f887740 1 Processor -- start 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f9c0 0x559a396bc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 0x559a396bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181be5c0 con 0x559a3969f9c0 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181bdef0 con 0x559a3969f0a0 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181bb130 con 0x559a396ab950 2024-09-10T22:52:56.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc31f887740 10 client.0 _ll_register_callbacks cb 0x7fc310009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc3167fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396c0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60248/0 (socket says 172.21.15.96:60248) 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc3167fc640 1 -- 172.21.15.96:0/4022833466 learned_addr learned my addr 172.21.15.96:0/4022833466 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc316ffd640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f9c0 0x559a396bc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.489+0000 7fc3167fc640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 0x559a396bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 -- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 msgr2=0x559a396c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 -- 172.21.15.96:0/4022833466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f9c0 msgr2=0x559a396bc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:52:56.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f9c0 0x559a396bc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 -- 172.21.15.96:0/4022833466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a39726e00 con 0x559a396ab950 2024-09-10T22:52:56.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.490+0000 7fc3167fc640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 0x559a396bc740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc30804d040 tx=0x7fc30803d730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3f8c20f914cac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.492+0000 7fc2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:52:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.492+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080044a0 con 0x559a396ab950 2024-09-10T22:52:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.492+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30804f030 con 0x559a396ab950 2024-09-10T22:52:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.492+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc308030bb0 con 0x559a396ab950 2024-09-10T22:52:56.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.496+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc308003be0 con 0x559a396ab950 2024-09-10T22:52:56.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.497+0000 7fc31f887740 10 client.26185 fetch_fsmap learned FSMap version 32 2024-09-10T22:52:56.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.497+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a39726e00 con 0x559a396ab950 2024-09-10T22:52:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30804f030 con 0x559a396ab950 2024-09-10T22:52:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc308005140 con 0x559a396ab950 2024-09-10T22:52:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 10 client.26185 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:52:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 10 client.26185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:52:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 20 client.26185 populate_metadata read hostname 'smithi096' 2024-09-10T22:52:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a39728350 0x559a3972a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 4 client.26185 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:52:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.502+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a39613850 con 0x559a39728350 2024-09-10T22:52:56.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.504+0000 7fc316ffd640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a39728350 0x559a3972a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:56.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.508+0000 7fc316ffd640 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a39728350 0x559a3972a770 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9514d0a2c21715 server_cookie=40d8826e095546e9 in_seq=0 out_seq=0 2024-09-10T22:52:56.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.508+0000 7fc2faff5640 10 client.26185 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:56.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.509+0000 7fc2faff5640 1 -- 172.21.15.96:0/4022833466 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x559a39613850 con 0x559a39728350 2024-09-10T22:52:56.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.509+0000 7fc2faff5640 10 client.26185 handle_command_reply: tid=0 2024-09-10T22:52:56.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.509+0000 7fc31f887740 1 client.26185 shutdown 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 msgr2=0x559a396bc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 0x559a396bc740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc30804d040 tx=0x7fc30803d730 comp rx=0 tx=0).stop 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 shutdown_connections 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559a39728350 0x559a3972a770 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a396ab950 0x559a396bc740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a3969f9c0 0x559a396bc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 --2- 172.21.15.96:0/4022833466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a3969f0a0 0x559a396c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 >> 172.21.15.96:0/4022833466 conn(0x559a3969ced0 msgr2=0x559a396a2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc2faff5640 0 client.26185 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 shutdown_connections 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 1 -- 172.21.15.96:0/4022833466 wait complete. 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.519+0000 7fc31f887740 20 client.26185 trim_cache size 0 max 16384 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:52:56.522 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.524 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.525 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:52:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:52:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:52:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:52:56.527 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 msgr2=0x7fc3000b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 0x7fc3000b7850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0a9960 tx=0x7fc30c0094a0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 msgr2=0x7fc318114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e940 tx=0x7fc30c00ee00 comp rx=0 tx=0).stop 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 shutdown_connections 2024-09-10T22:52:56.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3000b5390 0x7fc3000b7850 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc318117e00 0x7fc318114dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc31810aa70 0x7fc318114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 --2- 172.21.15.96:0/2078411774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc31810a150 0x7fc318117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 >> 172.21.15.96:0/2078411774 conn(0x7fc31806e610 msgr2=0x7fc318118240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 shutdown_connections 2024-09-10T22:52:56.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:56.520+0000 7fc31d4b2640 1 -- 172.21.15.96:0/2078411774 wait complete. 2024-09-10T22:52:57.190 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:52:57.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:52:57.232 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:56 smithi096 ceph-mon[30869]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.5 MiB/s wr, 820 op/s 2024-09-10T22:52:57.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:56 smithi121 ceph-mon[35119]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.5 MiB/s wr, 820 op/s 2024-09-10T22:52:57.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:56 smithi117 ceph-mon[35081]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 8.5 MiB/s wr, 820 op/s 2024-09-10T22:52:57.627 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:52:58.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 -- 172.21.15.96:0/63846789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 msgr2=0x7f6508115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 --2- 172.21.15.96:0/63846789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f6508115f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6500009860 tx=0x7f650002a340 comp rx=0 tx=0).stop 2024-09-10T22:52:58.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 -- 172.21.15.96:0/63846789 shutdown_connections 2024-09-10T22:52:58.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 --2- 172.21.15.96:0/63846789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f6508115f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 --2- 172.21.15.96:0/63846789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 --2- 172.21.15.96:0/63846789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f65081097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 -- 172.21.15.96:0/63846789 >> 172.21.15.96:0/63846789 conn(0x7f6508078aa0 msgr2=0x7f650807aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 -- 172.21.15.96:0/63846789 shutdown_connections 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.212+0000 7f650dbbe640 1 -- 172.21.15.96:0/63846789 wait complete. 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 Processor -- start 2024-09-10T22:52:58.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 -- start start 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f650819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f65081a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6508119380 con 0x7f650810edd0 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6508119200 con 0x7f650810f750 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f650dbbe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6508119500 con 0x7f65081093c0 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f6506ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f6506ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45234/0 (socket says 172.21.15.96:45234) 2024-09-10T22:52:58.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f6506ffd640 1 -- 172.21.15.96:0/1284759025 learned_addr learned my addr 172.21.15.96:0/1284759025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f6507fff640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f65081a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.213+0000 7f65077fe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f650819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 -- 172.21.15.96:0/1284759025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 msgr2=0x7f650819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f650819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 -- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 msgr2=0x7f650819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 -- 172.21.15.96:0/1284759025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65081a58c0 con 0x7f650810f750 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f65077fe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f650819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6506ffd640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6507fff640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f65081a50f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f650002a860 tx=0x7f6500032d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500037040 con 0x7f650810f750 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65081a5b50 con 0x7f650810f750 2024-09-10T22:52:58.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6500004050 con 0x7f650810f750 2024-09-10T22:52:58.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.214+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500005020 con 0x7f650810f750 2024-09-10T22:52:58.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.215+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65081ac270 con 0x7f650810f750 2024-09-10T22:52:58.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.219+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65081097a0 con 0x7f650810f750 2024-09-10T22:52:58.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.221+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f6500030b50 con 0x7f650810f750 2024-09-10T22:52:58.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.222+0000 7f6504ff9640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 0x7f64f00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:52:58.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.222+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6500105bc0 con 0x7f650810f750 2024-09-10T22:52:58.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.222+0000 7f65077fe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 0x7f64f00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:52:58.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.222+0000 7f65077fe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 0x7f64f00b7820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005fd0 tx=0x7f64f8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:52:58.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.225+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f650004a020 con 0x7f650810f750 2024-09-10T22:52:58.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.568+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6508111530 con 0x7f650810f750 2024-09-10T22:52:58.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.569+0000 7f6504ff9640 1 -- 172.21.15.96:0/1284759025 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f65000cca80 con 0x7f650810f750 2024-09-10T22:52:58.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:52:58.571 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:52:58.571 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:52:58.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.571+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 msgr2=0x7f64f00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 0x7f64f00b7820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005fd0 tx=0x7f64f8009500 comp rx=0 tx=0).stop 2024-09-10T22:52:58.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 msgr2=0x7f65081a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:52:58.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f65081a50f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f650002a860 tx=0x7f6500032d20 comp rx=0 tx=0).stop 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 shutdown_connections 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f650810f750 0x7f65081a50f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64f00b5360 0x7f64f00b7820 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650810edd0 0x7f650819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 --2- 172.21.15.96:0/1284759025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65081093c0 0x7f650819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 >> 172.21.15.96:0/1284759025 conn(0x7f6508078aa0 msgr2=0x7f650807ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 shutdown_connections 2024-09-10T22:52:58.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:52:58.572+0000 7f650dbbe640 1 -- 172.21.15.96:0/1284759025 wait complete. 2024-09-10T22:52:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:58 smithi096 ceph-mon[30869]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-09-10T22:52:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:52:58 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1284759025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:59.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:58 smithi121 ceph-mon[35119]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-09-10T22:52:59.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:52:58 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1284759025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:59.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:58 smithi117 ceph-mon[35081]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 589 op/s 2024-09-10T22:52:59.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:52:58 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1284759025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:52:59.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:52:59.770 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 -- 172.21.15.96:0/409245742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1168101c90 msgr2=0x7f1168102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 --2- 172.21.15.96:0/409245742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1168101c90 0x7f1168102070 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f115c009860 tx=0x7f115c02a340 comp rx=0 tx=0).stop 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 -- 172.21.15.96:0/409245742 shutdown_connections 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 --2- 172.21.15.96:0/409245742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f116810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 --2- 172.21.15.96:0/409245742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11681076a0 0x7f1168107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 --2- 172.21.15.96:0/409245742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1168101c90 0x7f1168102070 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 -- 172.21.15.96:0/409245742 >> 172.21.15.96:0/409245742 conn(0x7f11680fd700 msgr2=0x7f11680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 -- 172.21.15.96:0/409245742 shutdown_connections 2024-09-10T22:53:00.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.330+0000 7f116de76640 1 -- 172.21.15.96:0/409245742 wait complete. 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.331+0000 7f116de76640 1 Processor -- start 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.331+0000 7f116de76640 1 -- start start 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 0x7f1168197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11681076a0 0x7f11681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f1168198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1168111ca0 con 0x7f11681076a0 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1168111b20 con 0x7f1168108020 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f116de76640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1168111e20 con 0x7f1168101c90 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f1167fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f1168198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f1167fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f1168198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60302/0 (socket says 172.21.15.96:60302) 2024-09-10T22:53:00.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f1167fff640 1 -- 172.21.15.96:0/445009107 learned_addr learned my addr 172.21.15.96:0/445009107 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:00.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.332+0000 7f11677fe640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 0x7f1168197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 -- 172.21.15.96:0/445009107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 msgr2=0x7f1168198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f1168198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 -- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11681076a0 msgr2=0x7f11681983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11681076a0 0x7f11681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 -- 172.21.15.96:0/445009107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116819d260 con 0x7f1168101c90 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.333+0000 7f11677fe640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 0x7f1168197e90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f115c035040 tx=0x7f115c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.334+0000 7f1164ff9640 1 -- 172.21.15.96:0/445009107 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c040020 con 0x7f1168101c90 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.334+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116819d4f0 con 0x7f1168101c90 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.334+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116819d9d0 con 0x7f1168101c90 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.334+0000 7f1164ff9640 1 -- 172.21.15.96:0/445009107 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115c030e10 con 0x7f1168101c90 2024-09-10T22:53:00.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.334+0000 7f1164ff9640 1 -- 172.21.15.96:0/445009107 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c037040 con 0x7f1168101c90 2024-09-10T22:53:00.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.337+0000 7f117024b740 1 Processor -- start 2024-09-10T22:53:00.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.337+0000 7f117024b740 1 -- start start 2024-09-10T22:53:00.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.338+0000 7f1164ff9640 1 -- 172.21.15.96:0/445009107 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f115c04d060 con 0x7f1168101c90 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf5408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1164ff9640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 0x7f11500b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf54f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654cf4b6ea0 con 0x5654cf5404d0 2024-09-10T22:53:00.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654cf4a62d0 con 0x5654cf54cd80 2024-09-10T22:53:00.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f117024b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5654cf4aa0e0 con 0x5654cf540df0 2024-09-10T22:53:00.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1164ff9640 1 -- 172.21.15.96:0/445009107 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f115c105400 con 0x7f1168101c90 2024-09-10T22:53:00.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46776/0 (socket says 172.21.15.96:46776) 2024-09-10T22:53:00.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 -- 172.21.15.96:0/3201262775 learned_addr learned my addr 172.21.15.96:0/3201262775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 -- 172.21.15.96:0/3201262775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 msgr2=0x5654cf54f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf54f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 -- 172.21.15.96:0/3201262775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 msgr2=0x5654cf5408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf5408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 -- 172.21.15.96:0/3201262775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cf204650 con 0x5654cf540df0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1167fff640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 0x7f11500b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.339+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1160005c40 tx=0x7f1160009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e790b0677a1bb415 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.340+0000 7f1167fff640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 0x7f11500b78f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1158009c70 tx=0x7f1158005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.340+0000 7f114dffb640 1 -- 172.21.15.96:0/3201262775 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160003b70 con 0x5654cf540df0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.340+0000 7f114dffb640 1 -- 172.21.15.96:0/3201262775 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160003d10 con 0x5654cf540df0 2024-09-10T22:53:00.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.340+0000 7f114dffb640 1 -- 172.21.15.96:0/3201262775 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160004520 con 0x5654cf540df0 2024-09-10T22:53:00.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.341+0000 7f117024b740 1 -- 172.21.15.96:0/3201262775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 msgr2=0x5654cf54c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.341+0000 7f117024b740 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1160005c40 tx=0x7f1160009f90 comp rx=0 tx=0).stop 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 -- 172.21.15.96:0/3201262775 shutdown_connections 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf54f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf54c840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 --2- 172.21.15.96:0/3201262775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf5408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 -- 172.21.15.96:0/3201262775 >> 172.21.15.96:0/3201262775 conn(0x5654cf53e300 msgr2=0x5654cf53f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.342+0000 7f117024b740 1 -- 172.21.15.96:0/3201262775 shutdown_connections 2024-09-10T22:53:00.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.343+0000 7f117024b740 1 -- 172.21.15.96:0/3201262775 wait complete. 2024-09-10T22:53:00.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.343+0000 7f117024b740 1 -- start start 2024-09-10T22:53:00.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.343+0000 7f117024b740 1 Processor -- start 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf5404d0 0x5654cf54c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf54cd80 0x5654cf549f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5654cf307700 con 0x5654cf54cd80 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5654cf441160 con 0x5654cf5404d0 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5654cf54faa0 con 0x5654cf540df0 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f117024b740 10 client.0 _ll_register_callbacks cb 0x5654cf45a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46780/0 (socket says 172.21.15.96:46780) 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 -- 172.21.15.96:0/3151598247 learned_addr learned my addr 172.21.15.96:0/3151598247 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f11677fe640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf5404d0 0x5654cf54c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 -- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf5404d0 msgr2=0x5654cf54c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf5404d0 0x5654cf54c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 -- 172.21.15.96:0/3151598247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf54cd80 msgr2=0x5654cf549f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf54cd80 0x5654cf549f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.344+0000 7f1166ffd640 1 -- 172.21.15.96:0/3151598247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cf46ec90 con 0x5654cf540df0 2024-09-10T22:53:00.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f1166ffd640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1160006010 tx=0x7f1160009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4777298aa2ee611 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f1146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160024070 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f116000c640 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1160003d10 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f116002e050 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f117024b740 10 client.26203 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.345+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654cf55b380 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160003d10 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f116000b040 con 0x5654cf540df0 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 10 client.26203 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 10 client.26203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 20 client.26203 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:00.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf55c7e0 0x5654cf55ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 4 client.26203 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654cf4b8df0 con 0x5654cf55c7e0 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.346+0000 7f11677fe640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf55c7e0 0x5654cf55ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.347+0000 7f1146ffd640 10 client.26203 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.347+0000 7f11677fe640 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf55c7e0 0x5654cf55ebd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f957d6e0066a27 server_cookie=b5180837a52bdc6 in_seq=0 out_seq=0 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.347+0000 7f1146ffd640 1 -- 172.21.15.96:0/3151598247 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654cf4b8df0 con 0x5654cf55c7e0 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.347+0000 7f1146ffd640 10 client.26203 handle_command_reply: tid=0 2024-09-10T22:53:00.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.347+0000 7f117024b740 1 client.26203 shutdown 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 msgr2=0x5654cf5499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1160006010 tx=0x7f1160009d60 comp rx=0 tx=0).stop 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 shutdown_connections 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf55c7e0 0x5654cf55ebd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf54cd80 0x5654cf549f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf540df0 0x5654cf5499e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 --2- 172.21.15.96:0/3151598247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf5404d0 0x5654cf54c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f1146ffd640 0 client.26203 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 >> 172.21.15.96:0/3151598247 conn(0x5654cf53e300 msgr2=0x5654cf543e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 shutdown_connections 2024-09-10T22:53:00.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 1 -- 172.21.15.96:0/3151598247 wait complete. 2024-09-10T22:53:00.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.348+0000 7f117024b740 20 client.26203 trim_cache size 0 max 16384 2024-09-10T22:53:00.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 Processor -- start 2024-09-10T22:53:00.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 -- start start 2024-09-10T22:53:00.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf549790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 0x5654cf546bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f115400e430 con 0x5654cf5404d0 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f115400ff70 con 0x5654cf540df0 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f117024b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f115400eb00 con 0x5654cf54cd80 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f11677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf549790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f1166ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60322/0 (socket says 172.21.15.96:60322) 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f1166ffd640 1 -- 172.21.15.96:0/1737087368 learned_addr learned my addr 172.21.15.96:0/1737087368 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.370+0000 7f1167fff640 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 0x5654cf546bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 -- 172.21.15.96:0/1737087368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 msgr2=0x5654cf546bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 0x5654cf546bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 -- 172.21.15.96:0/1737087368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 msgr2=0x5654cf549790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf549790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 -- 172.21.15.96:0/1737087368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654cf5445d0 con 0x5654cf540df0 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1167fff640 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 0x5654cf546bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f1166ffd640 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1160004a80 tx=0x7f1160009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea6749b89c9ea30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f114d7fa640 1 -- 172.21.15.96:0/1737087368 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160038070 con 0x5654cf540df0 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f114d7fa640 1 -- 172.21.15.96:0/1737087368 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160005400 con 0x5654cf540df0 2024-09-10T22:53:00.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.371+0000 7f114d7fa640 1 -- 172.21.15.96:0/1737087368 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160024040 con 0x5654cf540df0 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- 172.21.15.96:0/1737087368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 msgr2=0x5654cf549cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1160004a80 tx=0x7f1160009380 comp rx=0 tx=0).stop 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- 172.21.15.96:0/1737087368 shutdown_connections 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf540df0 0x5654cf549cd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf54cd80 0x5654cf546bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 --2- 172.21.15.96:0/1737087368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf5404d0 0x5654cf549790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- 172.21.15.96:0/1737087368 >> 172.21.15.96:0/1737087368 conn(0x5654cf53e300 msgr2=0x5654cf548260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- 172.21.15.96:0/1737087368 shutdown_connections 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- 172.21.15.96:0/1737087368 wait complete. 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.372+0000 7f117024b740 1 -- start start 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.373+0000 7f117024b740 1 Processor -- start 2024-09-10T22:53:00.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.373+0000 7f117024b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.373+0000 7f117024b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf540df0 0x5654cf542360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.373+0000 7f117024b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf542970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f117024b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f115400f8a0 con 0x5654cf540df0 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f117024b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f115400f1d0 con 0x5654cf54cd80 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f117024b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f115400c410 con 0x5654cf5404d0 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f117024b740 10 client.0 _ll_register_callbacks cb 0x7f1160009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f11677fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f11677fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46786/0 (socket says 172.21.15.96:46786) 2024-09-10T22:53:00.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.374+0000 7f11677fe640 1 -- 172.21.15.96:0/432562490 learned_addr learned my addr 172.21.15.96:0/432562490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f1167fff640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf542970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f1166ffd640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf540df0 0x5654cf542360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 -- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 msgr2=0x5654cf542970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf542970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 -- 172.21.15.96:0/432562490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf540df0 msgr2=0x5654cf542360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf540df0 0x5654cf542360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 -- 172.21.15.96:0/432562490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654cf5431a0 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f1167fff640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf542970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f11677fe640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f115c0c8f50 tx=0x7f115c030fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2420d7013f8451a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f1146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.375+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0cb7f0 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f115c0d4de0 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1160005400 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f115c0cc3e0 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 10 client.26206 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654cf5c9680 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115c0cbce0 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f115c0cb060 con 0x5654cf5404d0 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 10 client.26206 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:00.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 10 client.26206 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 20 client.26206 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf5c9e50 0x5654cf5cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 4 client.26206 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.376+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1160005400 con 0x5654cf5c9e50 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.377+0000 7f1166ffd640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf5c9e50 0x5654cf5cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.377+0000 7f1146ffd640 10 client.26206 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.377+0000 7f1166ffd640 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf5c9e50 0x5654cf5cc240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=747a5178c30e37b3 server_cookie=4157fe136a74741d in_seq=0 out_seq=0 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f1146ffd640 1 -- 172.21.15.96:0/432562490 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1160005400 con 0x5654cf5c9e50 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f1146ffd640 10 client.26206 handle_command_reply: tid=0 2024-09-10T22:53:00.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 client.26206 shutdown 2024-09-10T22:53:00.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 msgr2=0x5654cf555620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f115c0c8f50 tx=0x7f115c030fc0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 shutdown_connections 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5654cf5c9e50 0x5654cf5cc240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f1146ffd640 0 client.26206 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.378+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654cf54cd80 0x5654cf542970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654cf540df0 0x5654cf542360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 1 --2- 172.21.15.96:0/432562490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654cf5404d0 0x5654cf555620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 >> 172.21.15.96:0/432562490 conn(0x5654cf53e300 msgr2=0x5654cf549790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 shutdown_connections 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 1 -- 172.21.15.96:0/432562490 wait complete. 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.379+0000 7f117024b740 20 client.26206 trim_cache size 0 max 16384 2024-09-10T22:53:00.383 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 msgr2=0x7f11500b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 0x7f11500b78f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1158009c70 tx=0x7f1158005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 msgr2=0x7f1168197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 0x7f1168197e90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f115c035040 tx=0x7f115c032d50 comp rx=0 tx=0).stop 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 shutdown_connections 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1168108020 0x7f1168198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11500b5430 0x7f11500b78f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11681076a0 0x7f11681983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 --2- 172.21.15.96:0/445009107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1168101c90 0x7f1168197e90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:00.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 >> 172.21.15.96:0/445009107 conn(0x7f11680fd700 msgr2=0x7f11680ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:00.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 shutdown_connections 2024-09-10T22:53:00.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:00.380+0000 7f116de76640 1 -- 172.21.15.96:0/445009107 wait complete. 2024-09-10T22:53:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 325.0 s, 2646.0 tps, lat 37.880 ms stddev 41.504 2024-09-10T22:53:01.191 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:00 smithi096 ceph-mon[30869]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T22:53:01.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:53:01.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:00 smithi121 ceph-mon[35119]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T22:53:01.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:00 smithi117 ceph-mon[35081]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T22:53:01.618 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/1167437862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b0108060 msgr2=0x7fd9b010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 --2- 172.21.15.96:0/1167437862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b0108060 0x7fd9b010e860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009be0 tx=0x7fd9a802a710 comp rx=0 tx=0).stop 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/1167437862 shutdown_connections 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 --2- 172.21.15.96:0/1167437862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b0108060 0x7fd9b010e860 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 --2- 172.21.15.96:0/1167437862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b01076e0 0x7fd9b0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 --2- 172.21.15.96:0/1167437862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b0101cd0 0x7fd9b01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.195+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/1167437862 >> 172.21.15.96:0/1167437862 conn(0x7fd9b00fd740 msgr2=0x7fd9b00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.200+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/1167437862 shutdown_connections 2024-09-10T22:53:02.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.202+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/1167437862 wait complete. 2024-09-10T22:53:02.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.202+0000 7fd9b5ee2640 1 Processor -- start 2024-09-10T22:53:02.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 -- start start 2024-09-10T22:53:02.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b0101cd0 0x7fd9b019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 0x7fd9b01a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 0x7fd9b01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0111ce0 con 0x7fd9b0108060 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0111b60 con 0x7fd9b01076e0 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9b5ee2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0111e60 con 0x7fd9b0101cd0 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 0x7fd9b01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 0x7fd9b01a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45302/0 (socket says 172.21.15.96:45302) 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9affff640 1 -- 172.21.15.96:0/4273079158 learned_addr learned my addr 172.21.15.96:0/4273079158 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:02.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.203+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 0x7fd9b01a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.204+0000 7fd9aeffd640 1 -- 172.21.15.96:0/4273079158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b0101cd0 msgr2=0x7fd9b019fb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:53:02.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.204+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b0101cd0 0x7fd9b019fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.204+0000 7fd9aeffd640 1 -- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 msgr2=0x7fd9b01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.204+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 0x7fd9b01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.204+0000 7fd9aeffd640 1 -- 172.21.15.96:0/4273079158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b01a2680 con 0x7fd9b01076e0 2024-09-10T22:53:02.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.206+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 0x7fd9b01a0070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d6d0 tx=0x7fd9a000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.209+0000 7fd9acff9640 1 -- 172.21.15.96:0/4273079158 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0017070 con 0x7fd9b01076e0 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.209+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/4273079158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b01a2970 con 0x7fd9b01076e0 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.209+0000 7fd9b5ee2640 1 -- 172.21.15.96:0/4273079158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b01ac2e0 con 0x7fd9b01076e0 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.212+0000 7fd9b82b7740 1 Processor -- start 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.212+0000 7fd9b82b7740 1 -- start start 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.213+0000 7fd9acff9640 1 -- 172.21.15.96:0/4273079158 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a0009820 con 0x7fd9b01076e0 2024-09-10T22:53:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.213+0000 7fd9acff9640 1 -- 172.21.15.96:0/4273079158 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004b90 con 0x7fd9b01076e0 2024-09-10T22:53:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb70e0 0x556128cc2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cc3070 0x556128cc5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556128c1bea0 con 0x556128cc3070 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556128c0b2d0 con 0x556128cb67c0 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556128c0f0e0 con 0x556128cb70e0 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60356/0 (socket says 172.21.15.96:60356) 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 -- 172.21.15.96:0/403915322 learned_addr learned my addr 172.21.15.96:0/403915322 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 -- 172.21.15.96:0/403915322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb70e0 msgr2=0x556128cc2b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb70e0 0x556128cc2b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 -- 172.21.15.96:0/403915322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cc3070 msgr2=0x556128cc5460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cc3070 0x556128cc5460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.214+0000 7fd9affff640 1 -- 172.21.15.96:0/403915322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556128969650 con 0x556128cb67c0 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.215+0000 7fd9affff640 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005f00 tx=0x7fd9a80303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4a193f165a50cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.215+0000 7fd9957fa640 1 -- 172.21.15.96:0/403915322 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8043070 con 0x556128cb67c0 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.215+0000 7fd9957fa640 1 -- 172.21.15.96:0/403915322 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8030d30 con 0x556128cb67c0 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.215+0000 7fd9957fa640 1 -- 172.21.15.96:0/403915322 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8036040 con 0x556128cb67c0 2024-09-10T22:53:02.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.216+0000 7fd9acff9640 1 -- 172.21.15.96:0/4273079158 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd9a0023060 con 0x7fd9b01076e0 2024-09-10T22:53:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.217+0000 7fd9acff9640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 0x7fd9980b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.218+0000 7fd9b82b7740 1 -- 172.21.15.96:0/403915322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 msgr2=0x556128cb6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.218+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005f00 tx=0x7fd9a80303a0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 0x7fd9980b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9b82b7740 1 -- 172.21.15.96:0/403915322 shutdown_connections 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.218+0000 7fd9acff9640 1 -- 172.21.15.96:0/4273079158 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd9a00e50a0 con 0x7fd9b01076e0 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cc3070 0x556128cc5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb70e0 0x556128cc2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/403915322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cb67c0 0x556128cb6ba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.219+0000 7fd9b82b7740 1 -- 172.21.15.96:0/403915322 >> 172.21.15.96:0/403915322 conn(0x556128cb45f0 msgr2=0x556128cb5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.220+0000 7fd9b82b7740 1 -- 172.21.15.96:0/403915322 shutdown_connections 2024-09-10T22:53:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.220+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 0x7fd9980b7970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x556128cb9940 tx=0x7fd9a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.220+0000 7fd9b82b7740 1 -- 172.21.15.96:0/403915322 wait complete. 2024-09-10T22:53:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.220+0000 7fd9b82b7740 1 -- start start 2024-09-10T22:53:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 Processor -- start 2024-09-10T22:53:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556128a6c700 con 0x556128cb70e0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556128ba6160 con 0x556128cc3070 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556128cc5be0 con 0x556128cb67c0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45304/0 (socket says 172.21.15.96:45304) 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2617509182 learned_addr learned my addr 172.21.15.96:0/2617509182 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9affff640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9b82b7740 10 client.0 _ll_register_callbacks cb 0x556128bbf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.221+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 msgr2=0x556128cc2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2617509182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 msgr2=0x556128cc0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2617509182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556128bd3c90 con 0x556128cb70e0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a8030d30 con 0x556128cb70e0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ab7f0 tx=0x7fd9a00ab370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57e220821e34961 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9937f6640 10 client.16599 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9937f6640 1 -- 172.21.15.96:0/2617509182 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0020040 con 0x556128cb70e0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9937f6640 1 -- 172.21.15.96:0/2617509182 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd9a00acb90 con 0x556128cb70e0 2024-09-10T22:53:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9937f6640 1 -- 172.21.15.96:0/2617509182 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd9a00acd90 con 0x556128cb70e0 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9b82b7740 10 client.16599 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.222+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x556128cd12a0 con 0x556128cb70e0 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9937f6640 1 -- 172.21.15.96:0/2617509182 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a00cebb0 con 0x556128cb70e0 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 10 client.16599 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 10 client.16599 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 20 client.16599 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128cd29c0 0x556128cd4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 4 client.16599 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556128c1ddf0 con 0x556128cd29c0 2024-09-10T22:53:02.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.223+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128cd29c0 0x556128cd4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.225+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128cd29c0 0x556128cd4db0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc7d30d74c31d773 server_cookie=70babee025a988a0 in_seq=0 out_seq=0 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.225+0000 7fd9937f6640 10 client.16599 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.225+0000 7fd9937f6640 1 -- 172.21.15.96:0/2617509182 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556128c1ddf0 con 0x556128cd29c0 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.225+0000 7fd9937f6640 10 client.16599 handle_command_reply: tid=0 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.225+0000 7fd9b82b7740 1 client.16599 shutdown 2024-09-10T22:53:02.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 msgr2=0x556128cbfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ab7f0 tx=0x7fd9a00ab370 comp rx=0 tx=0).stop 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 shutdown_connections 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128cd29c0 0x556128cd4db0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9937f6640 0 client.16599 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cbfbc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2617509182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 >> 172.21.15.96:0/2617509182 conn(0x556128cb45f0 msgr2=0x556128cba190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 shutdown_connections 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2617509182 wait complete. 2024-09-10T22:53:02.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.226+0000 7fd9b82b7740 20 client.16599 trim_cache size 0 max 16384 2024-09-10T22:53:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 Processor -- start 2024-09-10T22:53:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 -- start start 2024-09-10T22:53:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b019f410 con 0x556128cb70e0 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01adb40 con 0x556128cc3070 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01ac6d0 con 0x556128cb67c0 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45318/0 (socket says 172.21.15.96:45318) 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2748641490 learned_addr learned my addr 172.21.15.96:0/2748641490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9affff640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.246+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2748641490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 msgr2=0x556128cc1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2748641490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 msgr2=0x556128cc49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 -- 172.21.15.96:0/2748641490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556128c7e040 con 0x556128cb70e0 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9affff640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00a7910 tx=0x7fd9a00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5715d711595f7f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd994ff9640 1 -- 172.21.15.96:0/2748641490 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0020040 con 0x556128cb70e0 2024-09-10T22:53:02.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd994ff9640 1 -- 172.21.15.96:0/2748641490 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8030d30 con 0x556128cb70e0 2024-09-10T22:53:02.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.247+0000 7fd994ff9640 1 -- 172.21.15.96:0/2748641490 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00ab860 con 0x556128cb70e0 2024-09-10T22:53:02.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.248+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2748641490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 msgr2=0x556128cc4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.248+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00a7910 tx=0x7fd9a00a7000 comp rx=0 tx=0).stop 2024-09-10T22:53:02.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2748641490 shutdown_connections 2024-09-10T22:53:02.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cc49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cc4460 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/2748641490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cc1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2748641490 >> 172.21.15.96:0/2748641490 conn(0x556128cb45f0 msgr2=0x556128cc11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2748641490 shutdown_connections 2024-09-10T22:53:02.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 -- 172.21.15.96:0/2748641490 wait complete. 2024-09-10T22:53:02.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.249+0000 7fd9b82b7740 1 -- start start 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 Processor -- start 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cd8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cd53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01ad470 con 0x556128cb70e0 2024-09-10T22:53:02.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01acda0 con 0x556128cc3070 2024-09-10T22:53:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b01020b0 con 0x556128cb67c0 2024-09-10T22:53:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9b82b7740 10 client.0 _ll_register_callbacks cb 0x7fd9a00057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45322/0 (socket says 172.21.15.96:45322) 2024-09-10T22:53:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 -- 172.21.15.96:0/1862146334 learned_addr learned my addr 172.21.15.96:0/1862146334 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9affff640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cd53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cd8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 -- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 msgr2=0x556128cd8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cd8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 -- 172.21.15.96:0/1862146334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 msgr2=0x556128cd53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cd53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.250+0000 7fd9aeffd640 1 -- 172.21.15.96:0/1862146334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556128d3f530 con 0x556128cb70e0 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd9aeffd640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009610 tx=0x7fd9a00ab4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a56c49323acba38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd992ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:02.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0020070 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd9a0005220 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a8030d30 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd9a0004e10 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd9b82b7740 10 client.16605 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.252+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556128d40780 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a00ab860 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a00a7340 con 0x556128cb70e0 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 10 client.16605 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 10 client.16605 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 20 client.16605 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128d40d10 0x556128d43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 4 client.16605 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:02.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd9a8030d30 con 0x556128d40d10 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.253+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128d40d10 0x556128d43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9af7fe640 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128d40d10 0x556128d43130 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50dc22ae056269d6 server_cookie=c22cd8211392b434 in_seq=0 out_seq=0 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd992ff5640 10 client.16605 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd992ff5640 1 -- 172.21.15.96:0/1862146334 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd9a8030d30 con 0x556128d40d10 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd992ff5640 10 client.16605 handle_command_reply: tid=0 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 client.16605 shutdown 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 msgr2=0x556128cd4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009610 tx=0x7fd9a00ab4a0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 shutdown_connections 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556128d40d10 0x556128d43130 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556128cc3070 0x556128cd53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556128cb70e0 0x556128cd4c60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 --2- 172.21.15.96:0/1862146334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556128cb67c0 0x556128cd8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 >> 172.21.15.96:0/1862146334 conn(0x556128cb45f0 msgr2=0x556128cc1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd992ff5640 0 client.16605 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:02.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.254+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 shutdown_connections 2024-09-10T22:53:02.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.255+0000 7fd9b82b7740 1 -- 172.21.15.96:0/1862146334 wait complete. 2024-09-10T22:53:02.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.255+0000 7fd9b82b7740 20 client.16605 trim_cache size 0 max 16384 2024-09-10T22:53:02.267 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:02.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 msgr2=0x7fd9980b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 0x7fd9980b7970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x556128cb9940 tx=0x7fd9a4009260 comp rx=0 tx=0).stop 2024-09-10T22:53:02.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 msgr2=0x7fd9b01a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:02.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 0x7fd9b01a0070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d6d0 tx=0x7fd9a000db90 comp rx=0 tx=0).stop 2024-09-10T22:53:02.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 shutdown_connections 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b01076e0 0x7fd9b01a0070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd9980b54b0 0x7fd9980b7970 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9b0108060 0x7fd9b01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 --2- 172.21.15.96:0/4273079158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b0101cd0 0x7fd9b019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 >> 172.21.15.96:0/4273079158 conn(0x7fd9b00fd740 msgr2=0x7fd9b006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 shutdown_connections 2024-09-10T22:53:02.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:02.256+0000 7fd9967fc640 1 -- 172.21.15.96:0/4273079158 wait complete. 2024-09-10T22:53:03.075 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:53:03.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:02 smithi096 ceph-mon[30869]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 835 op/s 2024-09-10T22:53:03.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:02 smithi121 ceph-mon[35119]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 835 op/s 2024-09-10T22:53:03.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:02 smithi117 ceph-mon[35081]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 835 op/s 2024-09-10T22:53:03.458 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:04.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 -- 172.21.15.96:0/616032093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 msgr2=0x7fdba0101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 --2- 172.21.15.96:0/616032093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0101690 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009900 tx=0x7fdb9c02a410 comp rx=0 tx=0).stop 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 -- 172.21.15.96:0/616032093 shutdown_connections 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 --2- 172.21.15.96:0/616032093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba0105f90 0x7fdba010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 --2- 172.21.15.96:0/616032093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba01055d0 0x7fdba0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 --2- 172.21.15.96:0/616032093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0101690 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 -- 172.21.15.96:0/616032093 >> 172.21.15.96:0/616032093 conn(0x7fdba0078140 msgr2=0x7fdba00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 -- 172.21.15.96:0/616032093 shutdown_connections 2024-09-10T22:53:04.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.006+0000 7fdba7e08640 1 -- 172.21.15.96:0/616032093 wait complete. 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 Processor -- start 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 -- start start 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 0x7fdba01027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba0105f90 0x7fdba0102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0112dd0 con 0x7fdba01012b0 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0112c50 con 0x7fdba01055d0 2024-09-10T22:53:04.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba7e08640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0112f50 con 0x7fdba0105f90 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45344/0 (socket says 172.21.15.96:45344) 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba53a3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 0x7fdba01027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba5ba4640 1 -- 172.21.15.96:0/1674319649 learned_addr learned my addr 172.21.15.96:0/1674319649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba63a5640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba0105f90 0x7fdba0102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba53a3640 1 -- 172.21.15.96:0/1674319649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba0105f90 msgr2=0x7fdba0102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba0105f90 0x7fdba0102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba53a3640 1 -- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 msgr2=0x7fdba0102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.007+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdba53a3640 1 -- 172.21.15.96:0/1674319649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba01a7700 con 0x7fdba01055d0 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 0x7fdba01027c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9f0 tx=0x7fdb9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdb96ffd640 1 -- 172.21.15.96:0/1674319649 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000ccc0 con 0x7fdba01055d0 2024-09-10T22:53:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdb96ffd640 1 -- 172.21.15.96:0/1674319649 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb900108e0 con 0x7fdba01055d0 2024-09-10T22:53:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdb96ffd640 1 -- 172.21.15.96:0/1674319649 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90018880 con 0x7fdba01055d0 2024-09-10T22:53:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba01a79f0 con 0x7fdba01055d0 2024-09-10T22:53:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.008+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba01a7f30 con 0x7fdba01055d0 2024-09-10T22:53:04.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.012+0000 7fdb96ffd640 1 -- 172.21.15.96:0/1674319649 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdb90010430 con 0x7fdba01055d0 2024-09-10T22:53:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.012+0000 7fdbaa1dd740 1 Processor -- start 2024-09-10T22:53:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 -- start start 2024-09-10T22:53:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x5637216928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563721608ea0 con 0x563721692df0 2024-09-10T22:53:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637215f82d0 con 0x56372169ed80 2024-09-10T22:53:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5637215fc0e0 con 0x5637216924d0 2024-09-10T22:53:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45356/0 (socket says 172.21.15.96:45356) 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 -- 172.21.15.96:0/2423386468 learned_addr learned my addr 172.21.15.96:0/2423386468 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba63a5640 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x5637216928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba53a3640 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdb96ffd640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 0x7fdb8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 -- 172.21.15.96:0/2423386468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 msgr2=0x5637216928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x5637216928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 -- 172.21.15.96:0/2423386468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 msgr2=0x5637216a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.013+0000 7fdba5ba4640 1 -- 172.21.15.96:0/2423386468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563721356650 con 0x563721692df0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdba63a5640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 0x7fdb8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdb96ffd640 1 -- 172.21.15.96:0/1674319649 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb900e5e50 con 0x7fdba01055d0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006010 tx=0x7fdb9c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fee994193893b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdb8bfff640 1 -- 172.21.15.96:0/2423386468 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c040070 con 0x563721692df0 2024-09-10T22:53:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdb8bfff640 1 -- 172.21.15.96:0/2423386468 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c0047a0 con 0x563721692df0 2024-09-10T22:53:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.014+0000 7fdb8bfff640 1 -- 172.21.15.96:0/2423386468 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c035040 con 0x563721692df0 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.015+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/2423386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 msgr2=0x56372169e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.015+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006010 tx=0x7fdb9c0044b0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/2423386468 shutdown_connections 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169e840 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/2423386468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x5637216928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/2423386468 >> 172.21.15.96:0/2423386468 conn(0x563721690300 msgr2=0x5637216914a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/2423386468 shutdown_connections 2024-09-10T22:53:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/2423386468 wait complete. 2024-09-10T22:53:04.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdba63a5640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 0x7fdb8c0b77a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdba0104340 tx=0x7fdb98004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.016+0000 7fdbaa1dd740 1 -- start start 2024-09-10T22:53:04.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 Processor -- start 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x56372169c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46850/0 (socket says 172.21.15.96:46850) 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563721459700 con 0x563721692df0 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdba5ba4640 1 -- 172.21.15.96:0/4102568501 learned_addr learned my addr 172.21.15.96:0/4102568501 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563721593160 con 0x56372169ed80 2024-09-10T22:53:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5637216a19a0 con 0x5637216924d0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdbaa1dd740 10 client.0 _ll_register_callbacks cb 0x5637215ac9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.017+0000 7fdba53a3640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba63a5640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x56372169c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 -- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 msgr2=0x56372169c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x56372169c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 -- 172.21.15.96:0/4102568501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 msgr2=0x56372169ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637215c0c90 con 0x5637216924d0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005ab0 tx=0x7fdb9c004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0430d182318c91c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c035070 con 0x5637216924d0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb9c003c20 con 0x5637216924d0 2024-09-10T22:53:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.018+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c0047a0 con 0x5637216924d0 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb9c040040 con 0x5637216924d0 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdbaa1dd740 10 client.26230 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5637216b5af0 con 0x5637216924d0 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c0047a0 con 0x5637216924d0 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb9c031530 con 0x5637216924d0 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdbaa1dd740 10 client.26230 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.019+0000 7fdbaa1dd740 10 client.26230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.020+0000 7fdbaa1dd740 20 client.26230 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.020+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5637216b6fa0 0x5637216b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.020+0000 7fdbaa1dd740 4 client.26230 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.020+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56372160adf0 con 0x5637216b6fa0 2024-09-10T22:53:04.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.020+0000 7fdba53a3640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5637216b6fa0 0x5637216b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdba53a3640 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5637216b6fa0 0x5637216b9410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de2b2e4cd1303da3 server_cookie=9eec4aea5aa615d8 in_seq=0 out_seq=0 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdb897fa640 10 client.26230 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdb897fa640 1 -- 172.21.15.96:0/4102568501 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56372160adf0 con 0x5637216b6fa0 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdb897fa640 10 client.26230 handle_command_reply: tid=0 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdbaa1dd740 1 client.26230 shutdown 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 msgr2=0x56372169e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.021+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005ab0 tx=0x7fdb9c004b80 comp rx=0 tx=0).stop 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 shutdown_connections 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5637216b6fa0 0x5637216b9410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x56372169c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563721692df0 0x56372169ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/4102568501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372169e610 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 >> 172.21.15.96:0/4102568501 conn(0x563721690300 msgr2=0x563721695e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdb897fa640 0 client.26230 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 shutdown_connections 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/4102568501 wait complete. 2024-09-10T22:53:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.022+0000 7fdbaa1dd740 20 client.26230 trim_cache size 0 max 16384 2024-09-10T22:53:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.039+0000 7fdbaa1dd740 1 Processor -- start 2024-09-10T22:53:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.039+0000 7fdbaa1dd740 1 -- start start 2024-09-10T22:53:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.039+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.039+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563721692df0 0x563721698cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.039+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56372169ed80 0x5637216991f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01ac0d0 con 0x56372169ed80 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01adc10 con 0x563721692df0 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01ac7a0 con 0x5637216924d0 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46866/0 (socket says 172.21.15.96:46866) 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 -- 172.21.15.96:0/3631866579 learned_addr learned my addr 172.21.15.96:0/3631866579 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba53a3640 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563721692df0 0x563721698cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 -- 172.21.15.96:0/3631866579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563721692df0 msgr2=0x563721698cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563721692df0 0x563721698cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 -- 172.21.15.96:0/3631866579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56372169ed80 msgr2=0x5637216991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56372169ed80 0x5637216991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.040+0000 7fdba5ba4640 1 -- 172.21.15.96:0/3631866579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372169e240 con 0x5637216924d0 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.041+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005c40 tx=0x7fdb9c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9ee306801d641e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.041+0000 7fdb8b7fe640 1 -- 172.21.15.96:0/3631866579 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c05c070 con 0x5637216924d0 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.041+0000 7fdb8b7fe640 1 -- 172.21.15.96:0/3631866579 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c031dc0 con 0x5637216924d0 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.041+0000 7fdb8b7fe640 1 -- 172.21.15.96:0/3631866579 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c035040 con 0x5637216924d0 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/3631866579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 msgr2=0x56372168fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005c40 tx=0x7fdb9c005a20 comp rx=0 tx=0).stop 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/3631866579 shutdown_connections 2024-09-10T22:53:04.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563721692df0 0x563721698cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56372169ed80 0x5637216991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/3631866579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637216924d0 0x56372168fc10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/3631866579 >> 172.21.15.96:0/3631866579 conn(0x563721690300 msgr2=0x56372169d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/3631866579 shutdown_connections 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/3631866579 wait complete. 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 -- start start 2024-09-10T22:53:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.042+0000 7fdbaa1dd740 1 Processor -- start 2024-09-10T22:53:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 0x5637216af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 0x5637216afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01ad540 con 0x5637216924d0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01ace70 con 0x56372169ed80 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 0x5637216af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdba5ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 0x5637216af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45384/0 (socket says 172.21.15.96:45384) 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdba5ba4640 1 -- 172.21.15.96:0/1463033246 learned_addr learned my addr 172.21.15.96:0/1463033246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01aa0d0 con 0x563721692df0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 0x5637216afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdba63a5640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.043+0000 7fdbaa1dd740 10 client.0 _ll_register_callbacks cb 0x7fdb98009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 -- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 msgr2=0x5637216ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 -- 172.21.15.96:0/1463033246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 msgr2=0x5637216af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 0x5637216af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 -- 172.21.15.96:0/1463033246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637216b0100 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdba53a3640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 0x5637216afa90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb900e3a40 tx=0x7fdb90009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15bf5bf4355f0a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.044+0000 7fdb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900a6e60 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb900a89d0 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c031dc0 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb900ac270 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdbaa1dd740 10 client.26236 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.045+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563721724390 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9c031dc0 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb900e9020 con 0x563721692df0 2024-09-10T22:53:04.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 10 client.26236 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 10 client.26236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 20 client.26236 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x563721724a70 0x563721726e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 4 client.26236 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637216068a0 con 0x563721724a70 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.046+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x563721724a70 0x563721726e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:04.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.047+0000 7fdb897fa640 10 client.26236 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:04.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.047+0000 7fdba5ba4640 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x563721724a70 0x563721726e90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d9ca2c2b6923bfd server_cookie=41c770f492750e3f in_seq=0 out_seq=0 2024-09-10T22:53:04.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdb897fa640 1 -- 172.21.15.96:0/1463033246 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637216068a0 con 0x563721724a70 2024-09-10T22:53:04.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdb897fa640 10 client.26236 handle_command_reply: tid=0 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 client.26236 shutdown 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 msgr2=0x5637216afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 0x5637216afa90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb900e3a40 tx=0x7fdb90009700 comp rx=0 tx=0).stop 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 shutdown_connections 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x563721724a70 0x563721726e90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372169ed80 0x5637216ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563721692df0 0x5637216afa90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdb897fa640 0 client.26236 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 --2- 172.21.15.96:0/1463033246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637216924d0 0x5637216af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 >> 172.21.15.96:0/1463033246 conn(0x563721690300 msgr2=0x56372169d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 shutdown_connections 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 1 -- 172.21.15.96:0/1463033246 wait complete. 2024-09-10T22:53:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.048+0000 7fdbaa1dd740 20 client.26236 trim_cache size 0 max 16384 2024-09-10T22:53:04.054 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 msgr2=0x7fdb8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 0x7fdb8c0b77a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdba0104340 tx=0x7fdb98004a40 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 msgr2=0x7fdba01027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 0x7fdba01027c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9f0 tx=0x7fdb9000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 shutdown_connections 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdba01055d0 0x7fdba01027c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb8c0b52e0 0x7fdb8c0b77a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba0105f90 0x7fdba0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 --2- 172.21.15.96:0/1674319649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdba01012b0 0x7fdba0102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:04.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 >> 172.21.15.96:0/1674319649 conn(0x7fdba0078140 msgr2=0x7fdba0108fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:04.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 shutdown_connections 2024-09-10T22:53:04.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:04.049+0000 7fdba7e08640 1 -- 172.21.15.96:0/1674319649 wait complete. 2024-09-10T22:53:05.164 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:53:05.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:04 smithi096 ceph-mon[30869]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 825 op/s 2024-09-10T22:53:05.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:04 smithi121 ceph-mon[35119]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 825 op/s 2024-09-10T22:53:05.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:04 smithi117 ceph-mon[35081]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 825 op/s 2024-09-10T22:53:05.510 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 -- 172.21.15.96:0/3071661998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102150 msgr2=0x7fc1941025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3071661998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102150 0x7fc1941025d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc188009900 tx=0x7fc18802a410 comp rx=0 tx=0).stop 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 -- 172.21.15.96:0/3071661998 shutdown_connections 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3071661998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102b10 0x7fc194109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3071661998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102150 0x7fc1941025d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3071661998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 -- 172.21.15.96:0/3071661998 >> 172.21.15.96:0/3071661998 conn(0x7fc1940fc440 msgr2=0x7fc1940fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 -- 172.21.15.96:0/3071661998 shutdown_connections 2024-09-10T22:53:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.713+0000 7fc1988d4640 1 -- 172.21.15.96:0/3071661998 wait complete. 2024-09-10T22:53:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 Processor -- start 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 -- start start 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102150 0x7fc194198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102b10 0x7fc194198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d4d0 con 0x7fc194102b10 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d350 con 0x7fc194102150 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc1988d4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc19410d650 con 0x7fc194101190 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc19259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102b10 0x7fc194198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc19259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46904/0 (socket says 172.21.15.96:46904) 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc19259c640 1 -- 172.21.15.96:0/3740488388 learned_addr learned my addr 172.21.15.96:0/3740488388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.714+0000 7fc191d9b640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102150 0x7fc194198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 -- 172.21.15.96:0/3740488388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102150 msgr2=0x7fc194198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102150 0x7fc194198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 -- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102b10 msgr2=0x7fc194198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102b10 0x7fc194198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 -- 172.21.15.96:0/3740488388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194199740 con 0x7fc194101190 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc19259c640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc1837fe640 1 -- 172.21.15.96:0/3740488388 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400dd90 con 0x7fc194101190 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc194199a30 con 0x7fc194101190 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc1837fe640 1 -- 172.21.15.96:0/3740488388 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1840054d0 con 0x7fc194101190 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc1837fe640 1 -- 172.21.15.96:0/3740488388 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184016430 con 0x7fc194101190 2024-09-10T22:53:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.715+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc194109d10 con 0x7fc194101190 2024-09-10T22:53:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 Processor -- start 2024-09-10T22:53:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 -- start start 2024-09-10T22:53:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc1837fe640 1 -- 172.21.15.96:0/3740488388 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc184005020 con 0x7fc194101190 2024-09-10T22:53:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e30190 0x55fee1e32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e32bd0 0x55fee1e35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fee1d9f6f0 con 0x55fee1e32bd0 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fee1d8eb20 con 0x55fee1e2d860 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fee1d92930 con 0x55fee1e30190 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc191d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e32bd0 0x55fee1e35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60476/0 (socket says 172.21.15.96:60476) 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc192d9d640 1 -- 172.21.15.96:0/2486337267 learned_addr learned my addr 172.21.15.96:0/2486337267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.720+0000 7fc19259c640 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e30190 0x55fee1e32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 -- 172.21.15.96:0/2486337267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e30190 msgr2=0x55fee1e32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e30190 0x55fee1e32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 -- 172.21.15.96:0/2486337267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e32bd0 msgr2=0x55fee1e35090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e32bd0 0x55fee1e35090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 -- 172.21.15.96:0/2486337267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee1d54530 con 0x55fee1e2d860 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc192d9d640 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0075f0 tx=0x7fc18c00e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45aaa22e74891d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc180ff9640 1 -- 172.21.15.96:0/2486337267 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00e4e0 con 0x55fee1e2d860 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc1837fe640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 0x7fc17c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc180ff9640 1 -- 172.21.15.96:0/2486337267 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c01e070 con 0x55fee1e2d860 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc180ff9640 1 -- 172.21.15.96:0/2486337267 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0199b0 con 0x55fee1e2d860 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc19259c640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 0x7fc17c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.721+0000 7fc1837fe640 1 -- 172.21.15.96:0/3740488388 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc1840e5880 con 0x7fc194101190 2024-09-10T22:53:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 -- 172.21.15.96:0/2486337267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 msgr2=0x55fee1e2fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19259c640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 0x7fc17c0b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d6b0 tx=0x7fc1840b5040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0075f0 tx=0x7fc18c00e8e0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 -- 172.21.15.96:0/2486337267 shutdown_connections 2024-09-10T22:53:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e32bd0 0x55fee1e35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e30190 0x55fee1e32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/2486337267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e2d860 0x55fee1e2fc50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 -- 172.21.15.96:0/2486337267 >> 172.21.15.96:0/2486337267 conn(0x55fee1e289b0 msgr2=0x55fee1e29740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 -- 172.21.15.96:0/2486337267 shutdown_connections 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.722+0000 7fc19b6a6740 1 -- 172.21.15.96:0/2486337267 wait complete. 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 -- start start 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 Processor -- start 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 0x55fee1e374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e32bd0 0x55fee1e38080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fee1cc6ab0 con 0x55fee1e2d860 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fee1d34730 con 0x55fee1e30190 2024-09-10T22:53:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fee1e35b50 con 0x55fee1e32bd0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc19b6a6740 10 client.0 _ll_register_callbacks cb 0x55fee1d886b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45444/0 (socket says 172.21.15.96:45444) 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc192d9d640 1 -- 172.21.15.96:0/918728084 learned_addr learned my addr 172.21.15.96:0/918728084 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc191d9b640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 0x55fee1e374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc192d9d640 1 -- 172.21.15.96:0/918728084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e32bd0 msgr2=0x55fee1e38080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.723+0000 7fc192d9d640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e32bd0 0x55fee1e38080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc192d9d640 1 -- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 msgr2=0x55fee1e374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc192d9d640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 0x55fee1e374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc192d9d640 1 -- 172.21.15.96:0/918728084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee1af0790 con 0x55fee1e2d860 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc191d9b640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 0x55fee1e374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc192d9d640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00eea0 tx=0x7fc18c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaab0f00de239c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004d00 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc18c00c640 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.724+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c01e070 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.725+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc18c01a9d0 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.725+0000 7fc19b6a6740 10 client.16632 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.725+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fee1b69620 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c00ccf0 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc18c02b030 con 0x55fee1e2d860 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 10 client.16632 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 20 client.16632 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1e449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 4 client.16632 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc18c01e070 con 0x55fee1e425c0 2024-09-10T22:53:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.726+0000 7fc191d9b640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1e449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.727+0000 7fc191d9b640 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1e449b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc333fcde0ae1af server_cookie=de393fd8883bf42a in_seq=0 out_seq=0 2024-09-10T22:53:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.727+0000 7fc179ffb640 10 client.16632 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc179ffb640 1 -- 172.21.15.96:0/918728084 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc18c01e070 con 0x55fee1e425c0 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc179ffb640 10 client.16632 handle_command_reply: tid=0 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 client.16632 shutdown 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 msgr2=0x55fee1e36e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00eea0 tx=0x7fc18c003ef0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 shutdown_connections 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1e449b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc179ffb640 0 client.16632 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e30190 0x55fee1e374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e32bd0 0x55fee1e38080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/918728084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e2d860 0x55fee1e36e10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 >> 172.21.15.96:0/918728084 conn(0x55fee1e289b0 msgr2=0x55fee1e27bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 shutdown_connections 2024-09-10T22:53:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 1 -- 172.21.15.96:0/918728084 wait complete. 2024-09-10T22:53:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.728+0000 7fc19b6a6740 20 client.16632 trim_cache size 0 max 16384 2024-09-10T22:53:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 Processor -- start 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 -- start start 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e2ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e2d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a7b80 con 0x55fee1e30190 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a96c0 con 0x55fee1e32bd0 2024-09-10T22:53:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.757+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a8250 con 0x55fee1e2d860 2024-09-10T22:53:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.758+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.758+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46920/0 (socket says 172.21.15.96:46920) 2024-09-10T22:53:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.758+0000 7fc192d9d640 1 -- 172.21.15.96:0/701322389 learned_addr learned my addr 172.21.15.96:0/701322389 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.758+0000 7fc192d9d640 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e2d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 -- 172.21.15.96:0/701322389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 msgr2=0x55fee1e2d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e2d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 -- 172.21.15.96:0/701322389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 msgr2=0x55fee1e2ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e2ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 -- 172.21.15.96:0/701322389 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee1e328f0 con 0x55fee1e2d860 2024-09-10T22:53:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc192d9d640 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc18c003c20 tx=0x7fc18c00e9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641b65adf4f51d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.762+0000 7fc17bfff640 1 -- 172.21.15.96:0/701322389 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004420 con 0x55fee1e2d860 2024-09-10T22:53:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.763+0000 7fc17bfff640 1 -- 172.21.15.96:0/701322389 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c01e040 con 0x55fee1e2d860 2024-09-10T22:53:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.764+0000 7fc19b6a6740 1 -- 172.21.15.96:0/701322389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 msgr2=0x55fee1e35f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.764+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc18c003c20 tx=0x7fc18c00e9f0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.765+0000 7fc19b6a6740 1 -- 172.21.15.96:0/701322389 shutdown_connections 2024-09-10T22:53:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.765+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e2d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.765+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e2ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.765+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/701322389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e35f20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.765+0000 7fc19b6a6740 1 -- 172.21.15.96:0/701322389 >> 172.21.15.96:0/701322389 conn(0x55fee1e289b0 msgr2=0x55fee1e369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.771+0000 7fc19b6a6740 1 -- 172.21.15.96:0/701322389 shutdown_connections 2024-09-10T22:53:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.771+0000 7fc19b6a6740 1 -- 172.21.15.96:0/701322389 wait complete. 2024-09-10T22:53:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.772+0000 7fc19b6a6740 1 -- start start 2024-09-10T22:53:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.772+0000 7fc19b6a6740 1 Processor -- start 2024-09-10T22:53:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e39e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e3a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a8ff0 con 0x55fee1e30190 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a8920 con 0x55fee1e32bd0 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941a5b60 con 0x55fee1e2d860 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc19b6a6740 10 client.0 _ll_register_callbacks cb 0x7fc188009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.773+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.774+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46928/0 (socket says 172.21.15.96:46928) 2024-09-10T22:53:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.774+0000 7fc192d9d640 1 -- 172.21.15.96:0/411725837 learned_addr learned my addr 172.21.15.96:0/411725837 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.774+0000 7fc192d9d640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e3a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.775+0000 7fc192d9d640 1 -- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 msgr2=0x55fee1e3a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.775+0000 7fc192d9d640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e3a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.775+0000 7fc192d9d640 1 -- 172.21.15.96:0/411725837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 msgr2=0x55fee1e39e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.775+0000 7fc192d9d640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e39e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.776+0000 7fc192d9d640 1 -- 172.21.15.96:0/411725837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee1e3b210 con 0x55fee1e2d860 2024-09-10T22:53:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.777+0000 7fc192d9d640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007380 tx=0x7fc18c010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b455c3bc058073 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c01a460 con 0x55fee1e2d860 2024-09-10T22:53:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc18c00c3b0 con 0x55fee1e2d860 2024-09-10T22:53:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c01e040 con 0x55fee1e2d860 2024-09-10T22:53:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc18c01ab70 con 0x55fee1e2d860 2024-09-10T22:53:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc19b6a6740 10 client.26251 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.778+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fee1e3e590 con 0x55fee1e2d860 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c01e040 con 0x55fee1e2d860 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc18c02b070 con 0x55fee1e2d860 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 10 client.26251 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 20 client.26251 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1ebe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 4 client.26251 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.779+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fee1da1640 con 0x55fee1e425c0 2024-09-10T22:53:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.791+0000 7fc191d9b640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1ebe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.794+0000 7fc191d9b640 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1ebe410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a50179f173fdc70 server_cookie=650c1d01e859d6d3 in_seq=0 out_seq=0 2024-09-10T22:53:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.795+0000 7fc179ffb640 10 client.26251 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc179ffb640 1 -- 172.21.15.96:0/411725837 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fee1da1640 con 0x55fee1e425c0 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc179ffb640 10 client.26251 handle_command_reply: tid=0 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 client.26251 shutdown 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 msgr2=0x55fee1e39770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007380 tx=0x7fc18c010040 comp rx=0 tx=0).stop 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 shutdown_connections 2024-09-10T22:53:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fee1e425c0 0x55fee1ebe410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fee1e32bd0 0x55fee1e3a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fee1e30190 0x55fee1e39e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 --2- 172.21.15.96:0/411725837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fee1e2d860 0x55fee1e39770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc179ffb640 0 client.26251 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 >> 172.21.15.96:0/411725837 conn(0x55fee1e289b0 msgr2=0x55fee1e35f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 shutdown_connections 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.823+0000 7fc19b6a6740 1 -- 172.21.15.96:0/411725837 wait complete. 2024-09-10T22:53:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc19b6a6740 20 client.26251 trim_cache size 0 max 16384 2024-09-10T22:53:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:53:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:53:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:53:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:53:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 msgr2=0x7fc17c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 0x7fc17c0b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d6b0 tx=0x7fc1840b5040 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 msgr2=0x7fc194197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400edf0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 shutdown_connections 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.824+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc194102150 0x7fc194198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc17c0b52e0 0x7fc17c0b77a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc194102b10 0x7fc194198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 --2- 172.21.15.96:0/3740488388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc194101190 0x7fc194197c00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 >> 172.21.15.96:0/3740488388 conn(0x7fc1940fc440 msgr2=0x7fc194105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 shutdown_connections 2024-09-10T22:53:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:05.825+0000 7fc1988d4640 1 -- 172.21.15.96:0/3740488388 wait complete. 2024-09-10T22:53:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 330.0 s, 3211.8 tps, lat 31.256 ms stddev 35.842 2024-09-10T22:53:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.069+0000 7f8c4cb89640 1 -- 172.21.15.96:0/1348785555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c481045e0 msgr2=0x7f8c4810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.069+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/1348785555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c481045e0 0x7f8c4810cad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 -- 172.21.15.96:0/1348785555 shutdown_connections 2024-09-10T22:53:06.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/1348785555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c4810d120 0x7f8c4810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/1348785555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c481045e0 0x7f8c4810cad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/1348785555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c481040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 -- 172.21.15.96:0/1348785555 >> 172.21.15.96:0/1348785555 conn(0x7f8c480fd700 msgr2=0x7f8c480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 -- 172.21.15.96:0/1348785555 shutdown_connections 2024-09-10T22:53:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.070+0000 7f8c4cb89640 1 -- 172.21.15.96:0/1348785555 wait complete. 2024-09-10T22:53:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.071+0000 7f8c4cb89640 1 Processor -- start 2024-09-10T22:53:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 -- start start 2024-09-10T22:53:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 0x7f8c48101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c4810d120 0x7f8c481023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48112d90 con 0x7f8c48103cc0 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48112c10 con 0x7f8c481045e0 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.072+0000 7f8c4cb89640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48112f10 con 0x7f8c4810d120 2024-09-10T22:53:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 0x7f8c48101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45464/0 (socket says 172.21.15.96:45464) 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 -- 172.21.15.96:0/2222457719 learned_addr learned my addr 172.21.15.96:0/2222457719 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c4810d120 0x7f8c481023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 -- 172.21.15.96:0/2222457719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c4810d120 msgr2=0x7f8c481023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c4810d120 0x7f8c481023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 -- 172.21.15.96:0/2222457719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 msgr2=0x7f8c48101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 0x7f8c48101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 -- 172.21.15.96:0/2222457719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c481a7700 con 0x7f8c48103cc0 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 0x7f8c48101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4659c640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800ea60 tx=0x7f8c3800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c377fe640 1 -- 172.21.15.96:0/2222457719 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800cd20 con 0x7f8c48103cc0 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c377fe640 1 -- 172.21.15.96:0/2222457719 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c380035f0 con 0x7f8c48103cc0 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481a79f0 con 0x7f8c48103cc0 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c377fe640 1 -- 172.21.15.96:0/2222457719 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38010690 con 0x7f8c48103cc0 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.073+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a7f50 con 0x7f8c48103cc0 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.077+0000 7f8c4ef5e740 1 Processor -- start 2024-09-10T22:53:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 -- start start 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdb4d0 0x55ababbdb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ababb51ea0 con 0x55ababbdb4d0 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ababb412d0 con 0x55ababbdbdf0 2024-09-10T22:53:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ababb450e0 con 0x55ababbe7d80 2024-09-10T22:53:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c46d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdb4d0 0x55ababbdb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:46966/0 (socket says 172.21.15.96:46966) 2024-09-10T22:53:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c45d9b640 1 -- 172.21.15.96:0/116678006 learned_addr learned my addr 172.21.15.96:0/116678006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.078+0000 7f8c377fe640 1 -- 172.21.15.96:0/2222457719 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f8c38004000 con 0x7f8c48103cc0 2024-09-10T22:53:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 -- 172.21.15.96:0/116678006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 msgr2=0x55ababbe7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 -- 172.21.15.96:0/116678006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdb4d0 msgr2=0x55ababbdb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdb4d0 0x55ababbdb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 -- 172.21.15.96:0/116678006 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abab89f650 con 0x55ababbe7d80 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c4806c2d0 tx=0x7f8c3c005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1dce6915416d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c34ff9640 1 -- 172.21.15.96:0/116678006 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c040070 con 0x55ababbe7d80 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c34ff9640 1 -- 172.21.15.96:0/116678006 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c031820 con 0x55ababbe7d80 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c34ff9640 1 -- 172.21.15.96:0/116678006 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c035040 con 0x55ababbe7d80 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.079+0000 7f8c377fe640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 0x7f8c300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.080+0000 7f8c377fe640 1 -- 172.21.15.96:0/2222457719 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8c380e5d70 con 0x7f8c48103cc0 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.080+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 0x7f8c300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/116678006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 msgr2=0x55ababbea170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c4806c2d0 tx=0x7f8c3c005e20 comp rx=0 tx=0).stop 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 0x7f8c300b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c48103a30 tx=0x7f8c40009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/116678006 shutdown_connections 2024-09-10T22:53:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababbea170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/116678006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdb4d0 0x55ababbdb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.081+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/116678006 >> 172.21.15.96:0/116678006 conn(0x55ababbd9300 msgr2=0x55ababbda4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.082+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/116678006 shutdown_connections 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.082+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/116678006 wait complete. 2024-09-10T22:53:06.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.082+0000 7f8c4ef5e740 1 -- start start 2024-09-10T22:53:06.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 Processor -- start 2024-09-10T22:53:06.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababbe4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 0x55ababbe50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abab9a2700 con 0x55ababbdbdf0 2024-09-10T22:53:06.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ababadc160 con 0x55ababbe7d80 2024-09-10T22:53:06.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ababbea9a0 con 0x55ababbdb4d0 2024-09-10T22:53:06.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.083+0000 7f8c4ef5e740 10 client.0 _ll_register_callbacks cb 0x55ababaf59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52486/0 (socket says 172.21.15.96:52486) 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababbe4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 0x55ababbe50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c4659c640 1 -- 172.21.15.96:0/3732819449 learned_addr learned my addr 172.21.15.96:0/3732819449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 -- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 msgr2=0x55ababbe7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 -- 172.21.15.96:0/3732819449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 msgr2=0x55ababbe50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 0x55ababbe50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c45d9b640 1 -- 172.21.15.96:0/3732819449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ababb09c90 con 0x55ababbdbdf0 2024-09-10T22:53:06.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 0x55ababbe50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.084+0000 7f8c4659c640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababbe4ab0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02a910 tx=0x7f8c3c0319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97d111efdec8b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c04b070 con 0x55ababbdbdf0 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8c3c0035f0 con 0x55ababbdbdf0 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c031820 con 0x55ababbdbdf0 2024-09-10T22:53:06.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c057050 con 0x55ababbdbdf0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c4ef5e740 10 client.16644 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.086+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ababbfeb00 con 0x55ababbdbdf0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.087+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c031820 con 0x55ababbdbdf0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8c3c03c770 con 0x55ababbdbdf0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 10 client.16644 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 10 client.16644 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 20 client.16644 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababbffe50 0x55ababc02240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 4 client.16644 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ababb53df0 con 0x55ababbffe50 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.088+0000 7f8c4659c640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababbffe50 0x55ababc02240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.089+0000 7f8c4659c640 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababbffe50 0x55ababc02240 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39316d46533567d8 server_cookie=b3ce673c188050f8 in_seq=0 out_seq=0 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.089+0000 7f8c2dffb640 10 client.16644 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:06.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.090+0000 7f8c2dffb640 1 -- 172.21.15.96:0/3732819449 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ababb53df0 con 0x55ababbffe50 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.090+0000 7f8c2dffb640 10 client.16644 handle_command_reply: tid=0 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.090+0000 7f8c4ef5e740 1 client.16644 shutdown 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.090+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 msgr2=0x55ababbe4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.090+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababbe4ab0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02a910 tx=0x7f8c3c0319d0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 shutdown_connections 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababbffe50 0x55ababc02240 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbe7d80 0x55ababbe50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababbe4ab0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/3732819449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbe7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 >> 172.21.15.96:0/3732819449 conn(0x55ababbd9300 msgr2=0x55ababbdee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c2dffb640 0 client.16644 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 shutdown_connections 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/3732819449 wait complete. 2024-09-10T22:53:06.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.091+0000 7f8c4ef5e740 20 client.16644 trim_cache size 0 max 16384 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 Processor -- start 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 -- start start 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbe7d80 0x55ababbe96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481ac090 con 0x55ababbe7d80 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481adbd0 con 0x55ababbdbdf0 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481ac760 con 0x55ababbdb4d0 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.109+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52498/0 (socket says 172.21.15.96:52498) 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56138/0 (socket says 172.21.15.96:56138) 2024-09-10T22:53:06.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c4659c640 1 -- 172.21.15.96:0/2652229756 learned_addr learned my addr 172.21.15.96:0/2652229756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbe7d80 0x55ababbe96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 -- 172.21.15.96:0/2652229756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 msgr2=0x55ababbd8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 -- 172.21.15.96:0/2652229756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbe7d80 msgr2=0x55ababbe96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbe7d80 0x55ababbe96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 -- 172.21.15.96:0/2652229756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ababbd9d80 con 0x55ababbdbdf0 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c4659c640 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0055e0 tx=0x7f8c3c046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590f8a373239bc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c2ffff640 1 -- 172.21.15.96:0/2652229756 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c04b070 con 0x55ababbdbdf0 2024-09-10T22:53:06.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c2ffff640 1 -- 172.21.15.96:0/2652229756 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c03c8f0 con 0x55ababbdbdf0 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.110+0000 7f8c2ffff640 1 -- 172.21.15.96:0/2652229756 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c035030 con 0x55ababbdbdf0 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/2652229756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 msgr2=0x55ababbe9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0055e0 tx=0x7f8c3c046040 comp rx=0 tx=0).stop 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/2652229756 shutdown_connections 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdbdf0 0x55ababbe9170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbe7d80 0x55ababbe96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/2652229756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbdb4d0 0x55ababbd8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/2652229756 >> 172.21.15.96:0/2652229756 conn(0x55ababbd9300 msgr2=0x55ababbda350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/2652229756 shutdown_connections 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.112+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/2652229756 wait complete. 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.113+0000 7f8c4ef5e740 1 -- start start 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.113+0000 7f8c4ef5e740 1 Processor -- start 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababc057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababc008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481ad500 con 0x55ababbdbdf0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481ace30 con 0x55ababbdb4d0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481aa090 con 0x55ababbe7d80 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4ef5e740 10 client.0 _ll_register_callbacks cb 0x7f8c40009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56152/0 (socket says 172.21.15.96:56152) 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 -- 172.21.15.96:0/485400094 learned_addr learned my addr 172.21.15.96:0/485400094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababc057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c46d9d640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababc008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 -- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 msgr2=0x55ababc008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababc008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 -- 172.21.15.96:0/485400094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 msgr2=0x55ababc057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:06.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababc057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.114+0000 7f8c4659c640 1 -- 172.21.15.96:0/485400094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ababc010e0 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4659c640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004610 tx=0x7f8c38003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b8188f13cc106a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380d3020 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c03c8f0 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8c380a8840 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c380ac0e0 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4ef5e740 10 client.26234 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ababc6c680 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c03c8f0 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8c380a7f20 con 0x55ababbdb4d0 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4ef5e740 10 client.26234 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.115+0000 7f8c4ef5e740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.116+0000 7f8c4ef5e740 20 client.26234 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.116+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababc6ce00 0x55ababc6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.116+0000 7f8c4ef5e740 4 client.26234 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.116+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ababb4f8a0 con 0x55ababc6ce00 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.116+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababc6ce00 0x55ababc6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.117+0000 7f8c2dffb640 10 client.26234 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.117+0000 7f8c45d9b640 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababc6ce00 0x55ababc6f220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ab21f1b7d68bf8b server_cookie=edf1568fc7123a2f in_seq=0 out_seq=0 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.118+0000 7f8c2dffb640 1 -- 172.21.15.96:0/485400094 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ababb4f8a0 con 0x55ababc6ce00 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.118+0000 7f8c2dffb640 10 client.26234 handle_command_reply: tid=0 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.118+0000 7f8c4ef5e740 1 client.26234 shutdown 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.118+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 msgr2=0x55ababc05250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.118+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004610 tx=0x7f8c38003dd0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 shutdown_connections 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ababbe7d80 0x55ababc008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ababbdbdf0 0x55ababc057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55ababc6ce00 0x55ababc6f220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c2dffb640 0 client.26234 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 --2- 172.21.15.96:0/485400094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ababbdb4d0 0x55ababc05250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 >> 172.21.15.96:0/485400094 conn(0x55ababbd9300 msgr2=0x55ababbe60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 shutdown_connections 2024-09-10T22:53:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 1 -- 172.21.15.96:0/485400094 wait complete. 2024-09-10T22:53:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.119+0000 7f8c4ef5e740 20 client.26234 trim_cache size 0 max 16384 2024-09-10T22:53:06.183 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 msgr2=0x7f8c300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 0x7f8c300b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c48103a30 tx=0x7f8c40009500 comp rx=0 tx=0).stop 2024-09-10T22:53:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 msgr2=0x7f8c48101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800ea60 tx=0x7f8c3800ee60 comp rx=0 tx=0).stop 2024-09-10T22:53:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 shutdown_connections 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c481045e0 0x7f8c48101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8c300b5360 0x7f8c300b7820 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c4810d120 0x7f8c481023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 --2- 172.21.15.96:0/2222457719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c48103cc0 0x7f8c48101970 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 >> 172.21.15.96:0/2222457719 conn(0x7f8c480fd700 msgr2=0x7f8c480fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 shutdown_connections 2024-09-10T22:53:06.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:06.120+0000 7f8c4cb89640 1 -- 172.21.15.96:0/2222457719 wait complete. 2024-09-10T22:53:07.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:06 smithi096 ceph-mon[30869]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 886 op/s 2024-09-10T22:53:07.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:06 smithi121 ceph-mon[35119]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 886 op/s 2024-09-10T22:53:07.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:06 smithi117 ceph-mon[35081]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 886 op/s 2024-09-10T22:53:07.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:53:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:08.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.339+0000 7f13ab26e640 1 -- 172.21.15.96:0/2726683814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 msgr2=0x7f13a4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.339+0000 7f13ab26e640 1 --2- 172.21.15.96:0/2726683814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4102070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009900 tx=0x7f13a002a410 comp rx=0 tx=0).stop 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 -- 172.21.15.96:0/2726683814 shutdown_connections 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 --2- 172.21.15.96:0/2726683814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a41076a0 0x7f13a4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 --2- 172.21.15.96:0/2726683814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a4108020 0x7f13a410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 --2- 172.21.15.96:0/2726683814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4102070 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 -- 172.21.15.96:0/2726683814 >> 172.21.15.96:0/2726683814 conn(0x7f13a40fd700 msgr2=0x7f13a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 -- 172.21.15.96:0/2726683814 shutdown_connections 2024-09-10T22:53:08.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.340+0000 7f13ab26e640 1 -- 172.21.15.96:0/2726683814 wait complete. 2024-09-10T22:53:08.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 Processor -- start 2024-09-10T22:53:08.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 -- start start 2024-09-10T22:53:08.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 0x7f13a419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13a900a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13a900a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49952/0 (socket says 172.21.15.96:49952) 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 0x7f13a419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13a900a640 1 -- 172.21.15.96:0/526989979 learned_addr learned my addr 172.21.15.96:0/526989979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4111d30 con 0x7f13a4101c90 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4111bb0 con 0x7f13a4108020 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4111eb0 con 0x7f13a41076a0 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13a8809640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 0x7f13a419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.341+0000 7f13a980b640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 0x7f13a419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 -- 172.21.15.96:0/526989979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 msgr2=0x7f13a419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 0x7f13a419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 -- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 msgr2=0x7f13a4199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41a52e0 con 0x7f13a41076a0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a900a640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a980b640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 0x7f13a419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13a8809640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 0x7f13a419a440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1394009f20 tx=0x7f1394007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f139a7fc640 1 -- 172.21.15.96:0/526989979 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394007d60 con 0x7f13a41076a0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a4197d50 con 0x7f13a41076a0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f139a7fc640 1 -- 172.21.15.96:0/526989979 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13940054d0 con 0x7f13a41076a0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f139a7fc640 1 -- 172.21.15.96:0/526989979 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139401e420 con 0x7f13a41076a0 2024-09-10T22:53:08.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.342+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41982e0 con 0x7f13a41076a0 2024-09-10T22:53:08.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.348+0000 7f139a7fc640 1 -- 172.21.15.96:0/526989979 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1394005000 con 0x7f13a41076a0 2024-09-10T22:53:08.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.348+0000 7f13ad643740 1 Processor -- start 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.348+0000 7f13ad643740 1 -- start start 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556192ef8ea0 con 0x556192f8e9a0 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556192ee82d0 con 0x556192f82a10 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13ad643740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556192eec0e0 con 0x556192f820f0 2024-09-10T22:53:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f139a7fc640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 0x7f13900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a980b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56178/0 (socket says 172.21.15.96:56178) 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 -- 172.21.15.96:0/779602811 learned_addr learned my addr 172.21.15.96:0/779602811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f139a7fc640 1 -- 172.21.15.96:0/526989979 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13940ed3e0 con 0x7f13a41076a0 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a8809640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 -- 172.21.15.96:0/779602811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 msgr2=0x556192f824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 -- 172.21.15.96:0/779602811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 msgr2=0x556192f90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a900a640 1 -- 172.21.15.96:0/779602811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556192c46650 con 0x556192f82a10 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a8809640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.349+0000 7f13a980b640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:08.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.350+0000 7f13a980b640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 0x7f13900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.350+0000 7f13a900a640 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b1b0 tx=0x7f13a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6f3987ce9b8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.350+0000 7f138f7fe640 1 -- 172.21.15.96:0/779602811 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0040070 con 0x556192f82a10 2024-09-10T22:53:08.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.350+0000 7f138f7fe640 1 -- 172.21.15.96:0/779602811 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a0032d10 con 0x556192f82a10 2024-09-10T22:53:08.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.350+0000 7f138f7fe640 1 -- 172.21.15.96:0/779602811 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0035040 con 0x556192f82a10 2024-09-10T22:53:08.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 -- 172.21.15.96:0/779602811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 msgr2=0x556192f8e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b1b0 tx=0x7f13a0000bc0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13a980b640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 0x7f13900b7820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f139c005c40 tx=0x7f139c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 -- 172.21.15.96:0/779602811 shutdown_connections 2024-09-10T22:53:08.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8e460 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 --2- 172.21.15.96:0/779602811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 -- 172.21.15.96:0/779602811 >> 172.21.15.96:0/779602811 conn(0x556192f7ff20 msgr2=0x556192f810c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 -- 172.21.15.96:0/779602811 shutdown_connections 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.351+0000 7f13ad643740 1 -- 172.21.15.96:0/779602811 wait complete. 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.352+0000 7f13ad643740 1 -- start start 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.352+0000 7f13ad643740 1 Processor -- start 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 0x556192f8b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 0x556192f8bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49974/0 (socket says 172.21.15.96:49974) 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 -- 172.21.15.96:0/787996379 learned_addr learned my addr 172.21.15.96:0/787996379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556192d49700 con 0x556192f820f0 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556192e83160 con 0x556192f8e9a0 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556192f915c0 con 0x556192f82a10 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13ad643740 10 client.0 _ll_register_callbacks cb 0x556192e9c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a8809640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 0x556192f8b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a980b640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 0x556192f8bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 -- 172.21.15.96:0/787996379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 msgr2=0x556192f8b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 0x556192f8b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 -- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 msgr2=0x556192f8bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 0x556192f8bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.353+0000 7f13a900a640 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556192eb0c90 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.354+0000 7f13a8809640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 0x556192f8b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.354+0000 7f13a980b640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 0x556192f8bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.354+0000 7f13a900a640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b230 tx=0x7f13a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dc771420aca3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.354+0000 7f138cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0035070 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13a0032a70 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a0032d10 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13a0040050 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f13ad643740 10 client.16662 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.355+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556192f9cf40 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a0009520 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f13a003c820 con 0x556192f820f0 2024-09-10T22:53:08.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 10 client.16662 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 20 client.16662 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556192f9e290 0x556192fa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 4 client.16662 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13a0032d10 con 0x556192f9e290 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.356+0000 7f13a8809640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556192f9e290 0x556192fa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.357+0000 7f13a8809640 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556192f9e290 0x556192fa0680 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb942bff6b5c3a23 server_cookie=d4f95de62b25416e in_seq=0 out_seq=0 2024-09-10T22:53:08.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.357+0000 7f138cff9640 10 client.16662 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f138cff9640 1 -- 172.21.15.96:0/787996379 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f13a0032d10 con 0x556192f9e290 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f138cff9640 10 client.16662 handle_command_reply: tid=0 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 client.16662 shutdown 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 msgr2=0x556192f8e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b230 tx=0x7f13a0005fd0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 shutdown_connections 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556192f9e290 0x556192fa0680 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f138cff9640 0 client.16662 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:08.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f8e9a0 0x556192f8bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f82a10 0x556192f8b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 --2- 172.21.15.96:0/787996379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f820f0 0x556192f8e2e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 >> 172.21.15.96:0/787996379 conn(0x556192f7ff20 msgr2=0x556192f85ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.358+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 shutdown_connections 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.359+0000 7f13ad643740 1 -- 172.21.15.96:0/787996379 wait complete. 2024-09-10T22:53:08.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.359+0000 7f13ad643740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 Processor -- start 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 -- start start 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41ac210 con 0x556192f8e9a0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41add50 con 0x556192f82a10 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13ad643740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41ac8e0 con 0x556192f820f0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a900a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a980b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a980b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49986/0 (socket says 172.21.15.96:49986) 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a900a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52546/0 (socket says 172.21.15.96:52546) 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a980b640 1 -- 172.21.15.96:0/2811448199 learned_addr learned my addr 172.21.15.96:0/2811448199 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.376+0000 7f13a8809640 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 -- 172.21.15.96:0/2811448199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 msgr2=0x556192f8fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 -- 172.21.15.96:0/2811448199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 msgr2=0x556192f902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 -- 172.21.15.96:0/2811448199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556192f8c120 con 0x556192f820f0 2024-09-10T22:53:08.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f13a900a640 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13a003cd90 tx=0x7f13a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=747342cf2482fc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f138effd640 1 -- 172.21.15.96:0/2811448199 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0035070 con 0x556192f820f0 2024-09-10T22:53:08.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f138effd640 1 -- 172.21.15.96:0/2811448199 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a0005850 con 0x556192f820f0 2024-09-10T22:53:08.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.377+0000 7f138effd640 1 -- 172.21.15.96:0/2811448199 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a003c820 con 0x556192f820f0 2024-09-10T22:53:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.378+0000 7f13ad643740 1 -- 172.21.15.96:0/2811448199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 msgr2=0x556192f9a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.378+0000 7f13ad643740 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13a003cd90 tx=0x7f13a0000bc0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.379+0000 7f13ad643740 1 -- 172.21.15.96:0/2811448199 shutdown_connections 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.379+0000 7f13ad643740 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f82a10 0x556192f8fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.379+0000 7f13ad643740 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f8e9a0 0x556192f902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.379+0000 7f13ad643740 1 --2- 172.21.15.96:0/2811448199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f820f0 0x556192f9a860 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.379+0000 7f13ad643740 1 -- 172.21.15.96:0/2811448199 >> 172.21.15.96:0/2811448199 conn(0x556192f7ff20 msgr2=0x556192f89fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.380+0000 7f13ad643740 1 -- 172.21.15.96:0/2811448199 shutdown_connections 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.380+0000 7f13ad643740 1 -- 172.21.15.96:0/2811448199 wait complete. 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.380+0000 7f13ad643740 1 -- start start 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.380+0000 7f13ad643740 1 Processor -- start 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 0x556192f9fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f82a10 0x556192fa0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41ad680 con 0x556192f82a10 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41acfb0 con 0x556192f820f0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41aa230 con 0x556192f8e9a0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13ad643740 10 client.0 _ll_register_callbacks cb 0x7f139c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13a980b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13a900a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 0x556192f9fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13a980b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52550/0 (socket says 172.21.15.96:52550) 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13a980b640 1 -- 172.21.15.96:0/3568186642 learned_addr learned my addr 172.21.15.96:0/3568186642 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.381+0000 7f13a8809640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f82a10 0x556192fa0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 -- 172.21.15.96:0/3568186642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 msgr2=0x556193009df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 -- 172.21.15.96:0/3568186642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f82a10 msgr2=0x556192fa0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f82a10 0x556192fa0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 -- 172.21.15.96:0/3568186642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55619300a430 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a980b640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13a900a640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 0x556192f9fde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f13a003cc20 tx=0x7f13a0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2af9884c205de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f138cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0064060 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13a0032a70 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.382+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a0005850 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13a0035040 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 10 client.26249 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55619300b730 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a0005850 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f13a0069070 con 0x556192f820f0 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 10 client.26249 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:08.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 20 client.26249 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55619300be10 0x55619300e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 4 client.26249 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556192efadf0 con 0x55619300be10 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.383+0000 7f13a8809640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55619300be10 0x55619300e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f138cff9640 10 client.26249 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13a8809640 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55619300be10 0x55619300e230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ad23f1757961 server_cookie=366422bf76bac4f2 in_seq=0 out_seq=0 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f138cff9640 1 -- 172.21.15.96:0/3568186642 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556192efadf0 con 0x55619300be10 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f138cff9640 10 client.26249 handle_command_reply: tid=0 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 client.26249 shutdown 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 msgr2=0x556192f9fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 0x556192f9fde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f13a003cc20 tx=0x7f13a0005fd0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 shutdown_connections 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55619300be10 0x55619300e230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556192f8e9a0 0x556193009df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556192f82a10 0x556192fa0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 --2- 172.21.15.96:0/3568186642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556192f820f0 0x556192f9fde0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f138cff9640 0 client.26249 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 >> 172.21.15.96:0/3568186642 conn(0x556192f7ff20 msgr2=0x556192f8cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.384+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 shutdown_connections 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ad643740 1 -- 172.21.15.96:0/3568186642 wait complete. 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ad643740 20 client.26249 trim_cache size 0 max 16384 2024-09-10T22:53:08.391 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:08.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 msgr2=0x7f13900b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 0x7f13900b7820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f139c005c40 tx=0x7f139c004a40 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 msgr2=0x7f13a419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.385+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 0x7f13a419a440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1394009f20 tx=0x7f1394007590 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 shutdown_connections 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4108020 0x7f13a419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13900b5360 0x7f13900b7820 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13a41076a0 0x7f13a419a440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 --2- 172.21.15.96:0/526989979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13a4101c90 0x7f13a4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 >> 172.21.15.96:0/526989979 conn(0x7f13a40fd700 msgr2=0x7f13a41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 shutdown_connections 2024-09-10T22:53:08.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:08.386+0000 7f13ab26e640 1 -- 172.21.15.96:0/526989979 wait complete. 2024-09-10T22:53:09.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:08 smithi096 ceph-mon[30869]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-10T22:53:09.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:08 smithi121 ceph-mon[35119]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-10T22:53:09.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:08 smithi117 ceph-mon[35081]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-10T22:53:10.334 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:53:10.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:53:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 335.0 s, 2867.5 tps, lat 34.854 ms stddev 39.550 2024-09-10T22:53:11.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:10 smithi096 ceph-mon[30869]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 942 op/s 2024-09-10T22:53:11.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:10 smithi121 ceph-mon[35119]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 942 op/s 2024-09-10T22:53:11.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:10 smithi117 ceph-mon[35081]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 942 op/s 2024-09-10T22:53:11.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2694790841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c741045c0 msgr2=0x7f5c7410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2694790841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c741045c0 0x7f5c7410cab0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009900 tx=0x7f5c6802a400 comp rx=0 tx=0).stop 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2694790841 shutdown_connections 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2694790841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c7410d100 0x7f5c7410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2694790841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c741045c0 0x7f5c7410cab0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2694790841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c74103ca0 0x7f5c74104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2694790841 >> 172.21.15.96:0/2694790841 conn(0x7f5c740fd700 msgr2=0x7f5c740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2694790841 shutdown_connections 2024-09-10T22:53:11.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.625+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2694790841 wait complete. 2024-09-10T22:53:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.626+0000 7f5c7a83f640 1 Processor -- start 2024-09-10T22:53:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.626+0000 7f5c7a83f640 1 -- start start 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 0x7f5c74103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7410d100 0x7f5c74101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74112d70 con 0x7f5c7410d100 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74112bf0 con 0x7f5c74103ca0 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c7a83f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74112ef0 con 0x7f5c741045c0 2024-09-10T22:53:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c737fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c737fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52574/0 (socket says 172.21.15.96:52574) 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7410d100 0x7f5c74101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c737fe640 1 -- 172.21.15.96:0/2326916744 learned_addr learned my addr 172.21.15.96:0/2326916744 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.627+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 0x7f5c74103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 -- 172.21.15.96:0/2326916744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 msgr2=0x7f5c74103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 0x7f5c74103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 -- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7410d100 msgr2=0x7f5c74101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7410d100 0x7f5c74101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 -- 172.21.15.96:0/2326916744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c741abc20 con 0x7f5c741045c0 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 0x7f5c74103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c737fe640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c68032fd0 tx=0x7f5c68031970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c717fa640 1 -- 172.21.15.96:0/2326916744 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6803f030 con 0x7f5c741045c0 2024-09-10T22:53:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c717fa640 1 -- 172.21.15.96:0/2326916744 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c68004410 con 0x7f5c741045c0 2024-09-10T22:53:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c741abdf0 con 0x7f5c741045c0 2024-09-10T22:53:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c717fa640 1 -- 172.21.15.96:0/2326916744 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68035070 con 0x7f5c741045c0 2024-09-10T22:53:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.628+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741ac1e0 con 0x7f5c741045c0 2024-09-10T22:53:11.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.631+0000 7f5c717fa640 1 -- 172.21.15.96:0/2326916744 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5c6803b9c0 con 0x7f5c741045c0 2024-09-10T22:53:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.633+0000 7f5c717fa640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 0x7f5c5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.633+0000 7f5c7cc14740 1 Processor -- start 2024-09-10T22:53:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.633+0000 7f5c717fa640 1 -- 172.21.15.96:0/2326916744 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c68106100 con 0x7f5c741045c0 2024-09-10T22:53:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.633+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 0x7f5c5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 0x7f5c5c0b77a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009c20 tx=0x7f5c64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 -- start start 2024-09-10T22:53:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d32c800 0x55b21d338250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d338790 0x55b21d33ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b21d2a2e60 con 0x55b21d32bee0 2024-09-10T22:53:11.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b21d292290 con 0x55b21d338790 2024-09-10T22:53:11.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b21d2960a0 con 0x55b21d32c800 2024-09-10T22:53:11.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50014/0 (socket says 172.21.15.96:50014) 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 -- 172.21.15.96:0/2302623356 learned_addr learned my addr 172.21.15.96:0/2302623356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c737fe640 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d32c800 0x55b21d338250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 -- 172.21.15.96:0/2302623356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d32c800 msgr2=0x55b21d338250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d32c800 0x55b21d338250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 -- 172.21.15.96:0/2302623356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d338790 msgr2=0x55b21d33ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d338790 0x55b21d33ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.634+0000 7f5c73fff640 1 -- 172.21.15.96:0/2302623356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21cff0930 con 0x55b21d32bee0 2024-09-10T22:53:11.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.635+0000 7f5c73fff640 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004190 tx=0x7f5c64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f1f4e832468d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.635+0000 7f5c5a7fc640 1 -- 172.21.15.96:0/2302623356 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64003e00 con 0x55b21d32bee0 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.635+0000 7f5c5a7fc640 1 -- 172.21.15.96:0/2302623356 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c64025070 con 0x55b21d32bee0 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.636+0000 7f5c7cc14740 1 -- 172.21.15.96:0/2302623356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 msgr2=0x55b21d32c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.636+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5c64004190 tx=0x7f5c64014000 comp rx=0 tx=0).stop 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.637+0000 7f5c7cc14740 1 -- 172.21.15.96:0/2302623356 shutdown_connections 2024-09-10T22:53:11.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.637+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d338790 0x55b21d33ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.637+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d32c800 0x55b21d338250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.637+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/2302623356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d32c2c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.637+0000 7f5c7cc14740 1 -- 172.21.15.96:0/2302623356 >> 172.21.15.96:0/2302623356 conn(0x55b21d329d10 msgr2=0x55b21d32aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.640+0000 7f5c7cc14740 1 -- 172.21.15.96:0/2302623356 shutdown_connections 2024-09-10T22:53:11.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.641+0000 7f5c7cc14740 1 -- 172.21.15.96:0/2302623356 wait complete. 2024-09-10T22:53:11.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.641+0000 7f5c7cc14740 1 -- start start 2024-09-10T22:53:11.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 Processor -- start 2024-09-10T22:53:11.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d338210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d335660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b21d0f3890 con 0x55b21d32bee0 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.642+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b21d22cd40 con 0x55b21d32c800 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.643+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b21d33b490 con 0x55b21d338790 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.643+0000 7f5c7cc14740 10 client.0 _ll_register_callbacks cb 0x55b21d290c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52594/0 (socket says 172.21.15.96:52594) 2024-09-10T22:53:11.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 -- 172.21.15.96:0/3253333461 learned_addr learned my addr 172.21.15.96:0/3253333461 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:11.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 -- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 msgr2=0x55b21d335660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:53:11.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d335660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 -- 172.21.15.96:0/3253333461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 msgr2=0x55b21d338210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d338210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.644+0000 7f5c78ddc640 1 -- 172.21.15.96:0/3253333461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21d344810 con 0x55b21d338790 2024-09-10T22:53:11.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.646+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0119e0 tx=0x7f5c6c011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab49233df04a524 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.646+0000 7f5c537fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:11.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.646+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c003c40 con 0x55b21d338790 2024-09-10T22:53:11.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.647+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c64025070 con 0x55b21d338790 2024-09-10T22:53:11.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.648+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c6c003de0 con 0x55b21d338790 2024-09-10T22:53:11.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.648+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c004b20 con 0x55b21d338790 2024-09-10T22:53:11.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.654+0000 7f5c7cc14740 10 client.26299 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:11.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.654+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b21d3471d0 con 0x55b21d338790 2024-09-10T22:53:11.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c64025070 con 0x55b21d338790 2024-09-10T22:53:11.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5c6c019410 con 0x55b21d338790 2024-09-10T22:53:11.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 10 client.26299 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:11.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:11.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 20 client.26299 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:11.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d348840 0x55b21d34ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 4 client.26299 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:11.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.656+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b21d2a4db0 con 0x55b21d348840 2024-09-10T22:53:11.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.657+0000 7f5c73fff640 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d348840 0x55b21d34ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.658+0000 7f5c73fff640 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d348840 0x55b21d34ac90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f2ff1b12cc5a25 server_cookie=12ef71d9007948cd in_seq=0 out_seq=0 2024-09-10T22:53:11.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.658+0000 7f5c537fe640 10 client.26299 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.658+0000 7f5c537fe640 1 -- 172.21.15.96:0/3253333461 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b21d2a4db0 con 0x55b21d348840 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c537fe640 10 client.26299 handle_command_reply: tid=0 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 client.26299 shutdown 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 msgr2=0x55b21d335df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0119e0 tx=0x7f5c6c011ea0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 shutdown_connections 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d348840 0x55b21d34ac90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d335660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d335df0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/3253333461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d338210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 >> 172.21.15.96:0/3253333461 conn(0x55b21d329d10 msgr2=0x55b21d328f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.659+0000 7f5c537fe640 0 client.26299 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:11.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.661+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 shutdown_connections 2024-09-10T22:53:11.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.662+0000 7f5c7cc14740 1 -- 172.21.15.96:0/3253333461 wait complete. 2024-09-10T22:53:11.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.663+0000 7f5c7cc14740 20 client.26299 trim_cache size 0 max 16384 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 Processor -- start 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 -- start start 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d335260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d332870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b4a00 con 0x55b21d32bee0 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b6540 con 0x55b21d32c800 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.691+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b50d0 con 0x55b21d338790 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d335260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52598/0 (socket says 172.21.15.96:52598) 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 -- 172.21.15.96:0/1875323119 learned_addr learned my addr 172.21.15.96:0/1875323119 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c73fff640 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d335260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50040/0 (socket says 172.21.15.96:50040) 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c737fe640 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d332870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 -- 172.21.15.96:0/1875323119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 msgr2=0x55b21d332870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d332870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 -- 172.21.15.96:0/1875323119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 msgr2=0x55b21d335260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d335260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 -- 172.21.15.96:0/1875323119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21d32a790 con 0x55b21d338790 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006010 tx=0x7f5c6c009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb9b5bd33b0f0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c59ffb640 1 -- 172.21.15.96:0/1875323119 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c028070 con 0x55b21d338790 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c59ffb640 1 -- 172.21.15.96:0/1875323119 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c019c70 con 0x55b21d338790 2024-09-10T22:53:11.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.692+0000 7f5c59ffb640 1 -- 172.21.15.96:0/1875323119 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c024a20 con 0x55b21d338790 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 -- 172.21.15.96:0/1875323119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 msgr2=0x55b21d332db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006010 tx=0x7f5c6c009610 comp rx=0 tx=0).stop 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 -- 172.21.15.96:0/1875323119 shutdown_connections 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d332870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d332db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/1875323119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d335260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 -- 172.21.15.96:0/1875323119 >> 172.21.15.96:0/1875323119 conn(0x55b21d329d10 msgr2=0x55b21d334b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 -- 172.21.15.96:0/1875323119 shutdown_connections 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.693+0000 7f5c7cc14740 1 -- 172.21.15.96:0/1875323119 wait complete. 2024-09-10T22:53:11.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.694+0000 7f5c7cc14740 1 -- start start 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.694+0000 7f5c7cc14740 1 Processor -- start 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d340c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d3411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b5e70 con 0x55b21d32bee0 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b57a0 con 0x55b21d32c800 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741b29e0 con 0x55b21d338790 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c7cc14740 10 client.0 _ll_register_callbacks cb 0x7f5c6c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d3411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c78ddc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52612/0 (socket says 172.21.15.96:52612) 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.695+0000 7f5c78ddc640 1 -- 172.21.15.96:0/361205133 learned_addr learned my addr 172.21.15.96:0/361205133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 -- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 msgr2=0x55b21d3411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d3411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 -- 172.21.15.96:0/361205133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 msgr2=0x55b21d340c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d340c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 -- 172.21.15.96:0/361205133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21d329b40 con 0x55b21d338790 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c78ddc640 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00fc40 tx=0x7f5c6c00fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f5b863f862800bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c537fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.696+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c028040 con 0x55b21d338790 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6c019c70 con 0x55b21d338790 2024-09-10T22:53:11.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5c6c024a30 con 0x55b21d338790 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c00fe90 con 0x55b21d338790 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 10 client.26305 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b21d3b6340 con 0x55b21d338790 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c019c70 con 0x55b21d338790 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5c6c036500 con 0x55b21d338790 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 10 client.26305 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 20 client.26305 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:11.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d3b68a0 0x55b21d3b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 4 client.26305 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.697+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b21d2a0860 con 0x55b21d3b68a0 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.698+0000 7f5c73fff640 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d3b68a0 0x55b21d3b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.698+0000 7f5c537fe640 10 client.26305 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.698+0000 7f5c73fff640 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d3b68a0 0x55b21d3b8ce0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb327759efad61d1 server_cookie=6e3ee4a40d2fbcbc in_seq=0 out_seq=0 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.701+0000 7f5c537fe640 1 -- 172.21.15.96:0/361205133 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b21d2a0860 con 0x55b21d3b68a0 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.701+0000 7f5c537fe640 10 client.26305 handle_command_reply: tid=0 2024-09-10T22:53:11.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.701+0000 7f5c7cc14740 1 client.26305 shutdown 2024-09-10T22:53:11.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.701+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 msgr2=0x55b21d33b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.701+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00fc40 tx=0x7f5c6c00fc70 comp rx=0 tx=0).stop 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 shutdown_connections 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b21d3b68a0 0x55b21d3b8ce0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b21d32c800 0x55b21d3411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c537fe640 0 client.26305 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b21d338790 0x55b21d33b980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 --2- 172.21.15.96:0/361205133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b21d32bee0 0x55b21d340c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 >> 172.21.15.96:0/361205133 conn(0x55b21d329d10 msgr2=0x55b21d328f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.702+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 shutdown_connections 2024-09-10T22:53:11.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.703+0000 7f5c7cc14740 1 -- 172.21.15.96:0/361205133 wait complete. 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.703+0000 7f5c7cc14740 20 client.26305 trim_cache size 0 max 16384 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "12e98c91-25b6-43c1-b18c-73411aa73ff4", 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:53:11.705 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:53:11.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 msgr2=0x7f5c5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 0x7f5c5c0b77a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009c20 tx=0x7f5c64005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 msgr2=0x7f5c74101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c68032fd0 tx=0x7f5c68031970 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 shutdown_connections 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5c5c0b52e0 0x7f5c5c0b77a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c7410d100 0x7f5c74101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c741045c0 0x7f5c74101950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 --2- 172.21.15.96:0/2326916744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c74103ca0 0x7f5c74103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 >> 172.21.15.96:0/2326916744 conn(0x7f5c740fd700 msgr2=0x7f5c740fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 shutdown_connections 2024-09-10T22:53:11.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:11.704+0000 7f5c7a83f640 1 -- 172.21.15.96:0/2326916744 wait complete. 2024-09-10T22:53:13.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 730 op/s 2024-09-10T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:53:13.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:53:13.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:12 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 730 op/s 2024-09-10T22:53:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:53:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:53:13.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:53:13.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:12 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 730 op/s 2024-09-10T22:53:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:53:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:53:13.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:53:13.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:12 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:13.481 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 -- 172.21.15.96:0/1183333596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26181076a0 msgr2=0x7f2618107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 --2- 172.21.15.96:0/1183333596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26181076a0 0x7f2618107ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f260c009900 tx=0x7f260c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 -- 172.21.15.96:0/1183333596 shutdown_connections 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 --2- 172.21.15.96:0/1183333596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26181076a0 0x7f2618107ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 --2- 172.21.15.96:0/1183333596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618108020 0x7f261810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 --2- 172.21.15.96:0/1183333596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2618101c90 0x7f2618102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 -- 172.21.15.96:0/1183333596 >> 172.21.15.96:0/1183333596 conn(0x7f26180fd700 msgr2=0x7f26180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 -- 172.21.15.96:0/1183333596 shutdown_connections 2024-09-10T22:53:14.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.072+0000 7f261d316640 1 -- 172.21.15.96:0/1183333596 wait complete. 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 Processor -- start 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 -- start start 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 0x7f261806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26181076a0 0x7f261806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 0x7f2618194220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f261806e310 con 0x7f2618101c90 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f261806e190 con 0x7f2618108020 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.073+0000 7f261d316640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f261806e490 con 0x7f26181076a0 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 0x7f261806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 0x7f2618194220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 0x7f261806fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50072/0 (socket says 172.21.15.96:50072) 2024-09-10T22:53:14.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26167fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26181076a0 0x7f261806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f2616ffd640 1 -- 172.21.15.96:0/2607242184 learned_addr learned my addr 172.21.15.96:0/2607242184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 -- 172.21.15.96:0/2607242184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26181076a0 msgr2=0x7f261806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26181076a0 0x7f261806ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 -- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 msgr2=0x7f261806fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 0x7f261806fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.074+0000 7f26177fe640 1 -- 172.21.15.96:0/2607242184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26181949a0 con 0x7f2618108020 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.075+0000 7f26177fe640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 0x7f2618194220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26100042c0 tx=0x7f2610004780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.075+0000 7f25fffff640 1 -- 172.21.15.96:0/2607242184 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610004d20 con 0x7f2618108020 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.075+0000 7f25fffff640 1 -- 172.21.15.96:0/2607242184 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2610005020 con 0x7f2618108020 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.075+0000 7f25fffff640 1 -- 172.21.15.96:0/2607242184 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100296d0 con 0x7f2618108020 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.076+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2618194c90 con 0x7f2618108020 2024-09-10T22:53:14.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.076+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261819b2f0 con 0x7f2618108020 2024-09-10T22:53:14.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.078+0000 7f261f6eb740 1 Processor -- start 2024-09-10T22:53:14.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.079+0000 7f25fffff640 1 -- 172.21.15.96:0/2607242184 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f2610031030 con 0x7f2618108020 2024-09-10T22:53:14.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.080+0000 7f25fffff640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 0x7f26000b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.080+0000 7f25fffff640 1 -- 172.21.15.96:0/2607242184 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f26100f6c70 con 0x7f2618108020 2024-09-10T22:53:14.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 -- start start 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f26167fc640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 0x7f26000b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b3863fe50 con 0x563b386d5d30 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563b3862f280 con 0x563b386c9480 2024-09-10T22:53:14.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f261f6eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563b38633090 con 0x563b386c9da0 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f26177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f26177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50082/0 (socket says 172.21.15.96:50082) 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f26177fe640 1 -- 172.21.15.96:0/2378983597 learned_addr learned my addr 172.21.15.96:0/2378983597 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.081+0000 7f26167fc640 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26167fc640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 0x7f26000b9af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f260c005ab0 tx=0x7f260c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 -- 172.21.15.96:0/2378983597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 msgr2=0x563b386d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386d57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 -- 172.21.15.96:0/2378983597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 msgr2=0x563b386c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 -- 172.21.15.96:0/2378983597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b3838d650 con 0x563b386d5d30 2024-09-10T22:53:14.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f26177fe640 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2610004af0 tx=0x7f26100bce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addedbc74ce55db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f25fd7fa640 1 -- 172.21.15.96:0/2378983597 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100bdb30 con 0x563b386d5d30 2024-09-10T22:53:14.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.082+0000 7f25fd7fa640 1 -- 172.21.15.96:0/2378983597 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100bd430 con 0x563b386d5d30 2024-09-10T22:53:14.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.083+0000 7f25fd7fa640 1 -- 172.21.15.96:0/2378983597 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100f4b90 con 0x563b386d5d30 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.083+0000 7f261f6eb740 1 -- 172.21.15.96:0/2378983597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 msgr2=0x563b386d8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.083+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2610004af0 tx=0x7f26100bce40 comp rx=0 tx=0).stop 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 -- 172.21.15.96:0/2378983597 shutdown_connections 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d8120 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386d57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2378983597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 -- 172.21.15.96:0/2378983597 >> 172.21.15.96:0/2378983597 conn(0x563b386c72b0 msgr2=0x563b386c8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 -- 172.21.15.96:0/2378983597 shutdown_connections 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 -- 172.21.15.96:0/2378983597 wait complete. 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 -- start start 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.084+0000 7f261f6eb740 1 Processor -- start 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b38490700 con 0x563b386d5d30 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563b385ca110 con 0x563b386c9da0 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563b386d89b0 con 0x563b386c9480 2024-09-10T22:53:14.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f261f6eb740 10 client.0 _ll_register_callbacks cb 0x563b385e3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50090/0 (socket says 172.21.15.96:50090) 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 -- 172.21.15.96:0/2187527406 learned_addr learned my addr 172.21.15.96:0/2187527406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 -- 172.21.15.96:0/2187527406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 msgr2=0x563b386d5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386d5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 -- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 msgr2=0x563b386d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.085+0000 7f26177fe640 1 -- 172.21.15.96:0/2187527406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b385f7c40 con 0x563b386d5d30 2024-09-10T22:53:14.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386d5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f26177fe640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2610004ff0 tx=0x7f26100c6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ea95581e3e9204 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f25faff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100a8070 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f26100d81f0 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100bd430 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26100bdb30 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f261f6eb740 10 client.16692 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b386e1ce0 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.086+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100bd430 con 0x563b386d5d30 2024-09-10T22:53:14.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f26100bc690 con 0x563b386d5d30 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 10 client.16692 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 20 client.16692 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b386e32d0 0x563b386e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 4 client.16692 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b38641da0 con 0x563b386e32d0 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b386e32d0 0x563b386e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f25faff5640 10 client.16692 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:14.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.087+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b386e32d0 0x563b386e5770 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d20cbb953f0d8a server_cookie=ab428d650f9a4b0e in_seq=0 out_seq=0 2024-09-10T22:53:14.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f25faff5640 1 -- 172.21.15.96:0/2187527406 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b38641da0 con 0x563b386e32d0 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f25faff5640 10 client.16692 handle_command_reply: tid=0 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 client.16692 shutdown 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 msgr2=0x563b386d3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2610004ff0 tx=0x7f26100c6a90 comp rx=0 tx=0).stop 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 shutdown_connections 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b386e32d0 0x563b386e5770 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386d3250 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2187527406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386d5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 >> 172.21.15.96:0/2187527406 conn(0x563b386c72b0 msgr2=0x563b386cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.088+0000 7f25faff5640 0 client.16692 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.089+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 shutdown_connections 2024-09-10T22:53:14.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.089+0000 7f261f6eb740 1 -- 172.21.15.96:0/2187527406 wait complete. 2024-09-10T22:53:14.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.090+0000 7f261f6eb740 20 client.16692 trim_cache size 0 max 16384 2024-09-10T22:53:14.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.114+0000 7f261f6eb740 1 Processor -- start 2024-09-10T22:53:14.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.114+0000 7f261f6eb740 1 -- start start 2024-09-10T22:53:14.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.114+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.114+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.114+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f261f6eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a3890 con 0x563b386d5d30 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f261f6eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a53d0 con 0x563b386c9da0 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f261f6eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a3f60 con 0x563b386c9480 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f26177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52648/0 (socket says 172.21.15.96:52648) 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 -- 172.21.15.96:0/2478795921 learned_addr learned my addr 172.21.15.96:0/2478795921 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:14.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f26167fc640 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 -- 172.21.15.96:0/2478795921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 msgr2=0x563b386c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 -- 172.21.15.96:0/2478795921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 msgr2=0x563b386c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 -- 172.21.15.96:0/2478795921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b386c9bd0 con 0x563b386c9480 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f26167fc640 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d10 tx=0x7f26080098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c825732b04d0f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.115+0000 7f25fcff9640 1 -- 172.21.15.96:0/2478795921 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608010640 con 0x563b386c9480 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.116+0000 7f25fcff9640 1 -- 172.21.15.96:0/2478795921 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2608012e10 con 0x563b386c9480 2024-09-10T22:53:14.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.116+0000 7f25fcff9640 1 -- 172.21.15.96:0/2478795921 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608023ad0 con 0x563b386c9480 2024-09-10T22:53:14.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.116+0000 7f261f6eb740 1 -- 172.21.15.96:0/2478795921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 msgr2=0x563b386cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.116+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d10 tx=0x7f26080098f0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.117+0000 7f261f6eb740 1 -- 172.21.15.96:0/2478795921 shutdown_connections 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.117+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9da0 0x563b386c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.117+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.117+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2478795921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9480 0x563b386cd4f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.117+0000 7f261f6eb740 1 -- 172.21.15.96:0/2478795921 >> 172.21.15.96:0/2478795921 conn(0x563b386c72b0 msgr2=0x563b386c2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.118+0000 7f261f6eb740 1 -- 172.21.15.96:0/2478795921 shutdown_connections 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.118+0000 7f261f6eb740 1 -- 172.21.15.96:0/2478795921 wait complete. 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 -- start start 2024-09-10T22:53:14.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 Processor -- start 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a4d00 con 0x563b386d5d30 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.119+0000 7f261f6eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a4630 con 0x563b386c9480 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f261f6eb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26181a1870 con 0x563b386c9da0 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f261f6eb740 10 client.0 _ll_register_callbacks cb 0x7f26100e3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56284/0 (socket says 172.21.15.96:56284) 2024-09-10T22:53:14.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 -- 172.21.15.96:0/2317744851 learned_addr learned my addr 172.21.15.96:0/2317744851 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f26167fc640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f26177fe640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 -- 172.21.15.96:0/2317744851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 msgr2=0x563b386f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 -- 172.21.15.96:0/2317744851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 msgr2=0x563b386dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386dcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 -- 172.21.15.96:0/2317744851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b386dd330 con 0x563b386c9480 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f2616ffd640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2608023980 tx=0x7f26080239b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e293b6031ac3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.120+0000 7f25faff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608030020 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2608012e10 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2608005020 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2608023400 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f261f6eb740 10 client.26273 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.121+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b386de640 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2608012e10 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f260802c440 con 0x563b386c9480 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 10 client.26273 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 20 client.26273 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b3875b3b0 0x563b3875d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 4 client.26273 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b3863d850 con 0x563b3875b3b0 2024-09-10T22:53:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.122+0000 7f26167fc640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b3875b3b0 0x563b3875d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f25faff5640 10 client.26273 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f26167fc640 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b3875b3b0 0x563b3875d7d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168bd6499313e686 server_cookie=d9e31473af1eb585 in_seq=0 out_seq=0 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f25faff5640 1 -- 172.21.15.96:0/2317744851 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x563b3863d850 con 0x563b3875b3b0 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f25faff5640 10 client.26273 handle_command_reply: tid=0 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 client.26273 shutdown 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 msgr2=0x563b386f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2608023980 tx=0x7f26080239b0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 shutdown_connections 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563b3875b3b0 0x563b3875d7d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b386d5d30 0x563b386dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b386c9da0 0x563b386f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 --2- 172.21.15.96:0/2317744851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563b386c9480 0x563b386f5800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 >> 172.21.15.96:0/2317744851 conn(0x563b386c72b0 msgr2=0x563b386c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f25faff5640 0 client.26273 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 shutdown_connections 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.123+0000 7f261f6eb740 1 -- 172.21.15.96:0/2317744851 wait complete. 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261f6eb740 20 client.26273 trim_cache size 0 max 16384 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:53:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "12e98c91-25b6-43c1-b18c-73411aa73ff4", 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:53:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:53:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:53:14.135 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:53:14.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 msgr2=0x7f26000b9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 0x7f26000b9af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f260c005ab0 tx=0x7f260c005fd0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 msgr2=0x7f2618194220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 0x7f2618194220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26100042c0 tx=0x7f2610004780 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 shutdown_connections 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618108020 0x7f2618194220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f26000b7630 0x7f26000b9af0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26181076a0 0x7f261806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 --2- 172.21.15.96:0/2607242184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2618101c90 0x7f261806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.124+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 >> 172.21.15.96:0/2607242184 conn(0x7f26180fd700 msgr2=0x7f26180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.125+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 shutdown_connections 2024-09-10T22:53:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:14.125+0000 7f261d316640 1 -- 172.21.15.96:0/2607242184 wait complete. 2024-09-10T22:53:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:14 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:14.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:14 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:14.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:14 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:53:15.366 INFO:tasks.cephfs.filesystem:scrub status for tag:12e98c91-25b6-43c1-b18c-73411aa73ff4 - {'path': '/', 'tag': '12e98c91-25b6-43c1-b18c-73411aa73ff4', 'options': 'recursive,force'} 2024-09-10T22:53:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:15 smithi096 ceph-mon[30869]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 781 op/s 2024-09-10T22:53:15.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:15 smithi121 ceph-mon[35119]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 781 op/s 2024-09-10T22:53:15.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:15 smithi117 ceph-mon[35081]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 781 op/s 2024-09-10T22:53:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 340.0 s, 2745.4 tps, lat 36.121 ms stddev 42.962 2024-09-10T22:53:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:17 smithi096 ceph-mon[30869]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 873 op/s 2024-09-10T22:53:17.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:17 smithi121 ceph-mon[35119]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 873 op/s 2024-09-10T22:53:17.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:17 smithi117 ceph-mon[35081]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 873 op/s 2024-09-10T22:53:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.986+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1473232802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410ab10 msgr2=0x7fe17411a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.986+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1473232802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410ab10 0x7fe17411a090 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009900 tx=0x7fe16c02a410 comp rx=0 tx=0).stop 2024-09-10T22:53:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.987+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1473232802 shutdown_connections 2024-09-10T22:53:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.987+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1473232802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe17410a150 0x7fe17410a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.987+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1473232802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410ab10 0x7fe17411a090 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.987+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1473232802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174074290 0x7fe174074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.987+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1473232802 >> 172.21.15.96:0/1473232802 conn(0x7fe17406fa40 msgr2=0x7fe174071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.988+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1473232802 shutdown_connections 2024-09-10T22:53:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.991+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1473232802 wait complete. 2024-09-10T22:53:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.992+0000 7fe17b9ec640 1 Processor -- start 2024-09-10T22:53:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 -- start start 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe174074290 0x7fe174119590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17410ab10 0x7fe174117e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112180 con 0x7fe17410a150 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112000 con 0x7fe174074290 2024-09-10T22:53:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.995+0000 7fe17b9ec640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112300 con 0x7fe17410ab10 2024-09-10T22:53:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.997+0000 7fe178f87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.997+0000 7fe178f87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48342/0 (socket says 172.21.15.96:48342) 2024-09-10T22:53:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.997+0000 7fe178f87640 1 -- 172.21.15.96:0/1003837731 learned_addr learned my addr 172.21.15.96:0/1003837731 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.998+0000 7fe178f87640 1 -- 172.21.15.96:0/1003837731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17410ab10 msgr2=0x7fe174117e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.998+0000 7fe178f87640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17410ab10 0x7fe174117e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.998+0000 7fe178f87640 1 -- 172.21.15.96:0/1003837731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe174074290 msgr2=0x7fe174119590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.998+0000 7fe178f87640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe174074290 0x7fe174119590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:18.998+0000 7fe178f87640 1 -- 172.21.15.96:0/1003837731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1741185c0 con 0x7fe17410a150 2024-09-10T22:53:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.000+0000 7fe178f87640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9f0 tx=0x7fe16800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.003+0000 7fe1727fc640 1 -- 172.21.15.96:0/1003837731 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800cd80 con 0x7fe17410a150 2024-09-10T22:53:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.003+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1741188b0 con 0x7fe17410a150 2024-09-10T22:53:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.004+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741b87f0 con 0x7fe17410a150 2024-09-10T22:53:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.004+0000 7fe1727fc640 1 -- 172.21.15.96:0/1003837731 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1680108e0 con 0x7fe17410a150 2024-09-10T22:53:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.005+0000 7fe1727fc640 1 -- 172.21.15.96:0/1003837731 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168017920 con 0x7fe17410a150 2024-09-10T22:53:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 Processor -- start 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 -- start start 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7786860 0x5602a7788c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 0x5602a778b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a778e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5602a76f86f0 con 0x5602a7789190 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5602a76e7b20 con 0x5602a778bbd0 2024-09-10T22:53:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.013+0000 7fe17e3fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5602a76eb930 con 0x5602a7786860 2024-09-10T22:53:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.014+0000 7fe178f87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a778e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.014+0000 7fe178f87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a778e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57566/0 (socket says 172.21.15.96:57566) 2024-09-10T22:53:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.014+0000 7fe178f87640 1 -- 172.21.15.96:0/4037366367 learned_addr learned my addr 172.21.15.96:0/4037366367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.014+0000 7fe179f89640 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7786860 0x5602a7788c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.015+0000 7fe179788640 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 0x5602a778b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.016+0000 7fe179788640 1 -- 172.21.15.96:0/4037366367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7786860 msgr2=0x5602a7788c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.016+0000 7fe179788640 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7786860 0x5602a7788c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.016+0000 7fe179788640 1 -- 172.21.15.96:0/4037366367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 msgr2=0x5602a778e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.016+0000 7fe179788640 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a778e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.016+0000 7fe179788640 1 -- 172.21.15.96:0/4037366367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a76ad530 con 0x5602a7789190 2024-09-10T22:53:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.017+0000 7fe179788640 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 0x5602a778b580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe164007c30 tx=0x7fe16400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34bef552f6f7944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.018+0000 7fe1727fc640 1 -- 172.21.15.96:0/1003837731 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe168017b60 con 0x7fe17410a150 2024-09-10T22:53:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.018+0000 7fe15b7fe640 1 -- 172.21.15.96:0/4037366367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164014070 con 0x5602a7789190 2024-09-10T22:53:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.019+0000 7fe15b7fe640 1 -- 172.21.15.96:0/4037366367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16400fb00 con 0x5602a7789190 2024-09-10T22:53:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.019+0000 7fe1727fc640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 0x7fe15c0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.020+0000 7fe17e3fd740 1 -- 172.21.15.96:0/4037366367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 msgr2=0x5602a778b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.020+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 0x5602a778b580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe164007c30 tx=0x7fe16400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.020+0000 7fe1727fc640 1 -- 172.21.15.96:0/1003837731 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe1680eb9d0 con 0x7fe17410a150 2024-09-10T22:53:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.020+0000 7fe179f89640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 0x7fe15c0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.021+0000 7fe179f89640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 0x7fe15c0b7880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005ab0 tx=0x7fe16c02a910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.022+0000 7fe17e3fd740 1 -- 172.21.15.96:0/4037366367 shutdown_connections 2024-09-10T22:53:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.022+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a778e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.022+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7789190 0x5602a778b580 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.022+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/4037366367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7786860 0x5602a7788c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.022+0000 7fe17e3fd740 1 -- 172.21.15.96:0/4037366367 >> 172.21.15.96:0/4037366367 conn(0x5602a77819b0 msgr2=0x5602a7782740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.023+0000 7fe17e3fd740 1 -- 172.21.15.96:0/4037366367 shutdown_connections 2024-09-10T22:53:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.024+0000 7fe17e3fd740 1 -- 172.21.15.96:0/4037366367 wait complete. 2024-09-10T22:53:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.025+0000 7fe17e3fd740 1 -- start start 2024-09-10T22:53:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.025+0000 7fe17e3fd740 1 Processor -- start 2024-09-10T22:53:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.026+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 0x5602a778fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.026+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.026+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a7791110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe178f87640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe178f87640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52602/0 (socket says 172.21.15.96:52602) 2024-09-10T22:53:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe178f87640 1 -- 172.21.15.96:0/3149801483 learned_addr learned my addr 172.21.15.96:0/3149801483 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5602a761fab0 con 0x5602a7786860 2024-09-10T22:53:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5602a768d730 con 0x5602a778bbd0 2024-09-10T22:53:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5602a778ec00 con 0x5602a7789190 2024-09-10T22:53:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe179f89640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a7791110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.027+0000 7fe17e3fd740 10 client.0 _ll_register_callbacks cb 0x5602a76e16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.028+0000 7fe179788640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 0x5602a778fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.030+0000 7fe179788640 1 -- 172.21.15.96:0/3149801483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 msgr2=0x5602a7790540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.030+0000 7fe179788640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.030+0000 7fe179788640 1 -- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 msgr2=0x5602a7791110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.030+0000 7fe179788640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a7791110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.030+0000 7fe179788640 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a7449790 con 0x5602a7786860 2024-09-10T22:53:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.032+0000 7fe178f87640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.032+0000 7fe179f89640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a7791110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.032+0000 7fe179788640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 0x5602a778fea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe164007c30 tx=0x7fe16400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48ab371a9eec048 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.032+0000 7fe158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.038+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164014040 con 0x5602a7786860 2024-09-10T22:53:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.038+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe16400f860 con 0x5602a7786860 2024-09-10T22:53:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.039+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16400fb00 con 0x5602a7786860 2024-09-10T22:53:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.043+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe16400e3b0 con 0x5602a7786860 2024-09-10T22:53:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.043+0000 7fe17e3fd740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.043+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5602a74c2620 con 0x5602a7786860 2024-09-10T22:53:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.045+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16400fb00 con 0x5602a7786860 2024-09-10T22:53:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.045+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe1640214f0 con 0x5602a7786860 2024-09-10T22:53:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.047+0000 7fe17e3fd740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.048+0000 7fe17e3fd740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.048+0000 7fe17e3fd740 20 client.16707 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.048+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a779b480 0x5602a779d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.048+0000 7fe17e3fd740 4 client.16707 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.048+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602a76fa640 con 0x5602a779b480 2024-09-10T22:53:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.053+0000 7fe178f87640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a779b480 0x5602a779d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.056+0000 7fe178f87640 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a779b480 0x5602a779d870 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff063975e77188c server_cookie=aa5f9ec9ade7beb2 in_seq=0 out_seq=0 2024-09-10T22:53:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.058+0000 7fe158ff9640 10 client.16707 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.061+0000 7fe158ff9640 1 -- 172.21.15.96:0/3149801483 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602a76fa640 con 0x5602a779b480 2024-09-10T22:53:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.061+0000 7fe158ff9640 10 client.16707 handle_command_reply: tid=0 2024-09-10T22:53:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.062+0000 7fe17e3fd740 1 client.16707 shutdown 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 msgr2=0x5602a778fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 0x5602a778fea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe164007c30 tx=0x7fe16400ef20 comp rx=0 tx=0).stop 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 shutdown_connections 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a779b480 0x5602a779d870 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a778bbd0 0x5602a7791110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7789190 0x5602a7790540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/3149801483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a7786860 0x5602a778fea0 secure :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe164007c30 tx=0x7fe16400ef20 comp rx=0 tx=0).stop 2024-09-10T22:53:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.063+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 >> 172.21.15.96:0/3149801483 conn(0x5602a77819b0 msgr2=0x5602a7780bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.066+0000 7fe158ff9640 0 client.16707 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.066+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 shutdown_connections 2024-09-10T22:53:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.067+0000 7fe17e3fd740 1 -- 172.21.15.96:0/3149801483 wait complete. 2024-09-10T22:53:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.067+0000 7fe17e3fd740 20 client.16707 trim_cache size 0 max 16384 2024-09-10T22:53:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.105+0000 7fe17e3fd740 1 Processor -- start 2024-09-10T22:53:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.109+0000 7fe17e3fd740 1 -- start start 2024-09-10T22:53:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.109+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.109+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7782450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe179788640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe179788640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57592/0 (socket says 172.21.15.96:57592) 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe179788640 1 -- 172.21.15.96:0/2412421325 learned_addr learned my addr 172.21.15.96:0/2412421325 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7785bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000e490 con 0x5602a778bbd0 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000ffd0 con 0x5602a7789190 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe179f89640 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7785bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.110+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000eb60 con 0x5602a7782990 2024-09-10T22:53:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.113+0000 7fe178f87640 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7782450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.114+0000 7fe178f87640 1 -- 172.21.15.96:0/2412421325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 msgr2=0x5602a7785bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.114+0000 7fe178f87640 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7785bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.114+0000 7fe178f87640 1 -- 172.21.15.96:0/2412421325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 msgr2=0x5602a77919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.114+0000 7fe178f87640 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.114+0000 7fe178f87640 1 -- 172.21.15.96:0/2412421325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a7785850 con 0x5602a778bbd0 2024-09-10T22:53:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.119+0000 7fe178f87640 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7782450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe1680ebcb0 tx=0x7fe168021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b55ae171f011de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.120+0000 7fe15affd640 1 -- 172.21.15.96:0/2412421325 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168017e80 con 0x5602a778bbd0 2024-09-10T22:53:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.120+0000 7fe15affd640 1 -- 172.21.15.96:0/2412421325 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1680b1ca0 con 0x5602a778bbd0 2024-09-10T22:53:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.120+0000 7fe15affd640 1 -- 172.21.15.96:0/2412421325 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1680ae370 con 0x5602a778bbd0 2024-09-10T22:53:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.121+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 msgr2=0x5602a7782450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.121+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7782450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe1680ebcb0 tx=0x7fe168021000 comp rx=0 tx=0).stop 2024-09-10T22:53:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.124+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 shutdown_connections 2024-09-10T22:53:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.124+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7785bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.124+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7782450 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.124+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/2412421325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.124+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 >> 172.21.15.96:0/2412421325 conn(0x5602a77819b0 msgr2=0x5602a778fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.127+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 shutdown_connections 2024-09-10T22:53:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.127+0000 7fe17e3fd740 1 -- 172.21.15.96:0/2412421325 wait complete. 2024-09-10T22:53:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.128+0000 7fe17e3fd740 1 -- start start 2024-09-10T22:53:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.128+0000 7fe17e3fd740 1 Processor -- start 2024-09-10T22:53:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.128+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe17e3fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7795e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe179788640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe179788640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52624/0 (socket says 172.21.15.96:52624) 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe178f87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe178f87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57600/0 (socket says 172.21.15.96:57600) 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe179788640 1 -- 172.21.15.96:0/601248177 learned_addr learned my addr 172.21.15.96:0/601248177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.129+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000f900 con 0x5602a778bbd0 2024-09-10T22:53:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.130+0000 7fe179f89640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7795e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.130+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000f230 con 0x5602a7789190 2024-09-10T22:53:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.130+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe16000c470 con 0x5602a7782990 2024-09-10T22:53:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.130+0000 7fe17e3fd740 10 client.0 _ll_register_callbacks cb 0x7fe168005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.134+0000 7fe179788640 1 -- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 msgr2=0x5602a77952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.134+0000 7fe179788640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.134+0000 7fe179788640 1 -- 172.21.15.96:0/601248177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 msgr2=0x5602a7795e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.134+0000 7fe179788640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7795e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.134+0000 7fe179788640 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a77966a0 con 0x5602a7782990 2024-09-10T22:53:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.136+0000 7fe179788640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe164002230 tx=0x7fe164021e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6302de35192b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.140+0000 7fe158ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.140+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16403a070 con 0x5602a7782990 2024-09-10T22:53:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.141+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe164021620 con 0x5602a7782990 2024-09-10T22:53:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.141+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1680b1ca0 con 0x5602a7782990 2024-09-10T22:53:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.143+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe164043050 con 0x5602a7782990 2024-09-10T22:53:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.144+0000 7fe17e3fd740 10 client.26329 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.144+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5602a7797a70 con 0x5602a7782990 2024-09-10T22:53:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.145+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1680b1ca0 con 0x5602a7782990 2024-09-10T22:53:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.145+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe16402b450 con 0x5602a7782990 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 10 client.26329 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 20 client.26329 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a78086b0 0x5602a780aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 4 client.26329 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.146+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602a76f60f0 con 0x5602a78086b0 2024-09-10T22:53:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.147+0000 7fe178f87640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a78086b0 0x5602a780aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.148+0000 7fe178f87640 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a78086b0 0x5602a780aad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806832dcf027d87e server_cookie=752e0411eb6423a5 in_seq=0 out_seq=0 2024-09-10T22:53:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.148+0000 7fe158ff9640 10 client.26329 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe158ff9640 1 -- 172.21.15.96:0/601248177 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602a76f60f0 con 0x5602a78086b0 2024-09-10T22:53:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe158ff9640 10 client.26329 handle_command_reply: tid=0 2024-09-10T22:53:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 client.26329 shutdown 2024-09-10T22:53:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 msgr2=0x5602a7794c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe164002230 tx=0x7fe164021e70 comp rx=0 tx=0).stop 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 shutdown_connections 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5602a78086b0 0x5602a780aad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5602a7789190 0x5602a77952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe158ff9640 0 client.26329 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5602a778bbd0 0x5602a7795e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 --2- 172.21.15.96:0/601248177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5602a7782990 0x5602a7794c50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 >> 172.21.15.96:0/601248177 conn(0x5602a77819b0 msgr2=0x5602a778d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.226+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 shutdown_connections 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17e3fd740 1 -- 172.21.15.96:0/601248177 wait complete. 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17e3fd740 20 client.26329 trim_cache size 0 max 16384 2024-09-10T22:53:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:53:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:53:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:53:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:53:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 msgr2=0x7fe15c0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 0x7fe15c0b7880 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005ab0 tx=0x7fe16c02a910 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 msgr2=0x7fe174119ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9f0 tx=0x7fe16800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 shutdown_connections 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe15c0b53c0 0x7fe15c0b7880 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe17410ab10 0x7fe174117e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe17410a150 0x7fe174119ad0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.227+0000 7fe17b9ec640 1 --2- 172.21.15.96:0/1003837731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe174074290 0x7fe174119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.228+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 >> 172.21.15.96:0/1003837731 conn(0x7fe17406fa40 msgr2=0x7fe17410d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.228+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 shutdown_connections 2024-09-10T22:53:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:19.228+0000 7fe17b9ec640 1 -- 172.21.15.96:0/1003837731 wait complete. 2024-09-10T22:53:19.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:19 smithi096 ceph-mon[30869]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T22:53:19.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:19 smithi121 ceph-mon[35119]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T22:53:19.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:19 smithi117 ceph-mon[35081]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T22:53:20.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:20 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:53:20.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:20 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:53:20.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:20 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:53:20.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:20 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:53:20.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:20 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:53:20.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:20 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:53:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 345.0 s, 3016.0 tps, lat 33.569 ms stddev 56.532 2024-09-10T22:53:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:21 smithi096 ceph-mon[30869]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-10T22:53:21.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:21 smithi121 ceph-mon[35119]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-10T22:53:21.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:21 smithi117 ceph-mon[35081]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-10T22:53:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:23 smithi096 ceph-mon[30869]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 1.21k op/s 2024-09-10T22:53:23.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:23 smithi121 ceph-mon[35119]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 1.21k op/s 2024-09-10T22:53:23.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:23 smithi117 ceph-mon[35081]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 1.21k op/s 2024-09-10T22:53:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:25 smithi096 ceph-mon[30869]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 40 MiB/s wr, 1.54k op/s 2024-09-10T22:53:25.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:25 smithi121 ceph-mon[35119]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 40 MiB/s wr, 1.54k op/s 2024-09-10T22:53:25.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:25 smithi117 ceph-mon[35081]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 40 MiB/s wr, 1.54k op/s 2024-09-10T22:53:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 350.0 s, 3151.6 tps, lat 31.661 ms stddev 36.438 2024-09-10T22:53:27.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:27 smithi096 ceph-mon[30869]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 46 MiB/s wr, 1.71k op/s 2024-09-10T22:53:27.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:27 smithi121 ceph-mon[35119]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 46 MiB/s wr, 1.71k op/s 2024-09-10T22:53:27.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:27 smithi117 ceph-mon[35081]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 46 MiB/s wr, 1.71k op/s 2024-09-10T22:53:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:29 smithi096 ceph-mon[30869]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-09-10T22:53:29.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:29 smithi121 ceph-mon[35119]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-09-10T22:53:29.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:29 smithi117 ceph-mon[35081]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-09-10T22:53:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 355.0 s, 3104.4 tps, lat 32.031 ms stddev 34.378 2024-09-10T22:53:31.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:31 smithi096 ceph-mon[30869]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 MiB/s wr, 1.70k op/s 2024-09-10T22:53:31.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:31 smithi121 ceph-mon[35119]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 MiB/s wr, 1.70k op/s 2024-09-10T22:53:31.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:31 smithi117 ceph-mon[35081]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 46 MiB/s wr, 1.70k op/s 2024-09-10T22:53:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- 172.21.15.96:0/3275507141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678104490 msgr2=0x7f767810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 --2- 172.21.15.96:0/3275507141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678104490 0x7f767810ae10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7670009cd0 tx=0x7f767002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- 172.21.15.96:0/3275507141 shutdown_connections 2024-09-10T22:53:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 --2- 172.21.15.96:0/3275507141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678104490 0x7f767810ae10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 --2- 172.21.15.96:0/3275507141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678103ad0 0x7f7678103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 --2- 172.21.15.96:0/3275507141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- 172.21.15.96:0/3275507141 >> 172.21.15.96:0/3275507141 conn(0x7f76780fe240 msgr2=0x7f7678100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- 172.21.15.96:0/3275507141 shutdown_connections 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- 172.21.15.96:0/3275507141 wait complete. 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 Processor -- start 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.345+0000 7f767ea6b640 1 -- start start 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103ad0 0x7f76781993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 0x7f767819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767c807640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767c807640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52082/0 (socket says 172.21.15.96:52082) 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767c807640 1 -- 172.21.15.96:0/1154813580 learned_addr learned my addr 172.21.15.96:0/1154813580 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e640 con 0x7f7678103ad0 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e4c0 con 0x7f7678102980 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e7c0 con 0x7f7678104490 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f767d008640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 0x7f767819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.346+0000 7f7677fff640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103ad0 0x7f76781993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 -- 172.21.15.96:0/1154813580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 msgr2=0x7f7678198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 -- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103ad0 msgr2=0x7f76781993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103ad0 0x7f76781993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 -- 172.21.15.96:0/1154813580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767819a8d0 con 0x7f7678104490 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.347+0000 7f767d008640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 0x7f767819a010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f767002ae00 tx=0x7f7670003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.348+0000 7f7675ffb640 1 -- 172.21.15.96:0/1154813580 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670004040 con 0x7f7678104490 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.348+0000 7f7675ffb640 1 -- 172.21.15.96:0/1154813580 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7670004690 con 0x7f7678104490 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.348+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767819ab60 con 0x7f7678104490 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.348+0000 7f7675ffb640 1 -- 172.21.15.96:0/1154813580 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670030d40 con 0x7f7678104490 2024-09-10T22:53:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.348+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a1290 con 0x7f7678104490 2024-09-10T22:53:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.351+0000 7f7681830740 1 Processor -- start 2024-09-10T22:53:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.351+0000 7f7681830740 1 -- start start 2024-09-10T22:53:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.351+0000 7f7675ffb640 1 -- 172.21.15.96:0/1154813580 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7670030070 con 0x7f7678104490 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b284fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b28550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b27bf6f0 con 0x5589b284d880 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b27aeb20 con 0x5589b28501b0 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f7681830740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b27b2930 con 0x5589b2852bf0 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f767c807640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f767c807640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52098/0 (socket says 172.21.15.96:52098) 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f767c807640 1 -- 172.21.15.96:0/80389362 learned_addr learned my addr 172.21.15.96:0/80389362 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f767c807640 1 -- 172.21.15.96:0/80389362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 msgr2=0x5589b28550b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:53:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.352+0000 7f767c807640 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b28550b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f767c807640 1 -- 172.21.15.96:0/80389362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 msgr2=0x5589b284fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f767c807640 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b284fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f767c807640 1 -- 172.21.15.96:0/80389362 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b2774530 con 0x5589b28501b0 2024-09-10T22:53:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f767c807640 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f766c009e10 tx=0x7f766c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826dbf5a5ca6af58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7662ffd640 1 -- 172.21.15.96:0/80389362 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c01e070 con 0x5589b28501b0 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7675ffb640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 0x7f76680b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7662ffd640 1 -- 172.21.15.96:0/80389362 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c019ae0 con 0x5589b28501b0 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7662ffd640 1 -- 172.21.15.96:0/80389362 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00faa0 con 0x5589b28501b0 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7677fff640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 0x7f76680b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.353+0000 7f7675ffb640 1 -- 172.21.15.96:0/1154813580 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f767002e030 con 0x7f7678104490 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 -- 172.21.15.96:0/80389362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 msgr2=0x5589b28525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f766c009e10 tx=0x7f766c011810 comp rx=0 tx=0).stop 2024-09-10T22:53:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7677fff640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 0x7f76680b78a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7664009710 tx=0x7f7664009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 -- 172.21.15.96:0/80389362 shutdown_connections 2024-09-10T22:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b28525a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b28550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 --2- 172.21.15.96:0/80389362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b284fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.354+0000 7f7681830740 1 -- 172.21.15.96:0/80389362 >> 172.21.15.96:0/80389362 conn(0x5589b28489d0 msgr2=0x5589b2849760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.355+0000 7f7681830740 1 -- 172.21.15.96:0/80389362 shutdown_connections 2024-09-10T22:53:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.355+0000 7f7681830740 1 -- 172.21.15.96:0/80389362 wait complete. 2024-09-10T22:53:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.355+0000 7f7681830740 1 -- start start 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 Processor -- start 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2856f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b28575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b2858180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b26e6ab0 con 0x5589b284d880 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b2754730 con 0x5589b2852bf0 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b2855bd0 con 0x5589b28501b0 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7681830740 10 client.0 _ll_register_callbacks cb 0x5589b27a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7677fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b28575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767c807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2856f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7677fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b28575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:42194/0 (socket says 172.21.15.96:42194) 2024-09-10T22:53:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f7677fff640 1 -- 172.21.15.96:0/596176473 learned_addr learned my addr 172.21.15.96:0/596176473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767d008640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b2858180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767c807640 1 -- 172.21.15.96:0/596176473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 msgr2=0x5589b28575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767c807640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b28575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767c807640 1 -- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 msgr2=0x5589b2858180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.356+0000 7f767c807640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b2858180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f767c807640 1 -- 172.21.15.96:0/596176473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b2510790 con 0x5589b284d880 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f767d008640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b2858180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f767c807640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2856f10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f766c01b000 tx=0x7f766c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed731df50b9ecd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7657fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c01e050 con 0x5589b284d880 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766c019ae0 con 0x5589b284d880 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f766c019840 con 0x5589b284d880 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f766c004490 con 0x5589b284d880 2024-09-10T22:53:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7681830740 10 client.16722 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b2589620 con 0x5589b284d880 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.357+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c019ae0 con 0x5589b284d880 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f766c0046c0 con 0x5589b284d880 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 10 client.16722 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 20 client.16722 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b286afa0 0x5589b286d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 4 client.16722 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.358+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b27c1640 con 0x5589b286afa0 2024-09-10T22:53:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.359+0000 7f7677fff640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b286afa0 0x5589b286d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.360+0000 7f7677fff640 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b286afa0 0x5589b286d390 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988c4e13e48535cd server_cookie=9ea32fabdc9339fc in_seq=0 out_seq=0 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.360+0000 7f7657fff640 10 client.16722 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.360+0000 7f7657fff640 1 -- 172.21.15.96:0/596176473 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589b27c1640 con 0x5589b286afa0 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.360+0000 7f7657fff640 10 client.16722 handle_command_reply: tid=0 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 client.16722 shutdown 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 msgr2=0x5589b2856f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2856f10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f766c01b000 tx=0x7f766c004270 comp rx=0 tx=0).stop 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 shutdown_connections 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b286afa0 0x5589b286d390 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b2858180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b28575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 --2- 172.21.15.96:0/596176473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2856f10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7657fff640 0 client.16722 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 >> 172.21.15.96:0/596176473 conn(0x5589b28489d0 msgr2=0x5589b2847bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 shutdown_connections 2024-09-10T22:53:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 1 -- 172.21.15.96:0/596176473 wait complete. 2024-09-10T22:53:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.361+0000 7f7681830740 20 client.16722 trim_cache size 0 max 16384 2024-09-10T22:53:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 Processor -- start 2024-09-10T22:53:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 -- start start 2024-09-10T22:53:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b284a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b284ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a7ba0 con 0x5589b284d880 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a96e0 con 0x5589b2852bf0 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.381+0000 7f7681830740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a8270 con 0x5589b28501b0 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42040/0 (socket says 172.21.15.96:42040) 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f7677fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b284a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 -- 172.21.15.96:0/2503570006 learned_addr learned my addr 172.21.15.96:0/2503570006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767d008640 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b284ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 -- 172.21.15.96:0/2503570006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 msgr2=0x5589b284a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b284a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 -- 172.21.15.96:0/2503570006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 msgr2=0x5589b284ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b284ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 -- 172.21.15.96:0/2503570006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b2849230 con 0x5589b284d880 2024-09-10T22:53:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f767c807640 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f766c01afd0 tx=0x7f766c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0337b07e6e2b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f76627fc640 1 -- 172.21.15.96:0/2503570006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c01e040 con 0x5589b284d880 2024-09-10T22:53:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f76627fc640 1 -- 172.21.15.96:0/2503570006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c01f040 con 0x5589b284d880 2024-09-10T22:53:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.382+0000 7f76627fc640 1 -- 172.21.15.96:0/2503570006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c019b70 con 0x5589b284d880 2024-09-10T22:53:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 -- 172.21.15.96:0/2503570006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 msgr2=0x5589b2847ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f766c01afd0 tx=0x7f766c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 -- 172.21.15.96:0/2503570006 shutdown_connections 2024-09-10T22:53:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b2852bf0 0x5589b284ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b28501b0 0x5589b284a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 --2- 172.21.15.96:0/2503570006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2847ae0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.384+0000 7f7681830740 1 -- 172.21.15.96:0/2503570006 >> 172.21.15.96:0/2503570006 conn(0x5589b28489d0 msgr2=0x5589b28473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 -- 172.21.15.96:0/2503570006 shutdown_connections 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 -- 172.21.15.96:0/2503570006 wait complete. 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 -- start start 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 Processor -- start 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2865940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.385+0000 7f7681830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b2865fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b2866bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f767c807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2865940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a9010 con 0x5589b284d880 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f767c807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2865940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42044/0 (socket says 172.21.15.96:42044) 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f767c807640 1 -- 172.21.15.96:0/3146918350 learned_addr learned my addr 172.21.15.96:0/3146918350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a8940 con 0x5589b28501b0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a5ba0 con 0x5589b2852bf0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b2865fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 10 client.0 _ll_register_callbacks cb 0x7f766c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f767d008640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b2866bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 -- 172.21.15.96:0/3146918350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 msgr2=0x5589b2866bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b2866bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 -- 172.21.15.96:0/3146918350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 msgr2=0x5589b2865940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2865940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b28673e0 con 0x5589b28501b0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7677fff640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b2865fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5589b286d930 tx=0x7f7664005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b1f5d02e3bd885 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7657fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640293f0 con 0x5589b28501b0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f766402b040 con 0x5589b28501b0 2024-09-10T22:53:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.386+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766c01f040 con 0x5589b28501b0 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7664029590 con 0x5589b28501b0 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 10 client.26303 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b28687e0 con 0x5589b28501b0 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c01f040 con 0x5589b28501b0 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7664038860 con 0x5589b28501b0 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 10 client.26303 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 20 client.26303 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b28d8170 0x5589b28da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 4 client.26303 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.388+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b286d780 con 0x5589b28d8170 2024-09-10T22:53:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.389+0000 7f767c807640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b28d8170 0x5589b28da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.389+0000 7f7657fff640 10 client.26303 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.389+0000 7f767c807640 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b28d8170 0x5589b28da590 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ab8b7357f58170 server_cookie=c9107ad50da1df4d in_seq=0 out_seq=0 2024-09-10T22:53:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7657fff640 1 -- 172.21.15.96:0/3146918350 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b286d780 con 0x5589b28d8170 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7657fff640 10 client.26303 handle_command_reply: tid=0 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 client.26303 shutdown 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 msgr2=0x5589b2865fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b2865fe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5589b286d930 tx=0x7f7664005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 shutdown_connections 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b28d8170 0x5589b28da590 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b28501b0 0x5589b2865fe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7657fff640 0 client.26303 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b2852bf0 0x5589b2866bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 --2- 172.21.15.96:0/3146918350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b284d880 0x5589b2865940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 >> 172.21.15.96:0/3146918350 conn(0x5589b28489d0 msgr2=0x5589b284a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 shutdown_connections 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.488+0000 7f7681830740 1 -- 172.21.15.96:0/3146918350 wait complete. 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.489+0000 7f7681830740 20 client.26303 trim_cache size 0 max 16384 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:53:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.490+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 msgr2=0x7f76680b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.490+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 0x7f76680b78a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7664009710 tx=0x7f7664009860 comp rx=0 tx=0).stop 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.490+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 msgr2=0x7f767819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.490+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 0x7f767819a010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f767002ae00 tx=0x7f7670003ba0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 shutdown_connections 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f76680b53e0 0x7f76680b78a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678104490 0x7f767819a010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103ad0 0x7f76781993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 --2- 172.21.15.96:0/1154813580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7678102980 0x7f7678198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 >> 172.21.15.96:0/1154813580 conn(0x7f76780fe240 msgr2=0x7f7678106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 shutdown_connections 2024-09-10T22:53:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:32.491+0000 7f767ea6b640 1 -- 172.21.15.96:0/1154813580 wait complete. 2024-09-10T22:53:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:33 smithi096 ceph-mon[30869]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 41 MiB/s wr, 1.48k op/s 2024-09-10T22:53:33.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:33 smithi121 ceph-mon[35119]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 41 MiB/s wr, 1.48k op/s 2024-09-10T22:53:33.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:33 smithi117 ceph-mon[35081]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 41 MiB/s wr, 1.48k op/s 2024-09-10T22:53:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:34 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:34 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:34 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:34.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:34 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:53:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:34 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:34 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:34 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:34.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:34 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:53:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:34 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:53:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:34 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:53:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:34 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:53:34.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:34 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:53:35.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:35 smithi121 ceph-mon[35119]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-09-10T22:53:35.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:35 smithi117 ceph-mon[35081]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-09-10T22:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:35 smithi096 ceph-mon[30869]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 30 MiB/s wr, 1.21k op/s 2024-09-10T22:53:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 360.0 s, 3089.2 tps, lat 32.451 ms stddev 36.980 2024-09-10T22:53:37.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:37 smithi121 ceph-mon[35119]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 976 op/s 2024-09-10T22:53:37.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:37 smithi117 ceph-mon[35081]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 976 op/s 2024-09-10T22:53:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:37 smithi096 ceph-mon[30869]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 976 op/s 2024-09-10T22:53:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:39 smithi121 ceph-mon[35119]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-09-10T22:53:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:39 smithi117 ceph-mon[35081]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-09-10T22:53:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:39 smithi096 ceph-mon[30869]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.9 MiB/s wr, 641 op/s 2024-09-10T22:53:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 365.0 s, 3311.4 tps, lat 30.219 ms stddev 35.865 2024-09-10T22:53:41.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:41 smithi117 ceph-mon[35081]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 836 op/s 2024-09-10T22:53:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:41 smithi096 ceph-mon[30869]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 836 op/s 2024-09-10T22:53:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:41 smithi121 ceph-mon[35119]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 836 op/s 2024-09-10T22:53:43.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:43 smithi117 ceph-mon[35081]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T22:53:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:43 smithi096 ceph-mon[30869]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T22:53:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:43 smithi121 ceph-mon[35119]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T22:53:45.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:53:45.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:45 smithi117 ceph-mon[35081]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T22:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.628+0000 7f885e8ed640 1 -- 172.21.15.96:0/32949110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 msgr2=0x7f8858074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.628+0000 7f885e8ed640 1 --2- 172.21.15.96:0/32949110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 0x7f8858074690 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8850009860 tx=0x7f885002a360 comp rx=0 tx=0).stop 2024-09-10T22:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.630+0000 7f885e8ed640 1 -- 172.21.15.96:0/32949110 shutdown_connections 2024-09-10T22:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.630+0000 7f885e8ed640 1 --2- 172.21.15.96:0/32949110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810ab10 0x7f885811a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.630+0000 7f885e8ed640 1 --2- 172.21.15.96:0/32949110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810a150 0x7f885810a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.630+0000 7f885e8ed640 1 --2- 172.21.15.96:0/32949110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 0x7f8858074690 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.630+0000 7f885e8ed640 1 -- 172.21.15.96:0/32949110 >> 172.21.15.96:0/32949110 conn(0x7f885806fa40 msgr2=0x7f8858071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.638+0000 7f885e8ed640 1 -- 172.21.15.96:0/32949110 shutdown_connections 2024-09-10T22:53:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.638+0000 7f885e8ed640 1 -- 172.21.15.96:0/32949110 wait complete. 2024-09-10T22:53:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 Processor -- start 2024-09-10T22:53:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 -- start start 2024-09-10T22:53:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 0x7f88581169a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810ab10 0x7f8858117c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f885811d4b0 con 0x7f8858074290 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f885811d330 con 0x7f885810a150 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.639+0000 7f885e8ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f885811d630 con 0x7f885810ab10 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.640+0000 7f88577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.640+0000 7f88577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50212/0 (socket says 172.21.15.96:50212) 2024-09-10T22:53:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.640+0000 7f88577fe640 1 -- 172.21.15.96:0/2264994832 learned_addr learned my addr 172.21.15.96:0/2264994832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f885ce8a640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810ab10 0x7f8858117c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f88577fe640 1 -- 172.21.15.96:0/2264994832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810ab10 msgr2=0x7f8858117c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f88577fe640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810ab10 0x7f8858117c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f88577fe640 1 -- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 msgr2=0x7f88581169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f88577fe640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 0x7f88581169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.642+0000 7f88577fe640 1 -- 172.21.15.96:0/2264994832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88581184e0 con 0x7f885810a150 2024-09-10T22:53:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.645+0000 7f88577fe640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884800c420 tx=0x7f884800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.650+0000 7f88557fa640 1 -- 172.21.15.96:0/2264994832 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848004b20 con 0x7f885810a150 2024-09-10T22:53:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.650+0000 7f88557fa640 1 -- 172.21.15.96:0/2264994832 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8848003e20 con 0x7f885810a150 2024-09-10T22:53:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.650+0000 7f88557fa640 1 -- 172.21.15.96:0/2264994832 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884800b630 con 0x7f885810a150 2024-09-10T22:53:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.650+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581187d0 con 0x7f885810a150 2024-09-10T22:53:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.650+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8858118e70 con 0x7f885810a150 2024-09-10T22:53:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.654+0000 7f88616b8740 1 Processor -- start 2024-09-10T22:53:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.655+0000 7f88616b8740 1 -- start start 2024-09-10T22:53:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.657+0000 7f88616b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.657+0000 7f88616b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f885ce8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f885ce8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124bcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50228/0 (socket says 172.21.15.96:50228) 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f885ce8a640 1 -- 172.21.15.96:0/1361818277 learned_addr learned my addr 172.21.15.96:0/1361818277 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f8857fff640 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f88616b8740 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd1242c6f0 con 0x55fd124bfbd0 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd1241bb20 con 0x55fd124ba860 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.658+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd1241f930 con 0x55fd124bd190 2024-09-10T22:53:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 -- 172.21.15.96:0/1361818277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 msgr2=0x55fd124bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 -- 172.21.15.96:0/1361818277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 msgr2=0x55fd124bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124bcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.659+0000 7f88577fe640 1 -- 172.21.15.96:0/1361818277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd123e1530 con 0x55fd124bfbd0 2024-09-10T22:53:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.660+0000 7f88577fe640 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c2090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f88480e3590 tx=0x7f88480e9c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156f722639a18da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.661+0000 7f88557fa640 1 -- 172.21.15.96:0/2264994832 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f884800b040 con 0x7f885810a150 2024-09-10T22:53:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.662+0000 7f88557fa640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 0x7f88440b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.662+0000 7f88557fa640 1 -- 172.21.15.96:0/2264994832 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f88480e5af0 con 0x7f885810a150 2024-09-10T22:53:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.663+0000 7f885ce8a640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 0x7f88440b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.663+0000 7f884e7fc640 1 -- 172.21.15.96:0/1361818277 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480eeb70 con 0x55fd124bfbd0 2024-09-10T22:53:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.663+0000 7f884e7fc640 1 -- 172.21.15.96:0/1361818277 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88480eed10 con 0x55fd124bfbd0 2024-09-10T22:53:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.663+0000 7f884e7fc640 1 -- 172.21.15.96:0/1361818277 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480f2630 con 0x55fd124bfbd0 2024-09-10T22:53:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.665+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 msgr2=0x55fd124c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.665+0000 7f88616b8740 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c2090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f88480e3590 tx=0x7f88480e9c30 comp rx=0 tx=0).stop 2024-09-10T22:53:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.666+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 shutdown_connections 2024-09-10T22:53:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.666+0000 7f88616b8740 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c2090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.667+0000 7f88616b8740 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.667+0000 7f88616b8740 1 --2- 172.21.15.96:0/1361818277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.667+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 >> 172.21.15.96:0/1361818277 conn(0x55fd124b59b0 msgr2=0x55fd124b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.667+0000 7f885ce8a640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 0x7f88440b79b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f884000aa40 tx=0x7f8840008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.669+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 shutdown_connections 2024-09-10T22:53:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.670+0000 7f88616b8740 1 -- 172.21.15.96:0/1361818277 wait complete. 2024-09-10T22:53:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.670+0000 7f88616b8740 1 -- start start 2024-09-10T22:53:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.670+0000 7f88616b8740 1 Processor -- start 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd12353ab0 con 0x55fd124bfbd0 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd123c1730 con 0x55fd124ba860 2024-09-10T22:53:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd124c2bb0 con 0x55fd124bd190 2024-09-10T22:53:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.671+0000 7f88616b8740 10 client.0 _ll_register_callbacks cb 0x55fd124156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50240/0 (socket says 172.21.15.96:50240) 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 -- 172.21.15.96:0/651077683 learned_addr learned my addr 172.21.15.96:0/651077683 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 -- 172.21.15.96:0/651077683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 msgr2=0x55fd124c45c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124c45c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 -- 172.21.15.96:0/651077683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 msgr2=0x55fd124c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 -- 172.21.15.96:0/651077683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1217d790 con 0x55fd124ba860 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.672+0000 7f8857fff640 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8850009780 tx=0x7f8850005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe44e62441b30b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.673+0000 7f88377fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.673+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850041030 con 0x55fd124ba860 2024-09-10T22:53:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.673+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f88500055f0 con 0x55fd124ba860 2024-09-10T22:53:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.675+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88480eed10 con 0x55fd124ba860 2024-09-10T22:53:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.675+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8850037070 con 0x55fd124ba860 2024-09-10T22:53:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.676+0000 7f88616b8740 10 client.26312 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.676+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd121f6620 con 0x55fd124ba860 2024-09-10T22:53:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.677+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88480eed10 con 0x55fd124ba860 2024-09-10T22:53:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.677+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8850003900 con 0x55fd124ba860 2024-09-10T22:53:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 10 client.26312 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 20 client.26312 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd124cf8a0 0x55fd124d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 4 client.26312 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.680+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd1242e640 con 0x55fd124cf8a0 2024-09-10T22:53:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.681+0000 7f88577fe640 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd124cf8a0 0x55fd124d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.686+0000 7f88577fe640 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd124cf8a0 0x55fd124d1d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51246183126b92b2 server_cookie=eb457bfa108a87bb in_seq=0 out_seq=0 2024-09-10T22:53:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.686+0000 7f88377fe640 10 client.26312 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.687+0000 7f88377fe640 1 -- 172.21.15.96:0/651077683 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd1242e640 con 0x55fd124cf8a0 2024-09-10T22:53:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.687+0000 7f88377fe640 10 client.26312 handle_command_reply: tid=0 2024-09-10T22:53:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.687+0000 7f88616b8740 1 client.26312 shutdown 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.688+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 msgr2=0x55fd124c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.688+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8850009780 tx=0x7f8850005270 comp rx=0 tx=0).stop 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 shutdown_connections 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd124cf8a0 0x55fd124d1d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bd190 0x55fd124c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 --2- 172.21.15.96:0/651077683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124ba860 0x55fd124c3f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 >> 172.21.15.96:0/651077683 conn(0x55fd124b59b0 msgr2=0x55fd124b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88377fe640 0 client.26312 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 shutdown_connections 2024-09-10T22:53:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 1 -- 172.21.15.96:0/651077683 wait complete. 2024-09-10T22:53:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.689+0000 7f88616b8740 20 client.26312 trim_cache size 0 max 16384 2024-09-10T22:53:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 Processor -- start 2024-09-10T22:53:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 -- start start 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f883800e450 con 0x55fd124bfbd0 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f883800ff90 con 0x55fd124bd190 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88616b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f883800eb20 con 0x55fd124ba860 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36634/0 (socket says 172.21.15.96:36634) 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 -- 172.21.15.96:0/1844121943 learned_addr learned my addr 172.21.15.96:0/1844121943 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f885ce8a640 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f88577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124ba230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50246/0 (socket says 172.21.15.96:50246) 2024-09-10T22:53:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 -- 172.21.15.96:0/1844121943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 msgr2=0x55fd124ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 -- 172.21.15.96:0/1844121943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 msgr2=0x55fd124b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f8857fff640 1 -- 172.21.15.96:0/1844121943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd124c5500 con 0x55fd124ba860 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.716+0000 7f885ce8a640 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.717+0000 7f8857fff640 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8850004500 tx=0x7f88500304e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867c44fa773a7567 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.717+0000 7f884dffb640 1 -- 172.21.15.96:0/1844121943 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850041040 con 0x55fd124ba860 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.717+0000 7f884dffb640 1 -- 172.21.15.96:0/1844121943 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850054070 con 0x55fd124ba860 2024-09-10T22:53:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.717+0000 7f884dffb640 1 -- 172.21.15.96:0/1844121943 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850030bc0 con 0x55fd124ba860 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- 172.21.15.96:0/1844121943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 msgr2=0x55fd124b9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8850004500 tx=0x7f88500304e0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- 172.21.15.96:0/1844121943 shutdown_connections 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124bfbd0 0x55fd124b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 --2- 172.21.15.96:0/1844121943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124ba860 0x55fd124b9cf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- 172.21.15.96:0/1844121943 >> 172.21.15.96:0/1844121943 conn(0x55fd124b59b0 msgr2=0x55fd124c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- 172.21.15.96:0/1844121943 shutdown_connections 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- 172.21.15.96:0/1844121943 wait complete. 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.718+0000 7f88616b8740 1 -- start start 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f88616b8740 1 Processor -- start 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f88616b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124ba860 0x55fd124c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f88616b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f88616b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f8857fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124ba860 0x55fd124c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.719+0000 7f88616b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f883800f8c0 con 0x55fd124ba860 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f88616b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f883800f1f0 con 0x55fd124bd190 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f88616b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f883800c430 con 0x55fd124bfbd0 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f88616b8740 10 client.0 _ll_register_callbacks cb 0x7f8848009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f885ce8a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f885ce8a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36640/0 (socket says 172.21.15.96:36640) 2024-09-10T22:53:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f885ce8a640 1 -- 172.21.15.96:0/3523225533 learned_addr learned my addr 172.21.15.96:0/3523225533 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.720+0000 7f88577fe640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 -- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 msgr2=0x55fd124c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 -- 172.21.15.96:0/3523225533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124ba860 msgr2=0x55fd124c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124ba860 0x55fd124c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 -- 172.21.15.96:0/3523225533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd124bf8f0 con 0x55fd124bfbd0 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f885ce8a640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8840015780 tx=0x7f8840015b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19324077ff84662f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f88377fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840025030 con 0x55fd124bfbd0 2024-09-10T22:53:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8840013920 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.721+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8850054070 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8840013b60 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 10 client.26356 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd124cb8a0 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850054070 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f884001e050 con 0x55fd124bfbd0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 10 client.26356 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 20 client.26356 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd1253c870 0x55fd1253ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 4 client.26356 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd1242a0f0 con 0x55fd1253c870 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.722+0000 7f8857fff640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd1253c870 0x55fd1253ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:45.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.723+0000 7f88377fe640 10 client.26356 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:45.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.723+0000 7f8857fff640 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd1253c870 0x55fd1253ec90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673c9535d7df341a server_cookie=7c2c68019e10de32 in_seq=0 out_seq=0 2024-09-10T22:53:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:45 smithi096 ceph-mon[30869]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T22:53:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.749+0000 7f88377fe640 1 -- 172.21.15.96:0/3523225533 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd1242a0f0 con 0x55fd1253c870 2024-09-10T22:53:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.749+0000 7f88377fe640 10 client.26356 handle_command_reply: tid=0 2024-09-10T22:53:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.749+0000 7f88616b8740 1 client.26356 shutdown 2024-09-10T22:53:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 msgr2=0x55fd124c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8840015780 tx=0x7f8840015b80 comp rx=0 tx=0).stop 2024-09-10T22:53:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 shutdown_connections 2024-09-10T22:53:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55fd1253c870 0x55fd1253ec90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd124bd190 0x55fd124c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd124bfbd0 0x55fd124c9d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88377fe640 0 client.26356 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 --2- 172.21.15.96:0/3523225533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd124ba860 0x55fd124c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.750+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 >> 172.21.15.96:0/3523225533 conn(0x55fd124b59b0 msgr2=0x55fd124b98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 shutdown_connections 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f88616b8740 1 -- 172.21.15.96:0/3523225533 wait complete. 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f88616b8740 20 client.26356 trim_cache size 0 max 16384 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:53:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:53:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 msgr2=0x7f88440b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 0x7f88440b79b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f884000aa40 tx=0x7f8840008040 comp rx=0 tx=0).stop 2024-09-10T22:53:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 msgr2=0x7f8858116ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.751+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884800c420 tx=0x7f884800c8e0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 shutdown_connections 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f885810a150 0x7f8858116ff0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f88440b54f0 0x7f88440b79b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f885810ab10 0x7f8858117c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 --2- 172.21.15.96:0/2264994832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8858074290 0x7f88581169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 >> 172.21.15.96:0/2264994832 conn(0x7f885806fa40 msgr2=0x7f8858071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 shutdown_connections 2024-09-10T22:53:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:45.752+0000 7f885e8ed640 1 -- 172.21.15.96:0/2264994832 wait complete. 2024-09-10T22:53:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:45 smithi121 ceph-mon[35119]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T22:53:45.771 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 370.0 s, 3526.6 tps, lat 28.462 ms stddev 34.108 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 -- 172.21.15.96:0/2796504012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7190101c90 msgr2=0x7f7190102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 --2- 172.21.15.96:0/2796504012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7190101c90 0x7f7190102070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7180009900 tx=0x7f718002a410 comp rx=0 tx=0).stop 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 -- 172.21.15.96:0/2796504012 shutdown_connections 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 --2- 172.21.15.96:0/2796504012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190108020 0x7f719010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 --2- 172.21.15.96:0/2796504012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71901076a0 0x7f7190107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 --2- 172.21.15.96:0/2796504012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7190101c90 0x7f7190102070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 -- 172.21.15.96:0/2796504012 >> 172.21.15.96:0/2796504012 conn(0x7f71900fd700 msgr2=0x7f71900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 -- 172.21.15.96:0/2796504012 shutdown_connections 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.341+0000 7f719657d640 1 -- 172.21.15.96:0/2796504012 wait complete. 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 Processor -- start 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 -- start start 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71901076a0 0x7f71901978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7190108020 0x7f719019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f718ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7190111d30 con 0x7f7190101c90 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7190111bb0 con 0x7f71901076a0 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f719657d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7190111eb0 con 0x7f7190108020 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f718ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50176/0 (socket says 172.21.15.96:50176) 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f718ffff640 1 -- 172.21.15.96:0/3212103881 learned_addr learned my addr 172.21.15.96:0/3212103881 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:46.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f7187fff640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71901076a0 0x7f71901978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.342+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7190108020 0x7f719019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 -- 172.21.15.96:0/3212103881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7190108020 msgr2=0x7f719019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7190108020 0x7f719019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 -- 172.21.15.96:0/3212103881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71901076a0 msgr2=0x7f71901978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71901076a0 0x7f71901978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 -- 172.21.15.96:0/3212103881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719019d1a0 con 0x7f7190101c90 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718ffff640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7180005ab0 tx=0x7f7180003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.343+0000 7f718dffb640 1 -- 172.21.15.96:0/3212103881 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718003f070 con 0x7f7190101c90 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.344+0000 7f718dffb640 1 -- 172.21.15.96:0/3212103881 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180031d60 con 0x7f7190101c90 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.344+0000 7f718dffb640 1 -- 172.21.15.96:0/3212103881 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180035030 con 0x7f7190101c90 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.344+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719019d430 con 0x7f7190101c90 2024-09-10T22:53:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.344+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719019db90 con 0x7f7190101c90 2024-09-10T22:53:46.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.347+0000 7f7198952740 1 Processor -- start 2024-09-10T22:53:46.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.347+0000 7f718dffb640 1 -- 172.21.15.96:0/3212103881 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f718004c060 con 0x7f7190101c90 2024-09-10T22:53:46.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.349+0000 7f7198952740 1 -- start start 2024-09-10T22:53:46.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.349+0000 7f718dffb640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 0x7f717c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.349+0000 7f718dffb640 1 -- 172.21.15.96:0/3212103881 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7180105ed0 con 0x7f7190101c90 2024-09-10T22:53:46.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.349+0000 7f7198952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75d7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7198952740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 0x7f717c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7198952740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7198952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2c754de50 con 0x55a2c75d70a0 2024-09-10T22:53:46.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7198952740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2c753d280 con 0x55a2c75e3950 2024-09-10T22:53:46.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7198952740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2c7541090 con 0x55a2c75d79c0 2024-09-10T22:53:46.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:41956/0 (socket says 172.21.15.96:41956) 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 -- 172.21.15.96:0/2253994450 learned_addr learned my addr 172.21.15.96:0/2253994450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f718ffff640 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75d7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 -- 172.21.15.96:0/2253994450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 msgr2=0x55a2c75e5d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75e5d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 -- 172.21.15.96:0/2253994450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 msgr2=0x55a2c75d7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75d7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.350+0000 7f7187fff640 1 -- 172.21.15.96:0/2253994450 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c729b650 con 0x55a2c75d79c0 2024-09-10T22:53:46.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.351+0000 7f7187fff640 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f717800bcb0 tx=0x7f71780107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0a9c1dd2f7ede5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.352+0000 7f71867fc640 1 -- 172.21.15.96:0/2253994450 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178004030 con 0x55a2c75d79c0 2024-09-10T22:53:46.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.352+0000 7f71867fc640 1 -- 172.21.15.96:0/2253994450 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71780041d0 con 0x55a2c75d79c0 2024-09-10T22:53:46.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f71867fc640 1 -- 172.21.15.96:0/2253994450 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780117b0 con 0x55a2c75d79c0 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 -- 172.21.15.96:0/2253994450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 msgr2=0x55a2c75e3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f717800bcb0 tx=0x7f71780107a0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 0x7f717c0b78f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7188005fd0 tx=0x7f7188009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 -- 172.21.15.96:0/2253994450 shutdown_connections 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e3410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 --2- 172.21.15.96:0/2253994450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75d7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 -- 172.21.15.96:0/2253994450 >> 172.21.15.96:0/2253994450 conn(0x55a2c75d4ed0 msgr2=0x55a2c75d6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 -- 172.21.15.96:0/2253994450 shutdown_connections 2024-09-10T22:53:46.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.353+0000 7f7198952740 1 -- 172.21.15.96:0/2253994450 wait complete. 2024-09-10T22:53:46.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 -- start start 2024-09-10T22:53:46.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 Processor -- start 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d79c0 0x55a2c75e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75e3950 0x55a2c75e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2c739e700 con 0x55a2c75e3950 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2c74d8110 con 0x55a2c75d79c0 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2c75e6620 con 0x55a2c75d70a0 2024-09-10T22:53:46.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7198952740 10 client.0 _ll_register_callbacks cb 0x55a2c74f1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f718ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f718ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:41972/0 (socket says 172.21.15.96:41972) 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f718ffff640 1 -- 172.21.15.96:0/3282174414 learned_addr learned my addr 172.21.15.96:0/3282174414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.354+0000 7f7187fff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d79c0 0x55a2c75e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 -- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d79c0 msgr2=0x55a2c75e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d79c0 0x55a2c75e07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 -- 172.21.15.96:0/3282174414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75e3950 msgr2=0x55a2c75e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75e3950 0x55a2c75e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 -- 172.21.15.96:0/3282174414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c7505c40 con 0x55a2c75d70a0 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f718ffff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7180000c00 tx=0x7f718002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2d8f8cafa0b3472 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.355+0000 7f716f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.357+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180049040 con 0x55a2c75d70a0 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.357+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f71800ccba0 con 0x55a2c75d70a0 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.357+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71780041d0 con 0x55a2c75d70a0 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71800cc250 con 0x55a2c75d70a0 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 10 client.26368 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:46.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2c75f20c0 con 0x55a2c75d70a0 2024-09-10T22:53:46.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71780041d0 con 0x55a2c75d70a0 2024-09-10T22:53:46.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f71800d5790 con 0x55a2c75d70a0 2024-09-10T22:53:46.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 10 client.26368 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:46.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 10 client.26368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:46.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 20 client.26368 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c75f3570 0x55a2c75f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 4 client.26368 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c754fda0 con 0x55a2c75f3570 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.358+0000 7f7187fff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c75f3570 0x55a2c75f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.359+0000 7f716f7fe640 10 client.26368 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.359+0000 7f7187fff640 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c75f3570 0x55a2c75f59c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3361ccdde61b8a2d server_cookie=922eff784e54f2 in_seq=0 out_seq=0 2024-09-10T22:53:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.359+0000 7f716f7fe640 1 -- 172.21.15.96:0/3282174414 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2c754fda0 con 0x55a2c75f3570 2024-09-10T22:53:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.359+0000 7f716f7fe640 10 client.26368 handle_command_reply: tid=0 2024-09-10T22:53:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.359+0000 7f7198952740 1 client.26368 shutdown 2024-09-10T22:53:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 msgr2=0x55a2c75e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7180000c00 tx=0x7f718002a410 comp rx=0 tx=0).stop 2024-09-10T22:53:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 shutdown_connections 2024-09-10T22:53:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c75f3570 0x55a2c75f59c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d79c0 0x55a2c75e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75e3950 0x55a2c75e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 --2- 172.21.15.96:0/3282174414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d70a0 0x55a2c75e3380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f716f7fe640 0 client.26368 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 >> 172.21.15.96:0/3282174414 conn(0x55a2c75d4ed0 msgr2=0x55a2c75daa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 shutdown_connections 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 1 -- 172.21.15.96:0/3282174414 wait complete. 2024-09-10T22:53:46.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.360+0000 7f7198952740 20 client.26368 trim_cache size 0 max 16384 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.385+0000 7f7198952740 1 Processor -- start 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.385+0000 7f7198952740 1 -- start start 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.385+0000 7f7198952740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 0x55a2c75ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.385+0000 7f7198952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d79c0 0x55a2c75d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.385+0000 7f7198952740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 0x55a2c75dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7198952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71901bd120 con 0x55a2c75d79c0 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7198952740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71901bec60 con 0x55a2c75d70a0 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7198952740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71901bd7f0 con 0x55a2c75e3950 2024-09-10T22:53:46.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f718ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 0x55a2c75ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f718ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 0x55a2c75ef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:32816/0 (socket says 172.21.15.96:32816) 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 0x55a2c75dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f718ffff640 1 -- 172.21.15.96:0/2698709508 learned_addr learned my addr 172.21.15.96:0/2698709508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7187fff640 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d79c0 0x55a2c75d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 -- 172.21.15.96:0/2698709508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 msgr2=0x55a2c75ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 0x55a2c75ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 -- 172.21.15.96:0/2698709508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d79c0 msgr2=0x55a2c75d6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d79c0 0x55a2c75d6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.386+0000 7f7194b1a640 1 -- 172.21.15.96:0/2698709508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c75d0790 con 0x55a2c75e3950 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.387+0000 7f7194b1a640 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 0x55a2c75dd9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7188003c90 tx=0x7f7188013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7437693df3079e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.387+0000 7f7185ffb640 1 -- 172.21.15.96:0/2698709508 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718801a460 con 0x55a2c75e3950 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.387+0000 7f7185ffb640 1 -- 172.21.15.96:0/2698709508 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801aab0 con 0x55a2c75e3950 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.387+0000 7f7185ffb640 1 -- 172.21.15.96:0/2698709508 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718801bab0 con 0x55a2c75e3950 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 -- 172.21.15.96:0/2698709508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 msgr2=0x55a2c75dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 0x55a2c75dd9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7188003c90 tx=0x7f7188013040 comp rx=0 tx=0).stop 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 -- 172.21.15.96:0/2698709508 shutdown_connections 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75e3950 0x55a2c75dd9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d79c0 0x55a2c75d6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.388+0000 7f7198952740 1 --2- 172.21.15.96:0/2698709508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75d70a0 0x55a2c75ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.389+0000 7f7198952740 1 -- 172.21.15.96:0/2698709508 >> 172.21.15.96:0/2698709508 conn(0x55a2c75d4ed0 msgr2=0x55a2c75df050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.389+0000 7f7198952740 1 -- 172.21.15.96:0/2698709508 shutdown_connections 2024-09-10T22:53:46.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.389+0000 7f7198952740 1 -- 172.21.15.96:0/2698709508 wait complete. 2024-09-10T22:53:46.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.389+0000 7f7198952740 1 -- start start 2024-09-10T22:53:46.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.389+0000 7f7198952740 1 Processor -- start 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7198952740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7198952740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7198952740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7198952740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71901be590 con 0x55a2c75d70a0 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7187fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7187fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:41986/0 (socket says 172.21.15.96:41986) 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7187fff640 1 -- 172.21.15.96:0/3848138133 learned_addr learned my addr 172.21.15.96:0/3848138133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.390+0000 7f718ffff640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71901bdec0 con 0x55a2c75e3950 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71901bb100 con 0x55a2c75d79c0 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7198952740 10 client.0 _ll_register_callbacks cb 0x7f7180003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7194b1a640 1 -- 172.21.15.96:0/3848138133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 msgr2=0x55a2c75e9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.391+0000 7f7194b1a640 1 -- 172.21.15.96:0/3848138133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 msgr2=0x55a2c75ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.392+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.392+0000 7f7194b1a640 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c75ea840 con 0x55a2c75e3950 2024-09-10T22:53:46.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.392+0000 7f7194b1a640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75ea010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a2c75e0df0 tx=0x7f7188005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02d1d3a23027661 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:46.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.392+0000 7f716f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:46.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.392+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188032020 con 0x55a2c75e3950 2024-09-10T22:53:46.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f718801a670 con 0x55a2c75e3950 2024-09-10T22:53:46.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718801aab0 con 0x55a2c75e3950 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f718801a890 con 0x55a2c75e3950 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f7198952740 10 client.26333 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2c76613d0 con 0x55a2c75e3950 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801aab0 con 0x55a2c75e3950 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.393+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7188023070 con 0x55a2c75e3950 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 10 client.26333 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 20 client.26333 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c7661a20 0x55a2c7663e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 4 client.26333 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2c754b850 con 0x55a2c7661a20 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f718ffff640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c7661a20 0x55a2c7663e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f716f7fe640 10 client.26333 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:46.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.394+0000 7f718ffff640 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c7661a20 0x55a2c7663e40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74b8360c011b84f server_cookie=dc907590adeea1a6 in_seq=0 out_seq=0 2024-09-10T22:53:46.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.395+0000 7f716f7fe640 1 -- 172.21.15.96:0/3848138133 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55a2c754b850 con 0x55a2c7661a20 2024-09-10T22:53:46.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.395+0000 7f716f7fe640 10 client.26333 handle_command_reply: tid=0 2024-09-10T22:53:46.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.395+0000 7f7198952740 1 client.26333 shutdown 2024-09-10T22:53:46.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 msgr2=0x55a2c75ea010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75ea010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a2c75e0df0 tx=0x7f7188005a50 comp rx=0 tx=0).stop 2024-09-10T22:53:46.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 shutdown_connections 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a2c7661a20 0x55a2c7663e40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2c75e3950 0x55a2c75ea010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f716f7fe640 0 client.26333 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2c75d79c0 0x55a2c75e9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.396+0000 7f7198952740 1 --2- 172.21.15.96:0/3848138133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2c75d70a0 0x55a2c75ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.397+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 >> 172.21.15.96:0/3848138133 conn(0x55a2c75d4ed0 msgr2=0x55a2c75d3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.397+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 shutdown_connections 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.397+0000 7f7198952740 1 -- 172.21.15.96:0/3848138133 wait complete. 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.397+0000 7f7198952740 20 client.26333 trim_cache size 0 max 16384 2024-09-10T22:53:46.400 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:53:46.401 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.403 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.404 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:53:46.405 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:53:46.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 msgr2=0x7f717c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 0x7f717c0b78f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7188005fd0 tx=0x7f7188009500 comp rx=0 tx=0).stop 2024-09-10T22:53:46.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 msgr2=0x7f7190197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:46.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7180005ab0 tx=0x7f7180003c80 comp rx=0 tx=0).stop 2024-09-10T22:53:46.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 shutdown_connections 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71901076a0 0x7f71901978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f717c0b5430 0x7f717c0b78f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7190108020 0x7f719019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 --2- 172.21.15.96:0/3212103881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7190101c90 0x7f7190197370 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 >> 172.21.15.96:0/3212103881 conn(0x7f71900fd700 msgr2=0x7f719006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 shutdown_connections 2024-09-10T22:53:46.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:46.398+0000 7f719657d640 1 -- 172.21.15.96:0/3212103881 wait complete. 2024-09-10T22:53:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:47 smithi096 ceph-mon[30869]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 793 op/s 2024-09-10T22:53:47.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:47 smithi117 ceph-mon[35081]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 793 op/s 2024-09-10T22:53:47.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:47 smithi121 ceph-mon[35119]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 793 op/s 2024-09-10T22:53:47.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:53:47.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:53:48.089 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 -- 172.21.15.96:0/142401121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d01076a0 msgr2=0x7f18d0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 --2- 172.21.15.96:0/142401121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d01076a0 0x7f18d0107ae0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009860 tx=0x7f18b802a340 comp rx=0 tx=0).stop 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 -- 172.21.15.96:0/142401121 shutdown_connections 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 --2- 172.21.15.96:0/142401121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 --2- 172.21.15.96:0/142401121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d01076a0 0x7f18d0107ae0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 --2- 172.21.15.96:0/142401121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d0101c90 0x7f18d0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 -- 172.21.15.96:0/142401121 >> 172.21.15.96:0/142401121 conn(0x7f18d00fd700 msgr2=0x7f18d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 -- 172.21.15.96:0/142401121 shutdown_connections 2024-09-10T22:53:48.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.745+0000 7f18d4a92640 1 -- 172.21.15.96:0/142401121 wait complete. 2024-09-10T22:53:48.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.746+0000 7f18d4a92640 1 Processor -- start 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.746+0000 7f18d4a92640 1 -- start start 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d0101c90 0x7f18d0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0111ce0 con 0x7f18d0101c90 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0111b60 con 0x7f18d01076a0 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18d4a92640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0111e60 con 0x7f18d0108020 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:32858/0 (socket says 172.21.15.96:32858) 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18c7fff640 1 -- 172.21.15.96:0/2391633247 learned_addr learned my addr 172.21.15.96:0/2391633247 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18ced9d640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.747+0000 7f18ce59c640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d0101c90 0x7f18d0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:48.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 -- 172.21.15.96:0/2391633247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 msgr2=0x7f18d019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:48.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 -- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d0101c90 msgr2=0x7f18d0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d0101c90 0x7f18d0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 -- 172.21.15.96:0/2391633247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d019f320 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18ced9d640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c7fff640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18b80303f0 tx=0x7f18b8032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8040020 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18b8030dd0 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8037040 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18d4a92640 1 -- 172.21.15.96:0/2391633247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d019f5b0 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.748+0000 7f18d4a92640 1 -- 172.21.15.96:0/2391633247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d00fdae0 con 0x7f18d01076a0 2024-09-10T22:53:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.749+0000 7f18d4a92640 1 -- 172.21.15.96:0/2391633247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d0102f40 con 0x7f18d01076a0 2024-09-10T22:53:48.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.751+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f18b8030450 con 0x7f18d01076a0 2024-09-10T22:53:48.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.752+0000 7f18c77fe640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 0x7f18bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:48.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.752+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f18b8105940 con 0x7f18d01076a0 2024-09-10T22:53:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.752+0000 7f18ce59c640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 0x7f18bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.753+0000 7f18ce59c640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 0x7f18bc0b77a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009bd0 tx=0x7f18c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:48.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:48.755+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18b804a030 con 0x7f18d01076a0 2024-09-10T22:53:49.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.089+0000 7f18d4a92640 1 -- 172.21.15.96:0/2391633247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d0102070 con 0x7f18d01076a0 2024-09-10T22:53:49.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.092+0000 7f18c77fe640 1 -- 172.21.15.96:0/2391633247 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f18d0102070 con 0x7f18d01076a0 2024-09-10T22:53:49.094 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:53:49.095 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:53:49.095 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:53:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 msgr2=0x7f18bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 0x7f18bc0b77a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009bd0 tx=0x7f18c0005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 msgr2=0x7f18d01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18b80303f0 tx=0x7f18b8032da0 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 shutdown_connections 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01076a0 0x7f18d01997b0 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18b80303f0 tx=0x7f18b8032da0 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f18bc0b52e0 0x7f18bc0b77a0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009bd0 tx=0x7f18c0005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18d0108020 0x7f18d019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 --2- 172.21.15.96:0/2391633247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18d0101c90 0x7f18d0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.101+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 >> 172.21.15.96:0/2391633247 conn(0x7f18d00fd700 msgr2=0x7f18d006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.102+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 shutdown_connections 2024-09-10T22:53:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:49.102+0000 7f18c57fa640 1 -- 172.21.15.96:0/2391633247 wait complete. 2024-09-10T22:53:49.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:49 smithi096 ceph-mon[30869]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 MiB/s wr, 635 op/s 2024-09-10T22:53:49.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2391633247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:49 smithi121 ceph-mon[35119]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 MiB/s wr, 635 op/s 2024-09-10T22:53:49.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:49 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2391633247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:49 smithi117 ceph-mon[35081]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.9 MiB/s wr, 635 op/s 2024-09-10T22:53:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:49 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2391633247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:53:50.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:53:50.864 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:50.956 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 375.0 s, 2929.3 tps, lat 33.949 ms stddev 40.004 2024-09-10T22:53:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.475+0000 7f7de9320640 1 -- 172.21.15.96:0/1255839290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de410d140 msgr2=0x7f7de410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.475+0000 7f7de9320640 1 --2- 172.21.15.96:0/1255839290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de410d140 0x7f7de410f5a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009ca0 tx=0x7f7ddc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 -- 172.21.15.96:0/1255839290 shutdown_connections 2024-09-10T22:53:51.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 --2- 172.21.15.96:0/1255839290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7de4104600 0x7f7de410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 --2- 172.21.15.96:0/1255839290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de410d140 0x7f7de410f5a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 --2- 172.21.15.96:0/1255839290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4103ce0 0x7f7de41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 -- 172.21.15.96:0/1255839290 >> 172.21.15.96:0/1255839290 conn(0x7f7de40fd740 msgr2=0x7f7de40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 -- 172.21.15.96:0/1255839290 shutdown_connections 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.477+0000 7f7de9320640 1 -- 172.21.15.96:0/1255839290 wait complete. 2024-09-10T22:53:51.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.478+0000 7f7de9320640 1 Processor -- start 2024-09-10T22:53:51.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.478+0000 7f7de9320640 1 -- start start 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de4103ce0 0x7f7de4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7de410d140 0x7f7de419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4112db0 con 0x7f7de4103ce0 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4112c30 con 0x7f7de410d140 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de9320640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4112f30 con 0x7f7de4104600 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:42028/0 (socket says 172.21.15.96:42028) 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 -- 172.21.15.96:0/1743756376 learned_addr learned my addr 172.21.15.96:0/1743756376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 -- 172.21.15.96:0/1743756376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7de410d140 msgr2=0x7f7de419eb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7de410d140 0x7f7de419eb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 -- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de4103ce0 msgr2=0x7f7de4199280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.479+0000 7f7de27fc640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de4103ce0 0x7f7de4199280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.480+0000 7f7de27fc640 1 -- 172.21.15.96:0/1743756376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de419f330 con 0x7f7de4104600 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.480+0000 7f7de27fc640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e9c0 tx=0x7f7dd800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.480+0000 7f7dcbfff640 1 -- 172.21.15.96:0/1743756376 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800ccc0 con 0x7f7de4104600 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.480+0000 7f7dcbfff640 1 -- 172.21.15.96:0/1743756376 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd80035f0 con 0x7f7de4104600 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.480+0000 7f7dcbfff640 1 -- 172.21.15.96:0/1743756376 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800ce60 con 0x7f7de4104600 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.481+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de419f620 con 0x7f7de4104600 2024-09-10T22:53:51.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.481+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de41a5ce0 con 0x7f7de4104600 2024-09-10T22:53:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.484+0000 7f7dcbfff640 1 -- 172.21.15.96:0/1743756376 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7dd8004000 con 0x7f7de4104600 2024-09-10T22:53:51.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.485+0000 7f7deb6f5740 1 Processor -- start 2024-09-10T22:53:51.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.485+0000 7f7deb6f5740 1 -- start start 2024-09-10T22:53:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.486+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.486+0000 7f7de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.486+0000 7f7de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:32878/0 (socket says 172.21.15.96:32878) 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f047df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f05e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56006efbdea0 con 0x56006f05c430 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56006efad2d0 con 0x56006f0474d0 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56006efb10e0 con 0x56006f051830 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7de37fe640 1 -- 172.21.15.96:0/3649719689 learned_addr learned my addr 172.21.15.96:0/3649719689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:51.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7dcbfff640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 0x7f7dcc0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7de27fc640 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f05e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.487+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f047df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 -- 172.21.15.96:0/3649719689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 msgr2=0x56006f047df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f047df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 -- 172.21.15.96:0/3649719689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 msgr2=0x56006f05e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f05e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 -- 172.21.15.96:0/3649719689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006ed0b650 con 0x56006f0474d0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7dcbfff640 1 -- 172.21.15.96:0/1743756376 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7dd80e5f20 con 0x7f7de4104600 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 0x7f7dcc0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7de37fe640 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc004af0 tx=0x7f7ddc005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1953488cf066003 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7dc97fa640 1 -- 172.21.15.96:0/3649719689 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0318e0 con 0x56006f0474d0 2024-09-10T22:53:51.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.488+0000 7f7dc97fa640 1 -- 172.21.15.96:0/3649719689 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ddc036070 con 0x56006f0474d0 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.489+0000 7f7deb6f5740 1 -- 172.21.15.96:0/3649719689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 msgr2=0x56006f0478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.489+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc004af0 tx=0x7f7ddc005ef0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.489+0000 7f7de37fe640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 0x7f7dcc0b78a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc04b000 tx=0x7f7ddc048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 -- 172.21.15.96:0/3649719689 shutdown_connections 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f05e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f047df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/3649719689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f0478b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 -- 172.21.15.96:0/3649719689 >> 172.21.15.96:0/3649719689 conn(0x56006f045300 msgr2=0x56006f0464a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 -- 172.21.15.96:0/3649719689 shutdown_connections 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.490+0000 7f7deb6f5740 1 -- 172.21.15.96:0/3649719689 wait complete. 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 -- start start 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 Processor -- start 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f051090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f05c430 0x56006f04ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56006ee0e700 con 0x56006f051830 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56006ef48160 con 0x56006f0474d0 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56006f05f0f0 con 0x56006f05c430 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7de27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7de27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50246/0 (socket says 172.21.15.96:50246) 2024-09-10T22:53:51.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7de27fc640 1 -- 172.21.15.96:0/2808824329 learned_addr learned my addr 172.21.15.96:0/2808824329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.491+0000 7f7deb6f5740 10 client.0 _ll_register_callbacks cb 0x56006ef619e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f051090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de37fe640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f05c430 0x56006f04ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 -- 172.21.15.96:0/2808824329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f05c430 msgr2=0x56006f04ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f05c430 0x56006f04ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 -- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 msgr2=0x56006f051090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f051090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 -- 172.21.15.96:0/2808824329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006ef75c90 con 0x56006f051830 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f051090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7de27fc640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c6a0 tx=0x7f7dd80ac1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2a4d1bdfb1f863 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7dc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80b5c70 con 0x56006f051830 2024-09-10T22:53:51.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7dd8004320 con 0x56006f051830 2024-09-10T22:53:51.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ddc036070 con 0x56006f051830 2024-09-10T22:53:51.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.492+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dd80ac6e0 con 0x56006f051830 2024-09-10T22:53:51.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.493+0000 7f7deb6f5740 10 client.16770 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:51.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.493+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56006f06aa70 con 0x56006f051830 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.494+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ddc036070 con 0x56006f051830 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.494+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7dd8071020 con 0x56006f051830 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.494+0000 7f7deb6f5740 10 client.16770 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.494+0000 7f7deb6f5740 10 client.16770 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.494+0000 7f7deb6f5740 20 client.16770 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:51.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f06bed0 0x56006f06e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7deb6f5740 4 client.16770 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56006efbfdf0 con 0x56006f06bed0 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f06bed0 0x56006f06e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f06bed0 0x56006f06e300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc87ccdfbd1b4b73 server_cookie=f4af3e5fe796f4df in_seq=0 out_seq=0 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.495+0000 7f7dc6ff5640 10 client.16770 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.496+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2808824329 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56006efbfdf0 con 0x56006f06bed0 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.496+0000 7f7dc6ff5640 10 client.16770 handle_command_reply: tid=0 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.496+0000 7f7deb6f5740 1 client.16770 shutdown 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 msgr2=0x56006f04e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800c6a0 tx=0x7f7dd80ac1f0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 shutdown_connections 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f06bed0 0x56006f06e300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f05c430 0x56006f04ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f04e4e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2808824329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f051090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7dc6ff5640 0 client.16770 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.497+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 >> 172.21.15.96:0/2808824329 conn(0x56006f045300 msgr2=0x56006f043c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.498+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 shutdown_connections 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.498+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2808824329 wait complete. 2024-09-10T22:53:51.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.498+0000 7f7deb6f5740 20 client.16770 trim_cache size 0 max 16384 2024-09-10T22:53:51.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 Processor -- start 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 -- start start 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f04d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f044920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41ac050 con 0x56006f05c430 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41adb90 con 0x56006f0474d0 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.516+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41ac720 con 0x56006f051830 2024-09-10T22:53:51.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:32902/0 (socket says 172.21.15.96:32902) 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 -- 172.21.15.96:0/1223337856 learned_addr learned my addr 172.21.15.96:0/1223337856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de37fe640 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f044920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de27fc640 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f04d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 -- 172.21.15.96:0/1223337856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 msgr2=0x56006f04d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f04d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 -- 172.21.15.96:0/1223337856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 msgr2=0x56006f044920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f044920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 -- 172.21.15.96:0/1223337856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006f043c00 con 0x56006f0474d0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4004700 tx=0x7f7dd4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783a99043d3f4c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.517+0000 7f7dc8ff9640 1 -- 172.21.15.96:0/1223337856 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4004030 con 0x56006f0474d0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.518+0000 7f7dc8ff9640 1 -- 172.21.15.96:0/1223337856 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd40035f0 con 0x56006f0474d0 2024-09-10T22:53:51.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.518+0000 7f7dc8ff9640 1 -- 172.21.15.96:0/1223337856 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40126b0 con 0x56006f0474d0 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 -- 172.21.15.96:0/1223337856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 msgr2=0x56006f047fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4004700 tx=0x7f7dd4005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 -- 172.21.15.96:0/1223337856 shutdown_connections 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f05c430 0x56006f044920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f051830 0x56006f04d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/1223337856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f0474d0 0x56006f047fc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 -- 172.21.15.96:0/1223337856 >> 172.21.15.96:0/1223337856 conn(0x56006f045300 msgr2=0x56006f0478b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 -- 172.21.15.96:0/1223337856 shutdown_connections 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.519+0000 7f7deb6f5740 1 -- 172.21.15.96:0/1223337856 wait complete. 2024-09-10T22:53:51.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.520+0000 7f7deb6f5740 1 -- start start 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.520+0000 7f7deb6f5740 1 Processor -- start 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f0474d0 0x56006f0646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f064c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41ad4c0 con 0x56006f051830 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41acdf0 con 0x56006f05c430 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41aa030 con 0x56006f0474d0 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7deb6f5740 10 client.0 _ll_register_callbacks cb 0x7f7dd80ac070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:32916/0 (socket says 172.21.15.96:32916) 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 -- 172.21.15.96:0/2797374974 learned_addr learned my addr 172.21.15.96:0/2797374974 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f0474d0 0x56006f0646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de27fc640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f064c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 -- 172.21.15.96:0/2797374974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f0474d0 msgr2=0x56006f0646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f0474d0 0x56006f0646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 -- 172.21.15.96:0/2797374974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 msgr2=0x56006f064c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f064c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de37fe640 1 -- 172.21.15.96:0/2797374974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006f05fca0 con 0x56006f05c430 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.521+0000 7f7de27fc640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f064c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7de37fe640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc02ade0 tx=0x7f7ddc03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd83438ef38683eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0318e0 con 0x56006f05c430 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7ddc003f00 con 0x56006f05c430 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd40035f0 con 0x56006f05c430 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ddc004100 con 0x56006f05c430 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 10 client.26354 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:51.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56006f0d9950 con 0x56006f05c430 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ddc036050 con 0x56006f05c430 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7ddc02f920 con 0x56006f05c430 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 10 client.26354 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 10 client.26354 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 20 client.26354 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f0d9ef0 0x56006f0dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 4 client.26354 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.522+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7dd40035f0 con 0x56006f0d9ef0 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f0d9ef0 0x56006f0dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7dc6ff5640 10 client.26354 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7de2ffd640 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f0d9ef0 0x56006f0dc310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4894c52e1be8da58 server_cookie=c9e921c629af74d5 in_seq=0 out_seq=0 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7dc6ff5640 1 -- 172.21.15.96:0/2797374974 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7dd40035f0 con 0x56006f0d9ef0 2024-09-10T22:53:51.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7dc6ff5640 10 client.26354 handle_command_reply: tid=0 2024-09-10T22:53:51.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.523+0000 7f7deb6f5740 1 client.26354 shutdown 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 msgr2=0x56006f05f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc02ade0 tx=0x7f7ddc03d000 comp rx=0 tx=0).stop 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 shutdown_connections 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x56006f0d9ef0 0x56006f0dc310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56006f05c430 0x56006f05f470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56006f051830 0x56006f064c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 --2- 172.21.15.96:0/2797374974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56006f0474d0 0x56006f0646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.525+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 >> 172.21.15.96:0/2797374974 conn(0x56006f045300 msgr2=0x56006f044170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.526+0000 7f7dc6ff5640 0 client.26354 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.526+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 shutdown_connections 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.526+0000 7f7deb6f5740 1 -- 172.21.15.96:0/2797374974 wait complete. 2024-09-10T22:53:51.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.526+0000 7f7deb6f5740 20 client.26354 trim_cache size 0 max 16384 2024-09-10T22:53:51.532 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 msgr2=0x7f7dcc0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 0x7f7dcc0b78a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc04b000 tx=0x7f7ddc048000 comp rx=0 tx=0).stop 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 msgr2=0x7f7de41997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e9c0 tx=0x7f7dd800ee80 comp rx=0 tx=0).stop 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 shutdown_connections 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7de410d140 0x7f7de419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7dcc0b53e0 0x7f7dcc0b78a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7de4104600 0x7f7de41997c0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e9c0 tx=0x7f7dd800ee80 comp rx=0 tx=0).stop 2024-09-10T22:53:51.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 --2- 172.21.15.96:0/1743756376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de4103ce0 0x7f7de4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:51.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 >> 172.21.15.96:0/1743756376 conn(0x7f7de40fd740 msgr2=0x7f7de40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:51.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.527+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 shutdown_connections 2024-09-10T22:53:51.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:51.528+0000 7f7de9320640 1 -- 172.21.15.96:0/1743756376 wait complete. 2024-09-10T22:53:51.543 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:51 smithi096 ceph-mon[30869]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 915 op/s 2024-09-10T22:53:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:51 smithi121 ceph-mon[35119]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 915 op/s 2024-09-10T22:53:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:51 smithi117 ceph-mon[35081]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 915 op/s 2024-09-10T22:53:52.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:53:52.967 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:53.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:53 smithi096 ceph-mon[30869]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-09-10T22:53:53.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 -- 172.21.15.96:0/1876672226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 msgr2=0x7fe884115ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 --2- 172.21.15.96:0/1876672226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe884115ed0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c02a2d0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 -- 172.21.15.96:0/1876672226 shutdown_connections 2024-09-10T22:53:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 --2- 172.21.15.96:0/1876672226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8841111b0 0x7fe8841135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 --2- 172.21.15.96:0/1876672226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe884115ed0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 --2- 172.21.15.96:0/1876672226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe88410a3b0 0x7fe88410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 -- 172.21.15.96:0/1876672226 >> 172.21.15.96:0/1876672226 conn(0x7fe88406cf20 msgr2=0x7fe88406d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 -- 172.21.15.96:0/1876672226 shutdown_connections 2024-09-10T22:53:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.535+0000 7fe888ba2640 1 -- 172.21.15.96:0/1876672226 wait complete. 2024-09-10T22:53:53.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.536+0000 7fe888ba2640 1 Processor -- start 2024-09-10T22:53:53.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.536+0000 7fe888ba2640 1 -- start start 2024-09-10T22:53:53.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 0x7fe88419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8841111b0 0x7fe88419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe884119120 con 0x7fe884113ae0 2024-09-10T22:53:53.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe884118fa0 con 0x7fe88410a3b0 2024-09-10T22:53:53.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe888ba2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841192a0 con 0x7fe8841111b0 2024-09-10T22:53:53.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50284/0 (socket says 172.21.15.96:50284) 2024-09-10T22:53:53.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 0x7fe88419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe883fff640 1 -- 172.21.15.96:0/864846865 learned_addr learned my addr 172.21.15.96:0/864846865 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:53.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe882ffd640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8841111b0 0x7fe88419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 -- 172.21.15.96:0/864846865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8841111b0 msgr2=0x7fe88419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8841111b0 0x7fe88419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 -- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 msgr2=0x7fe8841a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe8837fe640 1 -- 172.21.15.96:0/864846865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8841a56c0 con 0x7fe88410a3b0 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.537+0000 7fe883fff640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:53.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe8837fe640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 0x7fe88419f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe878009b40 tx=0x7fe87800f0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe880ff9640 1 -- 172.21.15.96:0/864846865 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87800fd10 con 0x7fe88410a3b0 2024-09-10T22:53:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe880ff9640 1 -- 172.21.15.96:0/864846865 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8780054d0 con 0x7fe88410a3b0 2024-09-10T22:53:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8841a59b0 con 0x7fe88410a3b0 2024-09-10T22:53:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe880ff9640 1 -- 172.21.15.96:0/864846865 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878017660 con 0x7fe88410a3b0 2024-09-10T22:53:53.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.538+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841ac070 con 0x7fe88410a3b0 2024-09-10T22:53:53.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.541+0000 7fe88c1d9740 1 Processor -- start 2024-09-10T22:53:53.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.542+0000 7fe88c1d9740 1 -- start start 2024-09-10T22:53:53.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.542+0000 7fe880ff9640 1 -- 172.21.15.96:0/864846865 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe878017800 con 0x7fe88410a3b0 2024-09-10T22:53:53.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de97ac0 0x559c3de9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3dea2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c3de0fea0 con 0x559c3de970c0 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c3ddff2d0 con 0x559c3de9c870 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559c3de030e0 con 0x559c3de97ac0 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50288/0 (socket says 172.21.15.96:50288) 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe883fff640 1 -- 172.21.15.96:0/1830173815 learned_addr learned my addr 172.21.15.96:0/1830173815 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.543+0000 7fe882ffd640 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3dea2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 -- 172.21.15.96:0/1830173815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de97ac0 msgr2=0x559c3de9c220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:53:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de97ac0 0x559c3de9c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 -- 172.21.15.96:0/1830173815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 msgr2=0x559c3dea2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3dea2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 -- 172.21.15.96:0/1830173815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3db5d650 con 0x559c3de970c0 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe883fff640 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ace0 tx=0x7fe87c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e620b41f41a67e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe880ff9640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 0x7fe86c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe8837fe640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 0x7fe86c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe869ffb640 1 -- 172.21.15.96:0/1830173815 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c041070 con 0x559c3de970c0 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe869ffb640 1 -- 172.21.15.96:0/1830173815 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87c007450 con 0x559c3de970c0 2024-09-10T22:53:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe880ff9640 1 -- 172.21.15.96:0/864846865 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe8780e52b0 con 0x7fe88410a3b0 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.544+0000 7fe869ffb640 1 -- 172.21.15.96:0/1830173815 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00a820 con 0x559c3de970c0 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1830173815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 msgr2=0x559c3de974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe8837fe640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 0x7fe86c0b7850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x559c3de97ec0 tx=0x7fe878072540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ace0 tx=0x7fe87c005c50 comp rx=0 tx=0).stop 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1830173815 shutdown_connections 2024-09-10T22:53:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3dea2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de97ac0 0x559c3de9c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1830173815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de970c0 0x559c3de974a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1830173815 >> 172.21.15.96:0/1830173815 conn(0x559c3de99640 msgr2=0x559c3de99a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1830173815 shutdown_connections 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.545+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1830173815 wait complete. 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 -- start start 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 Processor -- start 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de970c0 0x559c3deb6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3debc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c3dc60700 con 0x559c3de97ac0 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c3dd9a160 con 0x559c3de9c870 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559c3dea2c50 con 0x559c3de970c0 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe88c1d9740 10 client.0 _ll_register_callbacks cb 0x559c3ddb39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50304/0 (socket says 172.21.15.96:50304) 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe882ffd640 1 -- 172.21.15.96:0/1813184005 learned_addr learned my addr 172.21.15.96:0/1813184005 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe883fff640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de970c0 0x559c3deb6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.546+0000 7fe883fff640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3debc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 -- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de970c0 msgr2=0x559c3deb6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de970c0 0x559c3deb6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 -- 172.21.15.96:0/1813184005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 msgr2=0x559c3debc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3debc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 -- 172.21.15.96:0/1813184005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3ddc7c90 con 0x559c3de97ac0 2024-09-10T22:53:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe883fff640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3debc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe882ffd640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8740126e0 tx=0x7fe874012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4b608974340ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740120d0 con 0x559c3de97ac0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87c007450 con 0x559c3de97ac0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe87400f920 con 0x559c3de97ac0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe874012270 con 0x559c3de97ac0 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe88c1d9740 10 client.16788 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:53.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.547+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c3dec5750 con 0x559c3de97ac0 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87c007450 con 0x559c3de97ac0 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe87401c420 con 0x559c3de97ac0 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 10 client.16788 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 10 client.16788 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 20 client.16788 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3dec6bb0 0x559c3dec8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 4 client.16788 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:53.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3de11df0 con 0x559c3dec6bb0 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.548+0000 7fe883fff640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3dec6bb0 0x559c3dec8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.549+0000 7fe883fff640 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3dec6bb0 0x559c3dec8fd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a657fecf1cf9b03 server_cookie=8f1f04a1be585988 in_seq=0 out_seq=0 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.549+0000 7fe862ffd640 10 client.16788 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe862ffd640 1 -- 172.21.15.96:0/1813184005 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c3de11df0 con 0x559c3dec6bb0 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe862ffd640 10 client.16788 handle_command_reply: tid=0 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 client.16788 shutdown 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 msgr2=0x559c3deb74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8740126e0 tx=0x7fe874012ba0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 shutdown_connections 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3dec6bb0 0x559c3dec8fd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de9c870 0x559c3debc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3deb74f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/1813184005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de970c0 0x559c3deb6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 >> 172.21.15.96:0/1813184005 conn(0x559c3de99640 msgr2=0x559c3de9d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe862ffd640 0 client.16788 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:53.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 shutdown_connections 2024-09-10T22:53:53.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 1 -- 172.21.15.96:0/1813184005 wait complete. 2024-09-10T22:53:53.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.550+0000 7fe88c1d9740 20 client.16788 trim_cache size 0 max 16384 2024-09-10T22:53:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.568+0000 7fe88c1d9740 1 Processor -- start 2024-09-10T22:53:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.568+0000 7fe88c1d9740 1 -- start start 2024-09-10T22:53:53.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3de9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3dec3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3de9f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b45d0 con 0x559c3de97ac0 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b6110 con 0x559c3de970c0 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b4ca0 con 0x559c3de9c870 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3dec3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3de9f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3dec3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50310/0 (socket says 172.21.15.96:50310) 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe882ffd640 1 -- 172.21.15.96:0/4215136548 learned_addr learned my addr 172.21.15.96:0/4215136548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3de9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 -- 172.21.15.96:0/4215136548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 msgr2=0x559c3de9ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3de9ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 -- 172.21.15.96:0/4215136548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 msgr2=0x559c3dec3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3dec3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 -- 172.21.15.96:0/4215136548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3de97810 con 0x559c3de9c870 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.569+0000 7fe883fff640 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3de9f510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034040 tx=0x7fe87c02a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=304d3d9dfcdc5e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe8697fa640 1 -- 172.21.15.96:0/4215136548 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c041040 con 0x559c3de9c870 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe8697fa640 1 -- 172.21.15.96:0/4215136548 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87c007450 con 0x559c3de9c870 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe8697fa640 1 -- 172.21.15.96:0/4215136548 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c008d00 con 0x559c3de9c870 2024-09-10T22:53:53.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 -- 172.21.15.96:0/4215136548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 msgr2=0x559c3de9f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3de9f510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034040 tx=0x7fe87c02a2d0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 -- 172.21.15.96:0/4215136548 shutdown_connections 2024-09-10T22:53:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3de9f510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3dec3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/4215136548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3de9ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.570+0000 7fe88c1d9740 1 -- 172.21.15.96:0/4215136548 >> 172.21.15.96:0/4215136548 conn(0x559c3de99640 msgr2=0x559c3de9e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- 172.21.15.96:0/4215136548 shutdown_connections 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- 172.21.15.96:0/4215136548 wait complete. 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- start start 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 Processor -- start 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3deceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3decd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b5a40 con 0x559c3de97ac0 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b5370 con 0x559c3de970c0 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.571+0000 7fe88c1d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8841b25b0 con 0x559c3de9c870 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe88c1d9740 10 client.0 _ll_register_callbacks cb 0x7fe874009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:53.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50318/0 (socket says 172.21.15.96:50318) 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 -- 172.21.15.96:0/2391266660 learned_addr learned my addr 172.21.15.96:0/2391266660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe883fff640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3decd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe883fff640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3deceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 -- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 msgr2=0x559c3decd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3decd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 -- 172.21.15.96:0/2391266660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 msgr2=0x559c3deceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3deceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 -- 172.21.15.96:0/2391266660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3decdf40 con 0x559c3de97ac0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe883fff640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3deceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe882ffd640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe8740059f0 tx=0x7fe87401c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8810e65f7ac7cd38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402b030 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe874018070 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.572+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87c007450 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe87401ccc0 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 10 client.16794 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c3df34460 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe87c007450 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe8740055c0 con 0x559c3de97ac0 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 10 client.16794 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 10 client.16794 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 20 client.16794 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3df34c00 0x559c3df37020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 4 client.16794 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c3de0d8a0 con 0x559c3df34c00 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.573+0000 7fe883fff640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3df34c00 0x559c3df37020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.574+0000 7fe883fff640 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3df34c00 0x559c3df37020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a31f05db86f9b876 server_cookie=6816ef61d323d6f in_seq=0 out_seq=0 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.574+0000 7fe862ffd640 10 client.16794 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.574+0000 7fe862ffd640 1 -- 172.21.15.96:0/2391266660 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c3de0d8a0 con 0x559c3df34c00 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.574+0000 7fe862ffd640 10 client.16794 handle_command_reply: tid=0 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.574+0000 7fe88c1d9740 1 client.16794 shutdown 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 msgr2=0x559c3decf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe8740059f0 tx=0x7fe87401c650 comp rx=0 tx=0).stop 2024-09-10T22:53:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 shutdown_connections 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x559c3df34c00 0x559c3df37020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c3de9c870 0x559c3decd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c3de97ac0 0x559c3decf0d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 --2- 172.21.15.96:0/2391266660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c3de970c0 0x559c3deceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 >> 172.21.15.96:0/2391266660 conn(0x559c3de99640 msgr2=0x559c3dea4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe862ffd640 0 client.16794 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 shutdown_connections 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 1 -- 172.21.15.96:0/2391266660 wait complete. 2024-09-10T22:53:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe88c1d9740 20 client.16794 trim_cache size 0 max 16384 2024-09-10T22:53:53.591 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 msgr2=0x7fe86c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 0x7fe86c0b7850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x559c3de97ec0 tx=0x7fe878072540 comp rx=0 tx=0).stop 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 msgr2=0x7fe88419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.575+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 0x7fe88419f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe878009b40 tx=0x7fe87800f0d0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 shutdown_connections 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe86c0b5390 0x7fe86c0b7850 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe884113ae0 0x7fe8841a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8841111b0 0x7fe88419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 --2- 172.21.15.96:0/864846865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88410a3b0 0x7fe88419f610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 >> 172.21.15.96:0/864846865 conn(0x7fe88406cf20 msgr2=0x7fe884113ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 shutdown_connections 2024-09-10T22:53:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:53.576+0000 7fe888ba2640 1 -- 172.21.15.96:0/864846865 wait complete. 2024-09-10T22:53:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:53 smithi121 ceph-mon[35119]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-09-10T22:53:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:53 smithi117 ceph-mon[35081]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-09-10T22:53:55.693 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:55 smithi096 ceph-mon[30869]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 765 op/s 2024-09-10T22:53:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:55 smithi121 ceph-mon[35119]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 765 op/s 2024-09-10T22:53:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:55 smithi117 ceph-mon[35081]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 765 op/s 2024-09-10T22:53:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 380.0 s, 2363.2 tps, lat 42.517 ms stddev 47.479 2024-09-10T22:53:56.147 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:53:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 -- 172.21.15.96:0/328838101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 msgr2=0x7f5bc410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/328838101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc410e820 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009ca0 tx=0x7f5bbc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 -- 172.21.15.96:0/328838101 shutdown_connections 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/328838101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc410e820 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/328838101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc41076a0 0x7f5bc4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/328838101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc4101c90 0x7f5bc4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 -- 172.21.15.96:0/328838101 >> 172.21.15.96:0/328838101 conn(0x7f5bc40fd700 msgr2=0x7f5bc40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.023+0000 7f5bc8fed640 1 -- 172.21.15.96:0/328838101 shutdown_connections 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.024+0000 7f5bc8fed640 1 -- 172.21.15.96:0/328838101 wait complete. 2024-09-10T22:53:57.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 Processor -- start 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 -- start start 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc4101c90 0x7f5bc4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 0x7f5bc4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4111ce0 con 0x7f5bc4108020 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4111b60 con 0x7f5bc41076a0 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc8fed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4111e60 con 0x7f5bc4101c90 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43064/0 (socket says 172.21.15.96:43064) 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 0x7f5bc4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.025+0000 7f5bc2d9d640 1 -- 172.21.15.96:0/2028189514 learned_addr learned my addr 172.21.15.96:0/2028189514 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:57.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc4101c90 0x7f5bc4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 -- 172.21.15.96:0/2028189514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc4101c90 msgr2=0x7f5bc4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc4101c90 0x7f5bc4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 -- 172.21.15.96:0/2028189514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 msgr2=0x7f5bc4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 0x7f5bc4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 -- 172.21.15.96:0/2028189514 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc41a7630 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 0x7f5bc4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc2d9d640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0317c0 tx=0x7f5bbc0317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bbb7fe640 1 -- 172.21.15.96:0/2028189514 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc003b70 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bbb7fe640 1 -- 172.21.15.96:0/2028189514 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bbc003d10 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc41a78c0 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bbb7fe640 1 -- 172.21.15.96:0/2028189514 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc02fa10 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.026+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41a7e00 con 0x7f5bc4108020 2024-09-10T22:53:57.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.030+0000 7f5bcb3c2740 1 Processor -- start 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.030+0000 7f5bbb7fe640 1 -- 172.21.15.96:0/2028189514 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5bbc02fc30 con 0x7f5bc4108020 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bcb3c2740 1 -- start start 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b6689840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b668c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.031+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56490/0 (socket says 172.21.15.96:56490) 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 -- 172.21.15.96:0/4018772207 learned_addr learned my addr 172.21.15.96:0/4018772207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7b65f3ea0 con 0x55e7b667ddf0 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e7b65e32d0 con 0x55e7b667d4d0 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e7b65e70e0 con 0x55e7b6689d80 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b668c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 -- 172.21.15.96:0/4018772207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 msgr2=0x55e7b668c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b668c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 -- 172.21.15.96:0/4018772207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 msgr2=0x55e7b6689840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b6689840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbbfff640 1 -- 172.21.15.96:0/4018772207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b6341650 con 0x55e7b667d4d0 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.032+0000 7f5bbb7fe640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 0x7f5bb00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.033+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00eb10 tx=0x7f5bac00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c562a07a880dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.033+0000 7f5bbb7fe640 1 -- 172.21.15.96:0/2028189514 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5bbc106e90 con 0x7f5bc4108020 2024-09-10T22:53:57.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.037+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 0x7f5bb00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.041+0000 7f5bb8ff9640 1 -- 172.21.15.96:0/4018772207 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac004020 con 0x55e7b667d4d0 2024-09-10T22:53:57.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.041+0000 7f5bb8ff9640 1 -- 172.21.15.96:0/4018772207 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0054d0 con 0x55e7b667d4d0 2024-09-10T22:53:57.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.041+0000 7f5bb8ff9640 1 -- 172.21.15.96:0/4018772207 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac004ba0 con 0x55e7b667d4d0 2024-09-10T22:53:57.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.046+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 msgr2=0x55e7b667d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.046+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00eb10 tx=0x7f5bac00bb20 comp rx=0 tx=0).stop 2024-09-10T22:53:57.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.054+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 0x7f5bb00b7820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e7b667c1e0 tx=0x7f5bb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.055+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 shutdown_connections 2024-09-10T22:53:57.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.055+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b668c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.055+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b6689840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.055+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/4018772207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b667d8b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.055+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 >> 172.21.15.96:0/4018772207 conn(0x55e7b667b300 msgr2=0x55e7b667c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.060+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 shutdown_connections 2024-09-10T22:53:57.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.063+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/4018772207 wait complete. 2024-09-10T22:53:57.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.063+0000 7f5bcb3c2740 1 -- start start 2024-09-10T22:53:57.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.063+0000 7f5bcb3c2740 1 Processor -- start 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667d4d0 0x55e7b66896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b66871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7b6444700 con 0x55e7b667d4d0 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e7b657e160 con 0x55e7b667ddf0 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e7b668ca00 con 0x55e7b6689d80 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bcb3c2740 10 client.0 _ll_register_callbacks cb 0x55e7b65979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56504/0 (socket says 172.21.15.96:56504) 2024-09-10T22:53:57.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bbbfff640 1 -- 172.21.15.96:0/496961867 learned_addr learned my addr 172.21.15.96:0/496961867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.064+0000 7f5bc2d9d640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b66871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.065+0000 7f5bbbfff640 1 -- 172.21.15.96:0/496961867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 msgr2=0x55e7b66871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.065+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b66871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.065+0000 7f5bbbfff640 1 -- 172.21.15.96:0/496961867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667d4d0 msgr2=0x55e7b66896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.065+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667d4d0 0x55e7b66896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.065+0000 7f5bbbfff640 1 -- 172.21.15.96:0/496961867 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b65abc90 con 0x55e7b667ddf0 2024-09-10T22:53:57.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.068+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004a70 tx=0x7f5bac004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885ddaac7209d2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.068+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:57.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.069+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00d9d0 con 0x55e7b667ddf0 2024-09-10T22:53:57.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.069+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac0054d0 con 0x55e7b667ddf0 2024-09-10T22:53:57.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.069+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5bac00db70 con 0x55e7b667ddf0 2024-09-10T22:53:57.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.072+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5bac00dd70 con 0x55e7b667ddf0 2024-09-10T22:53:57.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.074+0000 7f5bcb3c2740 10 client.26384 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:57.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.074+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7b66983c0 con 0x55e7b667ddf0 2024-09-10T22:53:57.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0054d0 con 0x55e7b667ddf0 2024-09-10T22:53:57.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5bac027a30 con 0x55e7b667ddf0 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 10 client.26384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 20 client.26384 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b66998c0 0x55e7b669bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 4 client.26384 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:57.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.078+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7b65f5df0 con 0x55e7b66998c0 2024-09-10T22:53:57.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.079+0000 7f5bc259c640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b66998c0 0x55e7b669bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.084+0000 7f5bc259c640 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b66998c0 0x55e7b669bd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c2b7d9565d83973 server_cookie=9fd8bf751ea2e7f9 in_seq=0 out_seq=0 2024-09-10T22:53:57.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.084+0000 7f5ba9ffb640 10 client.26384 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:57.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.087+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/496961867 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e7b65f5df0 con 0x55e7b66998c0 2024-09-10T22:53:57.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.087+0000 7f5ba9ffb640 10 client.26384 handle_command_reply: tid=0 2024-09-10T22:53:57.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.087+0000 7f5bcb3c2740 1 client.26384 shutdown 2024-09-10T22:53:57.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.088+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 msgr2=0x55e7b6686b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.088+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004a70 tx=0x7f5bac004aa0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 shutdown_connections 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b66998c0 0x55e7b669bd50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667ddf0 0x55e7b6686b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b6689d80 0x55e7b66871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/496961867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667d4d0 0x55e7b66896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 >> 172.21.15.96:0/496961867 conn(0x55e7b667b300 msgr2=0x55e7b6680ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5ba9ffb640 0 client.26384 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 shutdown_connections 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/496961867 wait complete. 2024-09-10T22:53:57.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.089+0000 7f5bcb3c2740 20 client.26384 trim_cache size 0 max 16384 2024-09-10T22:53:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.112+0000 7f5bcb3c2740 1 Processor -- start 2024-09-10T22:53:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 -- start start 2024-09-10T22:53:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667d4d0 0x55e7b6695c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b6689d80 0x55e7b667c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41ac0e0 con 0x55e7b667ddf0 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41adc20 con 0x55e7b6689d80 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bcb3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41ac7b0 con 0x55e7b667d4d0 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43074/0 (socket says 172.21.15.96:43074) 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 -- 172.21.15.96:0/3983912112 learned_addr learned my addr 172.21.15.96:0/3983912112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bc2d9d640 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b6689d80 0x55e7b667c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bc259c640 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667d4d0 0x55e7b6695c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 -- 172.21.15.96:0/3983912112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667d4d0 msgr2=0x55e7b6695c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667d4d0 0x55e7b6695c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 -- 172.21.15.96:0/3983912112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b6689d80 msgr2=0x55e7b667c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b6689d80 0x55e7b667c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 -- 172.21.15.96:0/3983912112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b6687a80 con 0x55e7b667ddf0 2024-09-10T22:53:57.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.113+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00f010 tx=0x7f5bac00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d23c1c23886e99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.114+0000 7f5babfff640 1 -- 172.21.15.96:0/3983912112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0305f0 con 0x55e7b667ddf0 2024-09-10T22:53:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.114+0000 7f5babfff640 1 -- 172.21.15.96:0/3983912112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac00e350 con 0x55e7b667ddf0 2024-09-10T22:53:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.114+0000 7f5babfff640 1 -- 172.21.15.96:0/3983912112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0055f0 con 0x55e7b667ddf0 2024-09-10T22:53:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/3983912112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 msgr2=0x55e7b667ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00f010 tx=0x7f5bac00e880 comp rx=0 tx=0).stop 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/3983912112 shutdown_connections 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b6689d80 0x55e7b667c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b667ddf0 0x55e7b667ac10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/3983912112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667d4d0 0x55e7b6695c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/3983912112 >> 172.21.15.96:0/3983912112 conn(0x55e7b667b300 msgr2=0x55e7b667c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/3983912112 shutdown_connections 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.115+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/3983912112 wait complete. 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.116+0000 7f5bcb3c2740 1 -- start start 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.116+0000 7f5bcb3c2740 1 Processor -- start 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 0x55e7b66a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bcb3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 0x55e7b6690cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bc259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bc259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56528/0 (socket says 172.21.15.96:56528) 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bc2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 0x55e7b6690cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bc2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 0x55e7b6690cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43080/0 (socket says 172.21.15.96:43080) 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 0x55e7b66a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 0x55e7b66a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45396/0 (socket says 172.21.15.96:45396) 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bc259c640 1 -- 172.21.15.96:0/2017802961 learned_addr learned my addr 172.21.15.96:0/2017802961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.117+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41ad550 con 0x55e7b6689d80 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41ace80 con 0x55e7b667d4d0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41aa0e0 con 0x55e7b667ddf0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bcb3c2740 10 client.0 _ll_register_callbacks cb 0x7f5bac00e5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 -- 172.21.15.96:0/2017802961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 msgr2=0x55e7b66a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 0x55e7b66a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 -- 172.21.15.96:0/2017802961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 msgr2=0x55e7b6690cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:57.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 0x55e7b6690cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b66913b0 con 0x55e7b667d4d0 2024-09-10T22:53:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5bc259c640 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011f70 tx=0x7f5bb4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c1e2a38d07e2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.118+0000 7f5ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb401e420 con 0x55e7b667d4d0 2024-09-10T22:53:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5bb401e5c0 con 0x55e7b667d4d0 2024-09-10T22:53:57.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bac00e350 con 0x55e7b667d4d0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5bb401e7c0 con 0x55e7b667d4d0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 10 client.26390 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e7b66926c0 con 0x55e7b667d4d0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb401ee10 con 0x55e7b667d4d0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5bb4038c00 con 0x55e7b667d4d0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 10 client.26390 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 10 client.26390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 20 client.26390 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b6706a70 0x55e7b6708e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.119+0000 7f5bcb3c2740 4 client.26390 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5bac00e350 con 0x55e7b6706a70 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b6706a70 0x55e7b6708e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:57.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5ba9ffb640 10 client.26390 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5bbbfff640 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b6706a70 0x55e7b6708e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3e6879ff80de091 server_cookie=17a688e80e32a4f9 in_seq=0 out_seq=0 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5ba9ffb640 1 -- 172.21.15.96:0/2017802961 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5bac00e350 con 0x55e7b6706a70 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5ba9ffb640 10 client.26390 handle_command_reply: tid=0 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.120+0000 7f5bcb3c2740 1 client.26390 shutdown 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 msgr2=0x55e7b66a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4011f70 tx=0x7f5bb4004730 comp rx=0 tx=0).stop 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 shutdown_connections 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55e7b6706a70 0x55e7b6708e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7b6689d80 0x55e7b6690cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e7b667ddf0 0x55e7b66a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 --2- 172.21.15.96:0/2017802961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e7b667d4d0 0x55e7b66a0e30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5ba9ffb640 0 client.26390 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 >> 172.21.15.96:0/2017802961 conn(0x55e7b667b300 msgr2=0x55e7b6688160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 shutdown_connections 2024-09-10T22:53:57.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 1 -- 172.21.15.96:0/2017802961 wait complete. 2024-09-10T22:53:57.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.121+0000 7f5bcb3c2740 20 client.26390 trim_cache size 0 max 16384 2024-09-10T22:53:57.127 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:57.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 msgr2=0x7f5bb00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 0x7f5bb00b7820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e7b667c1e0 tx=0x7f5bb4005f90 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 msgr2=0x7f5bc4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0317c0 tx=0x7f5bbc0317f0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 shutdown_connections 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc41076a0 0x7f5bc4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5bb00b5360 0x7f5bb00b7820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc4108020 0x7f5bc4198870 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 --2- 172.21.15.96:0/2028189514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bc4101c90 0x7f5bc4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 >> 172.21.15.96:0/2028189514 conn(0x7f5bc40fd700 msgr2=0x7f5bc40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 shutdown_connections 2024-09-10T22:53:57.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:57.122+0000 7f5bc8fed640 1 -- 172.21.15.96:0/2028189514 wait complete. 2024-09-10T22:53:57.683 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:57 smithi096 ceph-mon[30869]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T22:53:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:57 smithi121 ceph-mon[35119]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T22:53:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:57 smithi117 ceph-mon[35081]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T22:53:58.054 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:53:58.415 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 -- 172.21.15.96:0/3359877899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc06bc90 msgr2=0x7f82cc105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/3359877899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc06bc90 0x7f82cc105540 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 -- 172.21.15.96:0/3359877899 shutdown_connections 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/3359877899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc105ad0 0x7f82cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/3359877899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/3359877899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc06bc90 0x7f82cc105540 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 -- 172.21.15.96:0/3359877899 >> 172.21.15.96:0/3359877899 conn(0x7f82cc0fd130 msgr2=0x7f82cc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 -- 172.21.15.96:0/3359877899 shutdown_connections 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.877+0000 7f82d1d0e640 1 -- 172.21.15.96:0/3359877899 wait complete. 2024-09-10T22:53:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 Processor -- start 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 -- start start 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc06bc90 0x7f82cc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10e620 con 0x7f82cc105ad0 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10e4a0 con 0x7f82cc06bc90 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82d1d0e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc10e7a0 con 0x7f82cc108680 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43102/0 (socket says 172.21.15.96:43102) 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82caffd640 1 -- 172.21.15.96:0/2358383216 learned_addr learned my addr 172.21.15.96:0/2358383216 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82cbfff640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.878+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc06bc90 0x7f82cc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 -- 172.21.15.96:0/2358383216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc06bc90 msgr2=0x7f82cc198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc06bc90 0x7f82cc198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 -- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 msgr2=0x7f82cc1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 -- 172.21.15.96:0/2358383216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc19a890 con 0x7f82cc108680 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82caffd640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82cbfff640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc199fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009fd0 tx=0x7f82c400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.879+0000 7f82c8ff9640 1 -- 172.21.15.96:0/2358383216 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c400ce20 con 0x7f82cc108680 2024-09-10T22:53:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.880+0000 7f82c8ff9640 1 -- 172.21.15.96:0/2358383216 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c4005020 con 0x7f82cc108680 2024-09-10T22:53:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.880+0000 7f82c8ff9640 1 -- 172.21.15.96:0/2358383216 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4010690 con 0x7f82cc108680 2024-09-10T22:53:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.881+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc19ab80 con 0x7f82cc108680 2024-09-10T22:53:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.881+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc101320 con 0x7f82cc108680 2024-09-10T22:53:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.884+0000 7f82c8ff9640 1 -- 172.21.15.96:0/2358383216 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f82c4020070 con 0x7f82cc108680 2024-09-10T22:53:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 Processor -- start 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 -- start start 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82c8ff9640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 0x7f82b40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560cb4d986f0 con 0x560cb4e29190 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560cb4d87b20 con 0x560cb4e2bbd0 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82d4adb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560cb4d8b930 con 0x560cb4e26860 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 0x7f82b40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82c8ff9640 1 -- 172.21.15.96:0/2358383216 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82c40e6190 con 0x7f82cc108680 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.885+0000 7f82caffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45426/0 (socket says 172.21.15.96:45426) 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 -- 172.21.15.96:0/291239977 learned_addr learned my addr 172.21.15.96:0/291239977 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82cbfff640 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 -- 172.21.15.96:0/291239977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 msgr2=0x560cb4e2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 -- 172.21.15.96:0/291239977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 msgr2=0x560cb4e2b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e2b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 -- 172.21.15.96:0/291239977 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb4d4d530 con 0x560cb4e26860 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 0x7f82b40b78f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f82bc031500 tx=0x7f82bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82caffd640 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f82c0012c80 tx=0x7f82c000bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05598bea5eb5ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82b1ffb640 1 -- 172.21.15.96:0/291239977 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0012e30 con 0x560cb4e26860 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82b1ffb640 1 -- 172.21.15.96:0/291239977 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c00054d0 con 0x560cb4e26860 2024-09-10T22:53:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.886+0000 7f82b1ffb640 1 -- 172.21.15.96:0/291239977 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004c00 con 0x560cb4e26860 2024-09-10T22:53:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 -- 172.21.15.96:0/291239977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 msgr2=0x560cb4e28c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f82c0012c80 tx=0x7f82c000bcc0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 -- 172.21.15.96:0/291239977 shutdown_connections 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 --2- 172.21.15.96:0/291239977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e28c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 -- 172.21.15.96:0/291239977 >> 172.21.15.96:0/291239977 conn(0x560cb4e219b0 msgr2=0x560cb4e22740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 -- 172.21.15.96:0/291239977 shutdown_connections 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.887+0000 7f82d4adb740 1 -- 172.21.15.96:0/291239977 wait complete. 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.888+0000 7f82d4adb740 1 -- start start 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.888+0000 7f82d4adb740 1 Processor -- start 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e30680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45434/0 (socket says 172.21.15.96:45434) 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82cb7fe640 1 -- 172.21.15.96:0/217123161 learned_addr learned my addr 172.21.15.96:0/217123161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e31250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560cb4cbfab0 con 0x560cb4e29190 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.889+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560cb4d2d730 con 0x560cb4e2bbd0 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cbfff640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e31250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560cb4e2ebb0 con 0x560cb4e26860 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 -- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 msgr2=0x560cb4e31250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e31250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 -- 172.21.15.96:0/217123161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 msgr2=0x560cb4e30680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e30680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb4ae9790 con 0x560cb4e26860 2024-09-10T22:53:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.890+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a48deaa6301f966 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82aaffd640 1 -- 172.21.15.96:0/217123161 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc04c070 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82d4adb740 10 client.0 _ll_register_callbacks cb 0x560cb4d816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c00054d0 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82aaffd640 1 -- 172.21.15.96:0/217123161 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82bc0315a0 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82aaffd640 1 -- 172.21.15.96:0/217123161 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82bc058050 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82d4adb740 10 client.26443 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.891+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x560cb4b62620 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82aaffd640 1 -- 172.21.15.96:0/217123161 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f82bc03a040 con 0x560cb4e26860 2024-09-10T22:53:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 10 client.26443 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 20 client.26443 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4e3b620 0x560cb4e3da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 4 client.26443 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cb4d9a640 con 0x560cb4e3b620 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.892+0000 7f82caffd640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4e3b620 0x560cb4e3da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.893+0000 7f82caffd640 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4e3b620 0x560cb4e3da90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d04a01a2178768 server_cookie=fa574cb0897e4f71 in_seq=0 out_seq=0 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.893+0000 7f82aaffd640 10 client.26443 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.893+0000 7f82aaffd640 1 -- 172.21.15.96:0/217123161 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560cb4d9a640 con 0x560cb4e3b620 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.893+0000 7f82aaffd640 10 client.26443 handle_command_reply: tid=0 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.893+0000 7f82d4adb740 1 client.26443 shutdown 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 msgr2=0x560cb4e2ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc009260 comp rx=0 tx=0).stop 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 shutdown_connections 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4e3b620 0x560cb4e3da90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e2bbd0 0x560cb4e31250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82aaffd640 0 client.26443 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e29190 0x560cb4e30680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 --2- 172.21.15.96:0/217123161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ffe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 >> 172.21.15.96:0/217123161 conn(0x560cb4e219b0 msgr2=0x560cb4e20bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 shutdown_connections 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 1 -- 172.21.15.96:0/217123161 wait complete. 2024-09-10T22:53:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.894+0000 7f82d4adb740 20 client.26443 trim_cache size 0 max 16384 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.914+0000 7f82d4adb740 1 Processor -- start 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.914+0000 7f82d4adb740 1 -- start start 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b02e0 con 0x560cb4e2bbd0 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b1e20 con 0x560cb4e29190 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82d4adb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b09b0 con 0x560cb4e26860 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56572/0 (socket says 172.21.15.96:56572) 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 -- 172.21.15.96:0/2277043088 learned_addr learned my addr 172.21.15.96:0/2277043088 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82cbfff640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 -- 172.21.15.96:0/2277043088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 msgr2=0x560cb4e2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 -- 172.21.15.96:0/2277043088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e2bbd0 msgr2=0x560cb4e2bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82caffd640 1 -- 172.21.15.96:0/2277043088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb4e28fc0 con 0x560cb4e29190 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.915+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.916+0000 7f82caffd640 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f82c00057c0 tx=0x7f82c001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe2af3eb7375b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.916+0000 7f82b17fa640 1 -- 172.21.15.96:0/2277043088 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0010c90 con 0x560cb4e29190 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.916+0000 7f82b17fa640 1 -- 172.21.15.96:0/2277043088 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c0016040 con 0x560cb4e29190 2024-09-10T22:53:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.916+0000 7f82b17fa640 1 -- 172.21.15.96:0/2277043088 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00039e0 con 0x560cb4e29190 2024-09-10T22:53:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.917+0000 7f82d4adb740 1 -- 172.21.15.96:0/2277043088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 msgr2=0x560cb4e2f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.917+0000 7f82d4adb740 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f82c00057c0 tx=0x7f82c001b040 comp rx=0 tx=0).stop 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 -- 172.21.15.96:0/2277043088 shutdown_connections 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e2f4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e2bbd0 0x560cb4e2bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 --2- 172.21.15.96:0/2277043088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e26860 0x560cb4e2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 -- 172.21.15.96:0/2277043088 >> 172.21.15.96:0/2277043088 conn(0x560cb4e219b0 msgr2=0x560cb4e25e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 -- 172.21.15.96:0/2277043088 shutdown_connections 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 -- 172.21.15.96:0/2277043088 wait complete. 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 -- start start 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.918+0000 7f82d4adb740 1 Processor -- start 2024-09-10T22:53:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e2bbd0 0x560cb4e36b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b1750 con 0x560cb4e26860 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1b1080 con 0x560cb4e29190 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc1ae2e0 con 0x560cb4e2bbd0 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82d4adb740 10 client.0 _ll_register_callbacks cb 0x7f82bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43144/0 (socket says 172.21.15.96:43144) 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 -- 172.21.15.96:0/910650165 learned_addr learned my addr 172.21.15.96:0/910650165 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cbfff640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e2bbd0 0x560cb4e36b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82caffd640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 -- 172.21.15.96:0/910650165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e2bbd0 msgr2=0x560cb4e36b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e2bbd0 0x560cb4e36b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 -- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 msgr2=0x560cb4e365e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e365e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 -- 172.21.15.96:0/910650165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb4e2b8f0 con 0x560cb4e26860 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82cb7fe640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006010 tx=0x7f82bc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eeca4c9e8df39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.919+0000 7f82aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc061020 con 0x560cb4e26860 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82bc05d450 con 0x560cb4e26860 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c0016040 con 0x560cb4e26860 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82bc04c030 con 0x560cb4e26860 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 10 client.16821 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560cb4e35f50 con 0x560cb4e26860 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c0016040 con 0x560cb4e26860 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f82bc03a040 con 0x560cb4e26860 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 20 client.16821 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4ea8db0 0x560cb4eab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 4 client.16821 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.920+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560cb4d960f0 con 0x560cb4ea8db0 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.921+0000 7f82caffd640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4ea8db0 0x560cb4eab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.922+0000 7f82aaffd640 10 client.16821 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.922+0000 7f82caffd640 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4ea8db0 0x560cb4eab1d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef668c02ad64190 server_cookie=6ecd02a9efa17d0a in_seq=0 out_seq=0 2024-09-10T22:53:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.938+0000 7f82aaffd640 1 -- 172.21.15.96:0/910650165 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560cb4d960f0 con 0x560cb4ea8db0 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.938+0000 7f82aaffd640 10 client.16821 handle_command_reply: tid=0 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.938+0000 7f82d4adb740 1 client.16821 shutdown 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 msgr2=0x560cb4e378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006010 tx=0x7f82bc03f000 comp rx=0 tx=0).stop 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 shutdown_connections 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560cb4ea8db0 0x560cb4eab1d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb4e29190 0x560cb4e365e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb4e2bbd0 0x560cb4e36b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 --2- 172.21.15.96:0/910650165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cb4e26860 0x560cb4e378e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 >> 172.21.15.96:0/910650165 conn(0x560cb4e219b0 msgr2=0x560cb4e25c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.939+0000 7f82aaffd640 0 client.16821 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 shutdown_connections 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d4adb740 1 -- 172.21.15.96:0/910650165 wait complete. 2024-09-10T22:53:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d4adb740 20 client.16821 trim_cache size 0 max 16384 2024-09-10T22:53:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:53:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:53:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:53:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 msgr2=0x7f82b40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 0x7f82b40b78f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f82bc031500 tx=0x7f82bc005a20 comp rx=0 tx=0).stop 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.940+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 msgr2=0x7f82cc199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc199fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009fd0 tx=0x7f82c400ef10 comp rx=0 tx=0).stop 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 shutdown_connections 2024-09-10T22:53:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82b40b5430 0x7f82b40b78f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc108680 0x7f82cc199fd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82cc105ad0 0x7f82cc1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 --2- 172.21.15.96:0/2358383216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82cc06bc90 0x7f82cc198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 >> 172.21.15.96:0/2358383216 conn(0x7f82cc0fd130 msgr2=0x7f82cc103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 shutdown_connections 2024-09-10T22:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:53:58.941+0000 7f82d1d0e640 1 -- 172.21.15.96:0/2358383216 wait complete. 2024-09-10T22:53:58.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.974+0000 7f8a864e1640 1 -- 172.21.15.96:0/3953186686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8010a150 msgr2=0x7f8a801111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.974+0000 7f8a864e1640 1 --2- 172.21.15.96:0/3953186686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8010a150 0x7f8a801111b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009ca0 tx=0x7f8a7802a980 comp rx=0 tx=0).stop 2024-09-10T22:53:58.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 -- 172.21.15.96:0/3953186686 shutdown_connections 2024-09-10T22:53:58.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 --2- 172.21.15.96:0/3953186686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a530 0x7f8a8010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 --2- 172.21.15.96:0/3953186686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8011c0a0 0x7f8a8011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 --2- 172.21.15.96:0/3953186686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8010a150 0x7f8a801111b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 -- 172.21.15.96:0/3953186686 >> 172.21.15.96:0/3953186686 conn(0x7f8a8006dfc0 msgr2=0x7f8a8006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 -- 172.21.15.96:0/3953186686 shutdown_connections 2024-09-10T22:53:58.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.975+0000 7f8a864e1640 1 -- 172.21.15.96:0/3953186686 wait complete. 2024-09-10T22:53:58.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.976+0000 7f8a864e1640 1 Processor -- start 2024-09-10T22:53:58.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.976+0000 7f8a864e1640 1 -- start start 2024-09-10T22:53:58.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.977+0000 7f8a864e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 0x7f8a801a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.977+0000 7f8a864e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.978+0000 7f8a864e1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 0x7f8a801ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.978+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.978+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43160/0 (socket says 172.21.15.96:43160) 2024-09-10T22:53:58.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.978+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1084225382 learned_addr learned my addr 172.21.15.96:0/1084225382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.979+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80121bf0 con 0x7f8a8010a530 2024-09-10T22:53:58.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.979+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 0x7f8a801a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.979+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 0x7f8a801ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.979+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80121a70 con 0x7f8a8010a150 2024-09-10T22:53:58.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.979+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80121d70 con 0x7f8a8011c0a0 2024-09-10T22:53:58.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.983+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1084225382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 msgr2=0x7f8a801ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.983+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 0x7f8a801ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.983+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1084225382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 msgr2=0x7f8a801a80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.983+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 0x7f8a801a80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.983+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801ae170 con 0x7f8a8010a530 2024-09-10T22:53:58.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.984+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000d950 tx=0x7f8a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.984+0000 7f8a7d7fa640 1 -- 172.21.15.96:0/1084225382 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70017070 con 0x7f8a8010a530 2024-09-10T22:53:58.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.984+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 0x7f8a801ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:58.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.985+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801ae460 con 0x7f8a8010a530 2024-09-10T22:53:58.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.985+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801b4b20 con 0x7f8a8010a530 2024-09-10T22:53:58.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.985+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 0x7f8a801a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:53:58.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.985+0000 7f8a7d7fa640 1 -- 172.21.15.96:0/1084225382 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70009c30 con 0x7f8a8010a530 2024-09-10T22:53:58.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.985+0000 7f8a7d7fa640 1 -- 172.21.15.96:0/1084225382 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70020060 con 0x7f8a8010a530 2024-09-10T22:53:58.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.989+0000 7f8a888b6740 1 Processor -- start 2024-09-10T22:53:58.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.989+0000 7f8a888b6740 1 -- start start 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a7d7fa640 1 -- 172.21.15.96:0/1084225382 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f8a70010460 con 0x7f8a8010a530 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe38170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a84a7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a84a7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45488/0 (socket says 172.21.15.96:45488) 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55acfbd9fea0 con 0x55acfbe29df0 2024-09-10T22:53:58.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43164/0 (socket says 172.21.15.96:43164) 2024-09-10T22:53:58.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a7ffff640 1 -- 172.21.15.96:0/2846396052 learned_addr learned my addr 172.21.15.96:0/2846396052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:58.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acfbd8f2d0 con 0x55acfbe35d80 2024-09-10T22:53:58.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55acfbd930e0 con 0x55acfbe294d0 2024-09-10T22:53:58.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.990+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe38170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7ffff640 1 -- 172.21.15.96:0/2846396052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 msgr2=0x55acfbe298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7ffff640 1 -- 172.21.15.96:0/2846396052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 msgr2=0x55acfbe38170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe38170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7ffff640 1 -- 172.21.15.96:0/2846396052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfbaed650 con 0x55acfbe29df0 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7d7fa640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 0x7f8a680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.991+0000 7f8a7d7fa640 1 -- 172.21.15.96:0/1084225382 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8a700e5780 con 0x7f8a8010a530 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.992+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.992+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802fbf0 tx=0x7f8a7802fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9982625d9808b088 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.992+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 0x7f8a680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.992+0000 7f8a667fc640 1 -- 172.21.15.96:0/2846396052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78034070 con 0x55acfbe29df0 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.993+0000 7f8a667fc640 1 -- 172.21.15.96:0/2846396052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78003e20 con 0x55acfbe29df0 2024-09-10T22:53:58.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.993+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 0x7f8a680b7820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.994+0000 7f8a667fc640 1 -- 172.21.15.96:0/2846396052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78030a80 con 0x55acfbe29df0 2024-09-10T22:53:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 msgr2=0x55acfbe35840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802fbf0 tx=0x7f8a7802fc20 comp rx=0 tx=0).stop 2024-09-10T22:53:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 shutdown_connections 2024-09-10T22:53:58.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe38170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe35840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 --2- 172.21.15.96:0/2846396052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe294d0 0x55acfbe298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:58.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 >> 172.21.15.96:0/2846396052 conn(0x55acfbe27300 msgr2=0x55acfbe284a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:58.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 shutdown_connections 2024-09-10T22:53:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.995+0000 7f8a888b6740 1 -- 172.21.15.96:0/2846396052 wait complete. 2024-09-10T22:53:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.996+0000 7f8a888b6740 1 -- start start 2024-09-10T22:53:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.996+0000 7f8a888b6740 1 Processor -- start 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe35630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe33040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55acfbbf0700 con 0x55acfbe294d0 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55acfbd2a160 con 0x55acfbe35d80 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55acfbe389f0 con 0x55acfbe29df0 2024-09-10T22:53:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a888b6740 10 client.0 _ll_register_callbacks cb 0x55acfbd439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe35630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45500/0 (socket says 172.21.15.96:45500) 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1021288481 learned_addr learned my addr 172.21.15.96:0/1021288481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.997+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe33040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1021288481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 msgr2=0x55acfbe33040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe33040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1021288481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 msgr2=0x55acfbe35630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe35630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/1021288481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfbd57c90 con 0x55acfbe29df0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe33040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a700a91b0 tx=0x7f8a700ab890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e58af31674ac2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:53:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700ac640 con 0x55acfbe29df0 2024-09-10T22:53:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.998+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8a700e5d80 con 0x55acfbe29df0 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.999+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78003e20 con 0x55acfbe29df0 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.999+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a70009de0 con 0x55acfbe29df0 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.999+0000 7f8a888b6740 10 client.26458 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:58.999+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55acfbe443a0 con 0x55acfbe29df0 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700abf40 con 0x55acfbe29df0 2024-09-10T22:53:59.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8a700930b0 con 0x55acfbe29df0 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 10 client.26458 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 10 client.26458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 20 client.26458 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbe47bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 4 client.26458 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.000+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8a78003e20 con 0x55acfbe457b0 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.001+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbe47bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.001+0000 7f8a5f7fe640 10 client.26458 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.001+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbe47bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=418633c99d4c5e8e server_cookie=3ebc1c2c64cbae03 in_seq=0 out_seq=0 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/1021288481 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8a78003e20 con 0x55acfbe457b0 2024-09-10T22:53:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a5f7fe640 10 client.26458 handle_command_reply: tid=0 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 client.26458 shutdown 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 msgr2=0x55acfbe32a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a700a91b0 tx=0x7f8a700ab890 comp rx=0 tx=0).stop 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 shutdown_connections 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbe47bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe35d80 0x55acfbe33040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe29df0 0x55acfbe32a80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a5f7fe640 0 client.26458 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 --2- 172.21.15.96:0/1021288481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe35630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 >> 172.21.15.96:0/1021288481 conn(0x55acfbe27300 msgr2=0x55acfbe2ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 shutdown_connections 2024-09-10T22:53:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 1 -- 172.21.15.96:0/1021288481 wait complete. 2024-09-10T22:53:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.002+0000 7f8a888b6740 20 client.26458 trim_cache size 0 max 16384 2024-09-10T22:53:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.019+0000 7f8a888b6740 1 Processor -- start 2024-09-10T22:53:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.019+0000 7f8a888b6740 1 -- start start 2024-09-10T22:53:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 0x55acfbe31300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe31840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00e2f0 con 0x55acfbe294d0 2024-09-10T22:53:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00fe30 con 0x55acfbe29df0 2024-09-10T22:53:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a888b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00e9c0 con 0x55acfbe35d80 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe26c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43168/0 (socket says 172.21.15.96:43168) 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7ffff640 1 -- 172.21.15.96:0/395196805 learned_addr learned my addr 172.21.15.96:0/395196805 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 0x55acfbe31300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe31840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/395196805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 msgr2=0x55acfbe31840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe31840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/395196805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 msgr2=0x55acfbe26c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe26c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 -- 172.21.15.96:0/395196805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfbe333b0 con 0x55acfbe29df0 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.020+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 0x55acfbe31300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a700abed0 tx=0x7f8a700ab9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15d5309667b54e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.021+0000 7f8a65ffb640 1 -- 172.21.15.96:0/395196805 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a700abf40 con 0x55acfbe29df0 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.021+0000 7f8a65ffb640 1 -- 172.21.15.96:0/395196805 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a700b0a20 con 0x55acfbe29df0 2024-09-10T22:53:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.021+0000 7f8a65ffb640 1 -- 172.21.15.96:0/395196805 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70093110 con 0x55acfbe29df0 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- 172.21.15.96:0/395196805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 msgr2=0x55acfbe31300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 0x55acfbe31300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a700abed0 tx=0x7f8a700ab9c0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- 172.21.15.96:0/395196805 shutdown_connections 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe29df0 0x55acfbe31300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe31840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 --2- 172.21.15.96:0/395196805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe294d0 0x55acfbe26c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- 172.21.15.96:0/395196805 >> 172.21.15.96:0/395196805 conn(0x55acfbe27300 msgr2=0x55acfbe2dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- 172.21.15.96:0/395196805 shutdown_connections 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- 172.21.15.96:0/395196805 wait complete. 2024-09-10T22:53:59.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.022+0000 7f8a888b6740 1 -- start start 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.023+0000 7f8a888b6740 1 Processor -- start 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.023+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.023+0000 7f8a888b6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe3a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56616/0 (socket says 172.21.15.96:56616) 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a7ffff640 1 -- 172.21.15.96:0/3471234438 learned_addr learned my addr 172.21.15.96:0/3471234438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe3a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00f760 con 0x55acfbe29df0 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00f090 con 0x55acfbe294d0 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c00c2d0 con 0x55acfbe35d80 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a84a7e640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.024+0000 7f8a888b6740 10 client.0 _ll_register_callbacks cb 0x7f8a700a7de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 -- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 msgr2=0x55acfbe3a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe3a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 -- 172.21.15.96:0/3471234438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 msgr2=0x55acfbe3a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe3a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfbeb1920 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a7ffff640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802fbf0 tx=0x7f8a78003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4f1283af56aef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78034040 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8a7802f3d0 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a700b0a20 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a7805e050 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a888b6740 10 client.26423 fetch_fsmap learned FSMap version 32 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.025+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55acfbeb2c80 con 0x55acfbe294d0 2024-09-10T22:53:59.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78009b20 con 0x55acfbe294d0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8a7804e040 con 0x55acfbe294d0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 10 client.26423 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 10 client.26423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 20 client.26423 populate_metadata read hostname 'smithi096' 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbeb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 4 client.26423 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8a700b0a20 con 0x55acfbe457b0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbeb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a5f7fe640 10 client.26423 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.026+0000 7f8a7f7fe640 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbeb50a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8352b80e4a1ab04b server_cookie=b4500f5617003701 in_seq=0 out_seq=0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a5f7fe640 1 -- 172.21.15.96:0/3471234438 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8a700b0a20 con 0x55acfbe457b0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a5f7fe640 10 client.26423 handle_command_reply: tid=0 2024-09-10T22:53:59.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 client.26423 shutdown 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 msgr2=0x55acfbe39e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802fbf0 tx=0x7f8a78003ba0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 shutdown_connections 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acfbe35d80 0x55acfbe3a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55acfbe29df0 0x55acfbe3a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55acfbe457b0 0x55acfbeb50a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a5f7fe640 0 client.26423 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:53:59.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 --2- 172.21.15.96:0/3471234438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55acfbe294d0 0x55acfbe39e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 >> 172.21.15.96:0/3471234438 conn(0x55acfbe27300 msgr2=0x55acfbe340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:59.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 shutdown_connections 2024-09-10T22:53:59.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 1 -- 172.21.15.96:0/3471234438 wait complete. 2024-09-10T22:53:59.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.027+0000 7f8a888b6740 20 client.26423 trim_cache size 0 max 16384 2024-09-10T22:53:59.036 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 msgr2=0x7f8a680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 0x7f8a680b7820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74008040 comp rx=0 tx=0).stop 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 msgr2=0x7f8a801a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000d950 tx=0x7f8a7000de10 comp rx=0 tx=0).stop 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 shutdown_connections 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8a680b5360 0x7f8a680b7820 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a8011c0a0 0x7f8a801ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8010a530 0x7f8a801a8600 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 --2- 172.21.15.96:0/1084225382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a8010a150 0x7f8a801a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:53:59.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 >> 172.21.15.96:0/1084225382 conn(0x7f8a8006dfc0 msgr2=0x7f8a8011c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:53:59.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 shutdown_connections 2024-09-10T22:53:59.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:53:59.028+0000 7f8a864e1640 1 -- 172.21.15.96:0/1084225382 wait complete. 2024-09-10T22:53:59.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:53:59 smithi096 ceph-mon[30869]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 681 op/s 2024-09-10T22:53:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:53:59 smithi121 ceph-mon[35119]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 681 op/s 2024-09-10T22:53:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:53:59 smithi117 ceph-mon[35081]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 681 op/s 2024-09-10T22:53:59.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:54:00.363 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 385.0 s, 2363.8 tps, lat 42.232 ms stddev 44.099 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 -- 172.21.15.96:0/2020345067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178106ed0 msgr2=0x7fe178107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 --2- 172.21.15.96:0/2020345067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178106ed0 0x7fe178107320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe170009be0 tx=0x7fe17002a6c0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 -- 172.21.15.96:0/2020345067 shutdown_connections 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 --2- 172.21.15.96:0/2020345067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe178106ed0 0x7fe178107320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 --2- 172.21.15.96:0/2020345067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1781023b0 0x7fe178106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 --2- 172.21.15.96:0/2020345067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178101a90 0x7fe178101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 -- 172.21.15.96:0/2020345067 >> 172.21.15.96:0/2020345067 conn(0x7fe1780fd700 msgr2=0x7fe1780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 -- 172.21.15.96:0/2020345067 shutdown_connections 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.216+0000 7fe17df27640 1 -- 172.21.15.96:0/2020345067 wait complete. 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 Processor -- start 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 -- start start 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178106ed0 0x7fe1781a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe178119350 con 0x7fe178106ed0 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781191d0 con 0x7fe1781023b0 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe17df27640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781194d0 con 0x7fe178101a90 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.217+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45552/0 (socket says 172.21.15.96:45552) 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56636/0 (socket says 172.21.15.96:56636) 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 -- 172.21.15.96:0/1588338553 learned_addr learned my addr 172.21.15.96:0/1588338553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe177fff640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178106ed0 0x7fe1781a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 -- 172.21.15.96:0/1588338553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 msgr2=0x7fe17819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 -- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178106ed0 msgr2=0x7fe1781a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178106ed0 0x7fe1781a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 -- 172.21.15.96:0/1588338553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1781a5910 con 0x7fe178101a90 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe176ffd640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe1777fe640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9c0 tx=0x7fe16800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe174ff9640 1 -- 172.21.15.96:0/1588338553 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800ccc0 con 0x7fe178101a90 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe174ff9640 1 -- 172.21.15.96:0/1588338553 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1680054d0 con 0x7fe178101a90 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1781a5c00 con 0x7fe178101a90 2024-09-10T22:54:01.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe174ff9640 1 -- 172.21.15.96:0/1588338553 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168010430 con 0x7fe178101a90 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.218+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781ac2c0 con 0x7fe178101a90 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.222+0000 7fe174ff9640 1 -- 172.21.15.96:0/1588338553 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe168004f80 con 0x7fe178101a90 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe1802fc740 1 Processor -- start 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe1802fc740 1 -- start start 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe174ff9640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 0x7fe1600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.223+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe1802fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55560063dea0 con 0x5556006c7a10 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56640/0 (socket says 172.21.15.96:56640) 2024-09-10T22:54:01.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 -- 172.21.15.96:0/1847359270 learned_addr learned my addr 172.21.15.96:0/1847359270 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55560062d2d0 con 0x5556006c70f0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5556006310e0 con 0x5556006d39a0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe176ffd640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 0x7fe1600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe174ff9640 1 -- 172.21.15.96:0/1588338553 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe1680e5f20 con 0x7fe178101a90 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 -- 172.21.15.96:0/1847359270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 msgr2=0x5556006d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 -- 172.21.15.96:0/1847359270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 msgr2=0x5556006d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 -- 172.21.15.96:0/1847359270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560038b650 con 0x5556006c70f0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe176ffd640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 0x7fe1600b77a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe178100b20 tx=0x7fe16c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe177fff640 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe170009330 tx=0x7fe1700092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2add589ca8546201 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe15dffb640 1 -- 172.21.15.96:0/1847359270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036040 con 0x5556006c70f0 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe15dffb640 1 -- 172.21.15.96:0/1847359270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170031440 con 0x5556006c70f0 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.224+0000 7fe15dffb640 1 -- 172.21.15.96:0/1847359270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17003c680 con 0x5556006c70f0 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 msgr2=0x5556006c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe170009330 tx=0x7fe1700092c0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 shutdown_connections 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 --2- 172.21.15.96:0/1847359270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006c74d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 >> 172.21.15.96:0/1847359270 conn(0x5556006c4f20 msgr2=0x5556006c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 shutdown_connections 2024-09-10T22:54:01.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.226+0000 7fe1802fc740 1 -- 172.21.15.96:0/1847359270 wait complete. 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.227+0000 7fe1802fc740 1 -- start start 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.227+0000 7fe1802fc740 1 Processor -- start 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.227+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.227+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 0x5556006d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.227+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006d39a0 0x5556006d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1802fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55560048e700 con 0x5556006d39a0 2024-09-10T22:54:01.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1802fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5556005c8160 con 0x5556006c7a10 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1802fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5556006d65c0 con 0x5556006c70f0 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 0x5556006d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1802fc740 10 client.0 _ll_register_callbacks cb 0x5556005e19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006d3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45578/0 (socket says 172.21.15.96:45578) 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1777fe640 1 -- 172.21.15.96:0/4095510608 learned_addr learned my addr 172.21.15.96:0/4095510608 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:01.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe177fff640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006d39a0 0x5556006d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 -- 172.21.15.96:0/4095510608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 msgr2=0x5556006d3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006d3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 -- 172.21.15.96:0/4095510608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006d39a0 msgr2=0x5556006d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006d39a0 0x5556006d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 -- 172.21.15.96:0/4095510608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556005f5c90 con 0x5556006c7a10 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe176ffd640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 0x5556006d07a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe17806b950 tx=0x7fe16c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5aa21ba1272b54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:01.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c01d910 con 0x5556006c7a10 2024-09-10T22:54:01.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe16c01dab0 con 0x5556006c7a10 2024-09-10T22:54:01.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.228+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe170031440 con 0x5556006c7a10 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.229+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe16c01dcf0 con 0x5556006c7a10 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.229+0000 7fe1802fc740 10 client.26435 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.229+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5556006e2030 con 0x5556006c7a10 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.229+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170031440 con 0x5556006c7a10 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe16c01b530 con 0x5556006c7a10 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 10 client.26435 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 20 client.26435 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556006e3540 0x5556006e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 4 client.26435 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55560063fdf0 con 0x5556006e3540 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.230+0000 7fe1777fe640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556006e3540 0x5556006e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.231+0000 7fe1777fe640 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556006e3540 0x5556006e5930 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19479d6d7442e5bc server_cookie=569b6af110a79c19 in_seq=0 out_seq=0 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.231+0000 7fe156ffd640 10 client.26435 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.231+0000 7fe156ffd640 1 -- 172.21.15.96:0/4095510608 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55560063fdf0 con 0x5556006e3540 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.231+0000 7fe156ffd640 10 client.26435 handle_command_reply: tid=0 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.231+0000 7fe1802fc740 1 client.26435 shutdown 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 msgr2=0x5556006d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 0x5556006d07a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe17806b950 tx=0x7fe16c004730 comp rx=0 tx=0).stop 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 shutdown_connections 2024-09-10T22:54:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556006e3540 0x5556006e5930 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c7a10 0x5556006d07a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006d39a0 0x5556006d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 --2- 172.21.15.96:0/4095510608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006d3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe156ffd640 0 client.26435 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 >> 172.21.15.96:0/4095510608 conn(0x5556006c4f20 msgr2=0x5556006caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 shutdown_connections 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 1 -- 172.21.15.96:0/4095510608 wait complete. 2024-09-10T22:54:01.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.232+0000 7fe1802fc740 20 client.26435 trim_cache size 0 max 16384 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.250+0000 7fe1802fc740 1 Processor -- start 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 -- start start 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b4a00 con 0x5556006c7a10 2024-09-10T22:54:01.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b6540 con 0x5556006d39a0 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe1802fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b50d0 con 0x5556006c70f0 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56658/0 (socket says 172.21.15.96:56658) 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.251+0000 7fe177fff640 1 -- 172.21.15.96:0/379968452 learned_addr learned my addr 172.21.15.96:0/379968452 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe176ffd640 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe1777fe640 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 -- 172.21.15.96:0/379968452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 msgr2=0x5556006c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 -- 172.21.15.96:0/379968452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 msgr2=0x5556006df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 -- 172.21.15.96:0/379968452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556006d36c0 con 0x5556006d39a0 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe177fff640 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe17002abe0 tx=0x7fe17002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed1c346ff929872 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe15d7fa640 1 -- 172.21.15.96:0/379968452 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170050070 con 0x5556006d39a0 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe15d7fa640 1 -- 172.21.15.96:0/379968452 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170030790 con 0x5556006d39a0 2024-09-10T22:54:01.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.252+0000 7fe15d7fa640 1 -- 172.21.15.96:0/379968452 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036040 con 0x5556006d39a0 2024-09-10T22:54:01.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 -- 172.21.15.96:0/379968452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 msgr2=0x5556006c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe17002abe0 tx=0x7fe17002a6c0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 -- 172.21.15.96:0/379968452 shutdown_connections 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006d39a0 0x5556006c3e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 --2- 172.21.15.96:0/379968452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006c70f0 0x5556006c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 -- 172.21.15.96:0/379968452 >> 172.21.15.96:0/379968452 conn(0x5556006c4f20 msgr2=0x5556006c3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 -- 172.21.15.96:0/379968452 shutdown_connections 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.253+0000 7fe1802fc740 1 -- 172.21.15.96:0/379968452 wait complete. 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 -- start start 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 Processor -- start 2024-09-10T22:54:01.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b5e70 con 0x5556006c7a10 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b57a0 con 0x5556006c70f0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1802fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781b29e0 con 0x5556006d39a0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43222/0 (socket says 172.21.15.96:43222) 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe176ffd640 1 -- 172.21.15.96:0/2236292409 learned_addr learned my addr 172.21.15.96:0/2236292409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe1777fe640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.254+0000 7fe177fff640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe1802fc740 10 client.0 _ll_register_callbacks cb 0x7fe16c005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe177fff640 1 -- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 msgr2=0x5556006dbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe177fff640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006dbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe177fff640 1 -- 172.21.15.96:0/2236292409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 msgr2=0x5556006dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe177fff640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe177fff640 1 -- 172.21.15.96:0/2236292409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556006d70e0 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.255+0000 7fe176ffd640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.256+0000 7fe177fff640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d68b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe170004af0 tx=0x7fe1700303a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=178c704aac0f307d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.256+0000 7fe156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.256+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036070 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.256+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe170030d80 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.257+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe170030790 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.257+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe170050040 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.257+0000 7fe1802fc740 10 client.26482 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.257+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555600750f90 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.257+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170030790 con 0x5556006d39a0 2024-09-10T22:54:01.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe17004c440 con 0x5556006d39a0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 10 client.26482 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 20 client.26482 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556007511c0 0x5556007535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 4 client.26482 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55560063b8a0 con 0x5556007511c0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1777fe640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556007511c0 0x5556007535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe156ffd640 10 client.26482 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.258+0000 7fe1777fe640 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556007511c0 0x5556007535b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1faee983addeb2 server_cookie=aabec352f91aab53 in_seq=0 out_seq=0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe156ffd640 1 -- 172.21.15.96:0/2236292409 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55560063b8a0 con 0x5556007511c0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe156ffd640 10 client.26482 handle_command_reply: tid=0 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 client.26482 shutdown 2024-09-10T22:54:01.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 msgr2=0x5556006d68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d68b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe170004af0 tx=0x7fe1700303a0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 shutdown_connections 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5556007511c0 0x5556007535b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556006d39a0 0x5556006d68b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe156ffd640 0 client.26482 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556006c7a10 0x5556006dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 --2- 172.21.15.96:0/2236292409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556006c70f0 0x5556006dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.259+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 >> 172.21.15.96:0/2236292409 conn(0x5556006c4f20 msgr2=0x5556006d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.260+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 shutdown_connections 2024-09-10T22:54:01.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.260+0000 7fe1802fc740 1 -- 172.21.15.96:0/2236292409 wait complete. 2024-09-10T22:54:01.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.264+0000 7fe1802fc740 20 client.26482 trim_cache size 0 max 16384 2024-09-10T22:54:01.272 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 msgr2=0x7fe1600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 0x7fe1600b77a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe178100b20 tx=0x7fe16c005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 msgr2=0x7fe17819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9c0 tx=0x7fe16800ee80 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 shutdown_connections 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1781023b0 0x7fe17819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe1600b52e0 0x7fe1600b77a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe178106ed0 0x7fe1781a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 --2- 172.21.15.96:0/1588338553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178101a90 0x7fe17819f860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:01.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.278+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 >> 172.21.15.96:0/1588338553 conn(0x7fe1780fd700 msgr2=0x7fe178104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.280+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 shutdown_connections 2024-09-10T22:54:01.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:01.281+0000 7fe17df27640 1 -- 172.21.15.96:0/1588338553 wait complete. 2024-09-10T22:54:01.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:01 smithi096 ceph-mon[30869]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 887 op/s 2024-09-10T22:54:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:01 smithi121 ceph-mon[35119]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 887 op/s 2024-09-10T22:54:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:01 smithi117 ceph-mon[35081]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 887 op/s 2024-09-10T22:54:03.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:54:03.087 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:54:03.458 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:03.586 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:03 smithi096 ceph-mon[30869]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-10T22:54:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:03 smithi121 ceph-mon[35119]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-10T22:54:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:03 smithi117 ceph-mon[35081]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 -- 172.21.15.96:0/1730460693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c1018b0 msgr2=0x7f246c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 --2- 172.21.15.96:0/1730460693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c1018b0 0x7f246c101c90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2454009900 tx=0x7f245402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 -- 172.21.15.96:0/1730460693 shutdown_connections 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 --2- 172.21.15.96:0/1730460693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c107020 0x7f246c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 --2- 172.21.15.96:0/1730460693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1066a0 0x7f246c106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:03.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 --2- 172.21.15.96:0/1730460693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c1018b0 0x7f246c101c90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:03.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 -- 172.21.15.96:0/1730460693 >> 172.21.15.96:0/1730460693 conn(0x7f246c076e60 msgr2=0x7f246c077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:03.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 -- 172.21.15.96:0/1730460693 shutdown_connections 2024-09-10T22:54:03.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.994+0000 7f2470e31640 1 -- 172.21.15.96:0/1730460693 wait complete. 2024-09-10T22:54:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.995+0000 7f2470e31640 1 Processor -- start 2024-09-10T22:54:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.995+0000 7f2470e31640 1 -- start start 2024-09-10T22:54:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 0x7f246c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 0x7f246c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1193e0 con 0x7f246c107020 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f246c119260 con 0x7f246c1066a0 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f246c119560 con 0x7f246c1018b0 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 0x7f246c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43244/0 (socket says 172.21.15.96:43244) 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246ad9d640 1 -- 172.21.15.96:0/2461071350 learned_addr learned my addr 172.21.15.96:0/2461071350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246a59c640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 0x7f246c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 -- 172.21.15.96:0/2461071350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 msgr2=0x7f246c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 0x7f246c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 -- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 msgr2=0x7f246c1a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:03.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 -- 172.21.15.96:0/2461071350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c1a59c0 con 0x7f246c1066a0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f246a59c640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 0x7f246c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 0x7f246c19fe50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f245c009fd0 tx=0x7f245c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2463fff640 1 -- 172.21.15.96:0/2461071350 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004b20 con 0x7f246c1066a0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2463fff640 1 -- 172.21.15.96:0/2461071350 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c004cc0 con 0x7f246c1066a0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.996+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c1a5cb0 con 0x7f246c1066a0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.997+0000 7f2463fff640 1 -- 172.21.15.96:0/2461071350 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c010720 con 0x7f246c1066a0 2024-09-10T22:54:04.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:03.997+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c1ac370 con 0x7f246c1066a0 2024-09-10T22:54:04.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.000+0000 7f2473206740 1 Processor -- start 2024-09-10T22:54:04.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.000+0000 7f2473206740 1 -- start start 2024-09-10T22:54:04.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55edeb949e60 con 0x55edeb9df790 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55edeb939290 con 0x55edeb9d2ee0 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.001+0000 7f2473206740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55edeb93d0a0 con 0x55edeb9d3800 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43252/0 (socket says 172.21.15.96:43252) 2024-09-10T22:54:04.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 -- 172.21.15.96:0/2171784964 learned_addr learned my addr 172.21.15.96:0/2171784964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 -- 172.21.15.96:0/2171784964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 msgr2=0x55edeb9df250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9df250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 -- 172.21.15.96:0/2171784964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 msgr2=0x55edeb9d32c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9d32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.002+0000 7f2461d9b640 1 -- 172.21.15.96:0/2171784964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edeb697930 con 0x55edeb9df790 2024-09-10T22:54:04.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.003+0000 7f2461d9b640 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ee5b0 tx=0x7f245c0ee6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c8d20a3d8a8701 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.010+0000 7f2460d99640 1 -- 172.21.15.96:0/2171784964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0e8520 con 0x55edeb9df790 2024-09-10T22:54:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.010+0000 7f2460d99640 1 -- 172.21.15.96:0/2171784964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c0e6cc0 con 0x55edeb9df790 2024-09-10T22:54:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.010+0000 7f2460d99640 1 -- 172.21.15.96:0/2171784964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0f5b50 con 0x55edeb9df790 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.011+0000 7f2463fff640 1 -- 172.21.15.96:0/2461071350 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f245c005020 con 0x7f246c1066a0 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.012+0000 7f2463fff640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 0x7f24580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.012+0000 7f2463fff640 1 -- 172.21.15.96:0/2461071350 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f245c0e63e0 con 0x7f246c1066a0 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.014+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 0x7f24580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.015+0000 7f2473206740 1 -- 172.21.15.96:0/2171784964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 msgr2=0x55edeb9e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.015+0000 7f2473206740 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ee5b0 tx=0x7f245c0ee6a0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 0x7f24580b7820 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f246400b420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f2473206740 1 -- 172.21.15.96:0/2171784964 shutdown_connections 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f2473206740 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9e1b80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f2473206740 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9df250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f2473206740 1 --2- 172.21.15.96:0/2171784964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.022+0000 7f2473206740 1 -- 172.21.15.96:0/2171784964 >> 172.21.15.96:0/2171784964 conn(0x55edeb9d0d10 msgr2=0x55edeb9d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:04.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.025+0000 7f2473206740 1 -- 172.21.15.96:0/2171784964 shutdown_connections 2024-09-10T22:54:04.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.025+0000 7f2473206740 1 -- 172.21.15.96:0/2171784964 wait complete. 2024-09-10T22:54:04.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.025+0000 7f2473206740 1 -- start start 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.025+0000 7f2473206740 1 Processor -- start 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9dcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55edeb79a890 con 0x55edeb9df790 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55edeb8d3d40 con 0x55edeb9d2ee0 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55edeb9e2440 con 0x55edeb9d3800 2024-09-10T22:54:04.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.027+0000 7f2473206740 10 client.0 _ll_register_callbacks cb 0x55edeb937c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:04.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43258/0 (socket says 172.21.15.96:43258) 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 -- 172.21.15.96:0/2421220828 learned_addr learned my addr 172.21.15.96:0/2421220828 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 -- 172.21.15.96:0/2421220828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 msgr2=0x55edeb9dcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9dcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 -- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 msgr2=0x55edeb9dc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.030+0000 7f246ad9d640 1 -- 172.21.15.96:0/2421220828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edeb9fc900 con 0x55edeb9df790 2024-09-10T22:54:04.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.031+0000 7f246ad9d640 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2464011010 tx=0x7f2464004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196b992ee87db050 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.034+0000 7f2451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:04.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.034+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246401cca0 con 0x55edeb9df790 2024-09-10T22:54:04.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.034+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245c0e6cc0 con 0x55edeb9df790 2024-09-10T22:54:04.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.034+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2464004030 con 0x55edeb9df790 2024-09-10T22:54:04.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.039+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f246401ce40 con 0x55edeb9df790 2024-09-10T22:54:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.039+0000 7f2473206740 10 client.16866 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:04.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.039+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edeb9ff310 con 0x55edeb9df790 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c0e6cc0 con 0x55edeb9df790 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2464026420 con 0x55edeb9df790 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 10 client.16866 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 20 client.16866 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba00900 0x55edeba02dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 4 client.16866 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.041+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edeb94bdb0 con 0x55edeba00900 2024-09-10T22:54:04.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.042+0000 7f246a59c640 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba00900 0x55edeba02dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.051+0000 7f246a59c640 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba00900 0x55edeba02dc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c471365ebb03e3 server_cookie=2b271e8cfdf1ab52 in_seq=0 out_seq=0 2024-09-10T22:54:04.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.051+0000 7f2451ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.052+0000 7f2451ffb640 1 -- 172.21.15.96:0/2421220828 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55edeb94bdb0 con 0x55edeba00900 2024-09-10T22:54:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.052+0000 7f2451ffb640 10 client.16866 handle_command_reply: tid=0 2024-09-10T22:54:04.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.052+0000 7f2473206740 1 client.16866 shutdown 2024-09-10T22:54:04.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.055+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 msgr2=0x55edeb9dd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.055+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2464011010 tx=0x7f2464004730 comp rx=0 tx=0).stop 2024-09-10T22:54:04.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 shutdown_connections 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba00900 0x55edeba02dc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9df790 0x55edeb9dd2b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9d3800 0x55edeb9dcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 --2- 172.21.15.96:0/2421220828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 >> 172.21.15.96:0/2421220828 conn(0x55edeb9d0d10 msgr2=0x55edeb9cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2451ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 shutdown_connections 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 1 -- 172.21.15.96:0/2421220828 wait complete. 2024-09-10T22:54:04.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.056+0000 7f2473206740 20 client.16866 trim_cache size 0 max 16384 2024-09-10T22:54:04.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.088+0000 7f2473206740 1 Processor -- start 2024-09-10T22:54:04.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.088+0000 7f2473206740 1 -- start start 2024-09-10T22:54:04.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9d9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f246a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56718/0 (socket says 172.21.15.96:56718) 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f246a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43268/0 (socket says 172.21.15.96:43268) 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2461d9b640 1 -- 172.21.15.96:0/733392794 learned_addr learned my addr 172.21.15.96:0/733392794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f246c074f80 con 0x55edeb9d2ee0 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1add40 con 0x55edeb9d3800 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.089+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1ac8d0 con 0x55edeb9df790 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f246ad9d640 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9d9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 -- 172.21.15.96:0/733392794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 msgr2=0x55edeb9d9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9d9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 -- 172.21.15.96:0/733392794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 msgr2=0x55edeb9dcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 -- 172.21.15.96:0/733392794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edeb9d1c00 con 0x55edeb9d3800 2024-09-10T22:54:04.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2461d9b640 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f245c0a9170 tx=0x7f245c0e9990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4f88da7605bf67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.090+0000 7f2453fff640 1 -- 172.21.15.96:0/733392794 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0a8a40 con 0x55edeb9d3800 2024-09-10T22:54:04.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.091+0000 7f2453fff640 1 -- 172.21.15.96:0/733392794 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c0e9cb0 con 0x55edeb9d3800 2024-09-10T22:54:04.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.091+0000 7f2453fff640 1 -- 172.21.15.96:0/733392794 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0ac310 con 0x55edeb9d3800 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 msgr2=0x55edeb9d9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f245c0a9170 tx=0x7f245c0e9990 comp rx=0 tx=0).stop 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 shutdown_connections 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9d9a40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9d9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- 172.21.15.96:0/733392794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9dcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 >> 172.21.15.96:0/733392794 conn(0x55edeb9d0d10 msgr2=0x55edeb9dc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 shutdown_connections 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- 172.21.15.96:0/733392794 wait complete. 2024-09-10T22:54:04.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 -- start start 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 Processor -- start 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.092+0000 7f2473206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2473206740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2473206740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1ad670 con 0x55edeb9d2ee0 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2473206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1acfa0 con 0x55edeb9d3800 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2473206740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1a0a40 con 0x55edeb9df790 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2473206740 10 client.0 _ll_register_callbacks cb 0x7f246400bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f246a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f246ad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45630/0 (socket says 172.21.15.96:45630) 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f246ad9d640 1 -- 172.21.15.96:0/1776851866 learned_addr learned my addr 172.21.15.96:0/1776851866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f246a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43272/0 (socket says 172.21.15.96:43272) 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 -- 172.21.15.96:0/1776851866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 msgr2=0x55edeb9f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 -- 172.21.15.96:0/1776851866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 msgr2=0x55edeb9f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 -- 172.21.15.96:0/1776851866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edeb9f4370 con 0x55edeb9d3800 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2461d9b640 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9f3720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55edeb9d2e00 tx=0x7f245c0e9a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8726e8a1fa8420c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2451ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.093+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0ac4f0 con 0x55edeb9d3800 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.094+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f245c0ace00 con 0x55edeb9d3800 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.094+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245c0e9cb0 con 0x55edeb9d3800 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.094+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f245c0ac970 con 0x55edeb9d3800 2024-09-10T22:54:04.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.094+0000 7f2473206740 10 client.26453 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:04.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.094+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edeba6e850 con 0x55edeb9d3800 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c0e9cb0 con 0x55edeb9d3800 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f245c0b5ca0 con 0x55edeb9d3800 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 10 client.26453 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 20 client.26453 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba6ea80 0x55edeba70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 4 client.26453 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.095+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55edeb947860 con 0x55edeba6ea80 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.096+0000 7f246a59c640 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba6ea80 0x55edeba70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.096+0000 7f2451ffb640 10 client.26453 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:04.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.096+0000 7f246a59c640 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba6ea80 0x55edeba70e70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90048459f138131 server_cookie=d4cd917534de973d in_seq=0 out_seq=0 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2451ffb640 1 -- 172.21.15.96:0/1776851866 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55edeb947860 con 0x55edeba6ea80 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2451ffb640 10 client.26453 handle_command_reply: tid=0 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 client.26453 shutdown 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 msgr2=0x55edeb9f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9f3720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55edeb9d2e00 tx=0x7f245c0e9a00 comp rx=0 tx=0).stop 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 shutdown_connections 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55edeba6ea80 0x55edeba70e70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edeb9d3800 0x55edeb9f3720 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edeb9df790 0x55edeb9f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2451ffb640 0 client.26453 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.098+0000 7f2473206740 1 --2- 172.21.15.96:0/1776851866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edeb9d2ee0 0x55edeb9f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 >> 172.21.15.96:0/1776851866 conn(0x55edeb9d0d10 msgr2=0x55edeb9de680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:04.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 shutdown_connections 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2473206740 1 -- 172.21.15.96:0/1776851866 wait complete. 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2473206740 20 client.26453 trim_cache size 0 max 16384 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "6f5238a0-8988-48e6-bc63-8f6795c3cce5", 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:54:04.104 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 msgr2=0x7f24580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 0x7f24580b7820 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f246400b420 comp rx=0 tx=0).stop 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 msgr2=0x7f246c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.099+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 0x7f246c19fe50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f245c009fd0 tx=0x7f245c00ef90 comp rx=0 tx=0).stop 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 shutdown_connections 2024-09-10T22:54:04.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f246c1066a0 0x7f246c19fe50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24580b5360 0x7f24580b7820 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f246c107020 0x7f246c1a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 --2- 172.21.15.96:0/2461071350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f246c1018b0 0x7f246c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 >> 172.21.15.96:0/2461071350 conn(0x7f246c076e60 msgr2=0x7f246c104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 shutdown_connections 2024-09-10T22:54:04.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:04.100+0000 7f2470e31640 1 -- 172.21.15.96:0/2461071350 wait complete. 2024-09-10T22:54:04.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:05.319 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:05.599 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 699 op/s 2024-09-10T22:54:05.599 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:54:05.599 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:54:05.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:54:05.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:05 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 699 op/s 2024-09-10T22:54:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:54:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:54:05.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:54:05.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:05 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 699 op/s 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:54:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:05 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 -- 172.21.15.96:0/1961064567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea8102de0 msgr2=0x7faea81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 --2- 172.21.15.96:0/1961064567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea8102de0 0x7faea81031c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faea4009be0 tx=0x7faea402a6d0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 -- 172.21.15.96:0/1961064567 shutdown_connections 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 --2- 172.21.15.96:0/1961064567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea81087f0 0x7faea8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 --2- 172.21.15.96:0/1961064567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 --2- 172.21.15.96:0/1961064567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea8102de0 0x7faea81031c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 -- 172.21.15.96:0/1961064567 >> 172.21.15.96:0/1961064567 conn(0x7faea80fe830 msgr2=0x7faea8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 -- 172.21.15.96:0/1961064567 shutdown_connections 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.844+0000 7faeaf460640 1 -- 172.21.15.96:0/1961064567 wait complete. 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 Processor -- start 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 -- start start 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 0x7faea819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 0x7faea819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faea8075830 con 0x7faea8109170 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faea80756b0 con 0x7faea8102de0 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeaf460640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faea80759b0 con 0x7faea81087f0 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faead9fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faeac9fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 0x7faea819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faead1fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 0x7faea819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faead9fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43300/0 (socket says 172.21.15.96:43300) 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.845+0000 7faead9fd640 1 -- 172.21.15.96:0/1035275413 learned_addr learned my addr 172.21.15.96:0/1035275413 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 -- 172.21.15.96:0/1035275413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 msgr2=0x7faea819c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 0x7faea819c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 -- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 msgr2=0x7faea819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 -- 172.21.15.96:0/1035275413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea819e350 con 0x7faea8102de0 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faeac9fb640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 0x7faea819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead9fd640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faead1fc640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 0x7faea819bc70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faea402acc0 tx=0x7faea4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7fae9e7fc640 1 -- 172.21.15.96:0/1035275413 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4041020 con 0x7faea8102de0 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7fae9e7fc640 1 -- 172.21.15.96:0/1035275413 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4030970 con 0x7faea8102de0 2024-09-10T22:54:05.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea819e5e0 con 0x7faea8102de0 2024-09-10T22:54:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7fae9e7fc640 1 -- 172.21.15.96:0/1035275413 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4036040 con 0x7faea8102de0 2024-09-10T22:54:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.846+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a7f90 con 0x7faea8102de0 2024-09-10T22:54:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.850+0000 7faeb1835740 1 Processor -- start 2024-09-10T22:54:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.850+0000 7faeb1835740 1 -- start start 2024-09-10T22:54:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.850+0000 7fae9e7fc640 1 -- 172.21.15.96:0/1035275413 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7faea40043c0 con 0x7faea8102de0 2024-09-10T22:54:05.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a761860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55618a6d7e50 con 0x55618a761da0 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55618a6c7280 con 0x55618a761480 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeb1835740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55618a6cb090 con 0x55618a76dd30 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faead9fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a761860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faead9fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a761860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56752/0 (socket says 172.21.15.96:56752) 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faead9fd640 1 -- 172.21.15.96:0/25782500 learned_addr learned my addr 172.21.15.96:0/25782500 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeac9fb640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faead1fc640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeac9fb640 1 -- 172.21.15.96:0/25782500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 msgr2=0x55618a761860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeac9fb640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a761860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.851+0000 7faeac9fb640 1 -- 172.21.15.96:0/25782500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 msgr2=0x55618a76d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7faeac9fb640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7faeac9fb640 1 -- 172.21.15.96:0/25782500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618a425650 con 0x55618a76dd30 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7fae9e7fc640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7faead1fc640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7faead9fd640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7faeac9fb640 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a770120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faea810fe80 tx=0x7fae980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a97954041339043 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7fae9e7fc640 1 -- 172.21.15.96:0/1035275413 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faea4106940 con 0x7faea8102de0 2024-09-10T22:54:05.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.852+0000 7fae937fe640 1 -- 172.21.15.96:0/25782500 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98004020 con 0x55618a76dd30 2024-09-10T22:54:05.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7fae937fe640 1 -- 172.21.15.96:0/25782500 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980041c0 con 0x55618a76dd30 2024-09-10T22:54:05.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7fae937fe640 1 -- 172.21.15.96:0/25782500 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9800f850 con 0x55618a76dd30 2024-09-10T22:54:05.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 -- 172.21.15.96:0/25782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 msgr2=0x55618a770120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a770120 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faea810fe80 tx=0x7fae980118d0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faead9fd640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 0x7fae940b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faea0012760 tx=0x7faea0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 -- 172.21.15.96:0/25782500 shutdown_connections 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a770120 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 --2- 172.21.15.96:0/25782500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a761860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 -- 172.21.15.96:0/25782500 >> 172.21.15.96:0/25782500 conn(0x55618a75f2b0 msgr2=0x55618a760450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 -- 172.21.15.96:0/25782500 shutdown_connections 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.853+0000 7faeb1835740 1 -- 172.21.15.96:0/25782500 wait complete. 2024-09-10T22:54:05.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.854+0000 7faeb1835740 1 -- start start 2024-09-10T22:54:05.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.854+0000 7faeb1835740 1 Processor -- start 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a76d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55618a528700 con 0x55618a761da0 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55618a662110 con 0x55618a761480 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55618a7791c0 con 0x55618a76dd30 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faeb1835740 10 client.0 _ll_register_callbacks cb 0x55618a67b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faead9fd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faead9fd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45662/0 (socket says 172.21.15.96:45662) 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.855+0000 7faead9fd640 1 -- 172.21.15.96:0/3572210687 learned_addr learned my addr 172.21.15.96:0/3572210687 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 -- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 msgr2=0x55618a76d620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:05.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a76d620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 -- 172.21.15.96:0/3572210687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 msgr2=0x55618a76aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 -- 172.21.15.96:0/3572210687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618a68fc40 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7fae90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.856+0000 7faead9fd640 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faea000f800 tx=0x7faea0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5811935674f682 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000fe40 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faea0011bd0 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae980041c0 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faea0011e10 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7faeb1835740 10 client.26506 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55618a795b40 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980041c0 con 0x55618a76dd30 2024-09-10T22:54:05.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.857+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faea0027070 con 0x55618a76dd30 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 10 client.26506 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 10 client.26506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 20 client.26506 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a799470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 4 client.26506 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618a6d9da0 con 0x55618a796ff0 2024-09-10T22:54:05.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.858+0000 7faead1fc640 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a799470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7fae90ff9640 10 client.26506 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:05.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7faead1fc640 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a799470 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8573245c82cd43 server_cookie=f5ba935d815dd04 in_seq=0 out_seq=0 2024-09-10T22:54:05.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7fae90ff9640 1 -- 172.21.15.96:0/3572210687 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55618a6d9da0 con 0x55618a796ff0 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7fae90ff9640 10 client.26506 handle_command_reply: tid=0 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7faeb1835740 1 client.26506 shutdown 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 msgr2=0x55618a76b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.859+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faea000f800 tx=0x7faea0004730 comp rx=0 tx=0).stop 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 shutdown_connections 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a799470 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7fae90ff9640 0 client.26506 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a76b0a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a76aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 --2- 172.21.15.96:0/3572210687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a76d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 >> 172.21.15.96:0/3572210687 conn(0x55618a75f2b0 msgr2=0x55618a764e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 shutdown_connections 2024-09-10T22:54:05.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 1 -- 172.21.15.96:0/3572210687 wait complete. 2024-09-10T22:54:05.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.860+0000 7faeb1835740 20 client.26506 trim_cache size 0 max 16384 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 Processor -- start 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 -- start start 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761480 0x55618a76f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 0x55618a767c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faea8070b30 con 0x55618a761480 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faea8072670 con 0x55618a76dd30 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeb1835740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faea8071200 con 0x55618a761da0 2024-09-10T22:54:05.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeac9fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeac9fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45678/0 (socket says 172.21.15.96:45678) 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faeac9fb640 1 -- 172.21.15.96:0/2216014418 learned_addr learned my addr 172.21.15.96:0/2216014418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.883+0000 7faead9fd640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 0x55618a767c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faead1fc640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761480 0x55618a76f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 -- 172.21.15.96:0/2216014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 msgr2=0x55618a767c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 0x55618a767c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 -- 172.21.15.96:0/2216014418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761480 msgr2=0x55618a76f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761480 0x55618a76f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 -- 172.21.15.96:0/2216014418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618a760990 con 0x55618a761da0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faead9fd640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 0x55618a767c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7faeac9fb640 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae9801b040 tx=0x7fae98009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6bf5017f965fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7fae92ffd640 1 -- 172.21.15.96:0/2216014418 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9801d050 con 0x55618a761da0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7fae92ffd640 1 -- 172.21.15.96:0/2216014418 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae98029070 con 0x55618a761da0 2024-09-10T22:54:05.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.884+0000 7fae92ffd640 1 -- 172.21.15.96:0/2216014418 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003850 con 0x55618a761da0 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 -- 172.21.15.96:0/2216014418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 msgr2=0x55618a76f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae9801b040 tx=0x7fae98009530 comp rx=0 tx=0).stop 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 -- 172.21.15.96:0/2216014418 shutdown_connections 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a76dd30 0x55618a767c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a761da0 0x55618a76f660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 --2- 172.21.15.96:0/2216014418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761480 0x55618a76f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 -- 172.21.15.96:0/2216014418 >> 172.21.15.96:0/2216014418 conn(0x55618a75f2b0 msgr2=0x55618a76c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 -- 172.21.15.96:0/2216014418 shutdown_connections 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.885+0000 7faeb1835740 1 -- 172.21.15.96:0/2216014418 wait complete. 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 -- start start 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 Processor -- start 2024-09-10T22:54:05.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a77e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a779580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faea8071fa0 con 0x55618a761da0 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faea80718d0 con 0x55618a761480 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faea806eb30 con 0x55618a76dd30 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeac9fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeac9fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43338/0 (socket says 172.21.15.96:43338) 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeac9fb640 1 -- 172.21.15.96:0/3845108980 learned_addr learned my addr 172.21.15.96:0/3845108980 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faeb1835740 10 client.0 _ll_register_callbacks cb 0x7faea000f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faead1fc640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a77e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.886+0000 7faead9fd640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a779580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.887+0000 7faeac9fb640 1 -- 172.21.15.96:0/3845108980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 msgr2=0x55618a779580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.887+0000 7faeac9fb640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a779580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.887+0000 7faeac9fb640 1 -- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 msgr2=0x55618a77e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.887+0000 7faeac9fb640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a77e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.887+0000 7faeac9fb640 1 -- 172.21.15.96:0/3845108980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618a779db0 con 0x55618a761da0 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.888+0000 7faeac9fb640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fae9800f3f0 tx=0x7fae980044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748b2824e5c6497c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.888+0000 7fae90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.888+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98031070 con 0x55618a761da0 2024-09-10T22:54:05.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.888+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae98029070 con 0x55618a761da0 2024-09-10T22:54:05.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fae98003800 con 0x55618a761da0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae9803b050 con 0x55618a761da0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7faeb1835740 10 client.16884 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55618a804bb0 con 0x55618a761da0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae98029070 con 0x55618a761da0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fae9803e020 con 0x55618a761da0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7faeb1835740 10 client.16884 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.889+0000 7faeb1835740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faeb1835740 20 client.16884 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a807070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faeb1835740 4 client.16884 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55618a6d5850 con 0x55618a796ff0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faead1fc640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a807070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:05.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7faead1fc640 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a807070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0a8f4620614f08 server_cookie=4bf355e1f94acdd5 in_seq=0 out_seq=0 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.890+0000 7fae90ff9640 10 client.16884 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7fae90ff9640 1 -- 172.21.15.96:0/3845108980 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55618a6d5850 con 0x55618a796ff0 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7fae90ff9640 10 client.16884 handle_command_reply: tid=0 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 client.16884 shutdown 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 msgr2=0x55618a77eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fae9800f3f0 tx=0x7fae980044d0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 shutdown_connections 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55618a796ff0 0x55618a807070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7fae90ff9640 0 client.16884 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55618a76dd30 0x55618a779580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55618a761da0 0x55618a77eda0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 --2- 172.21.15.96:0/3845108980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55618a761480 0x55618a77e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 >> 172.21.15.96:0/3845108980 conn(0x55618a75f2b0 msgr2=0x55618a75dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 shutdown_connections 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 1 -- 172.21.15.96:0/3845108980 wait complete. 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.891+0000 7faeb1835740 20 client.16884 trim_cache size 0 max 16384 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "6f5238a0-8988-48e6-bc63-8f6795c3cce5", 2024-09-10T22:54:05.897 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:54:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:54:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:05.903 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 msgr2=0x7fae940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 0x7fae940b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faea0012760 tx=0x7faea0005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 msgr2=0x7faea819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 0x7faea819bc70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faea402acc0 tx=0x7faea4003c20 comp rx=0 tx=0).stop 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 shutdown_connections 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faea8109170 0x7faea819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faea81087f0 0x7faea819c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 --2- 172.21.15.96:0/1035275413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faea8102de0 0x7faea819bc70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:05.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 >> 172.21.15.96:0/1035275413 conn(0x7faea80fe830 msgr2=0x7faea80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:05.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 shutdown_connections 2024-09-10T22:54:05.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:05.892+0000 7faeaf460640 1 -- 172.21.15.96:0/1035275413 wait complete. 2024-09-10T22:54:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 390.0 s, 2470.0 tps, lat 40.278 ms stddev 45.172 2024-09-10T22:54:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:06 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:06.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:06 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:06.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:06 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:07.262 INFO:tasks.cephfs.filesystem:scrub status for tag:6f5238a0-8988-48e6-bc63-8f6795c3cce5 - {'path': '/', 'tag': '6f5238a0-8988-48e6-bc63-8f6795c3cce5', 'options': 'recursive,force'} 2024-09-10T22:54:07.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:07 smithi096 ceph-mon[30869]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 813 op/s 2024-09-10T22:54:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:07 smithi121 ceph-mon[35119]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 813 op/s 2024-09-10T22:54:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:07 smithi117 ceph-mon[35081]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 MiB/s wr, 813 op/s 2024-09-10T22:54:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:09 smithi096 ceph-mon[30869]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-09-10T22:54:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:09 smithi121 ceph-mon[35119]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-09-10T22:54:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:09 smithi117 ceph-mon[35081]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 620 op/s 2024-09-10T22:54:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:10 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:54:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:10 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:54:10.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:10 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:54:10.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:10 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:54:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:10 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:54:10.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:10 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:54:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 395.0 s, 2842.8 tps, lat 35.437 ms stddev 39.162 2024-09-10T22:54:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:11 smithi096 ceph-mon[30869]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.2 MiB/s wr, 817 op/s 2024-09-10T22:54:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:11 smithi121 ceph-mon[35119]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.2 MiB/s wr, 817 op/s 2024-09-10T22:54:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:11 smithi117 ceph-mon[35081]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.2 MiB/s wr, 817 op/s 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 -- 172.21.15.96:0/4061026452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248103380 msgr2=0x7f4248109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 --2- 172.21.15.96:0/4061026452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248103380 0x7f4248109d00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4240009cd0 tx=0x7f424002a8f0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 -- 172.21.15.96:0/4061026452 shutdown_connections 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 --2- 172.21.15.96:0/4061026452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248103380 0x7f4248109d00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 --2- 172.21.15.96:0/4061026452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42481029c0 0x7f4248102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 --2- 172.21.15.96:0/4061026452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 0x7f4248101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 -- 172.21.15.96:0/4061026452 >> 172.21.15.96:0/4061026452 conn(0x7f42480fd150 msgr2=0x7f42480ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 -- 172.21.15.96:0/4061026452 shutdown_connections 2024-09-10T22:54:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.058+0000 7f424cb7c640 1 -- 172.21.15.96:0/4061026452 wait complete. 2024-09-10T22:54:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 Processor -- start 2024-09-10T22:54:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 -- start start 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 0x7f4248196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4248103380 0x7f4248197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c400 con 0x7f42481029c0 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c280 con 0x7f4248101870 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424cb7c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c580 con 0x7f4248103380 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38148/0 (socket says 172.21.15.96:38148) 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 -- 172.21.15.96:0/2946203273 learned_addr learned my addr 172.21.15.96:0/2946203273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f4246d9d640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4248103380 0x7f4248197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f424659c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 0x7f4248196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 -- 172.21.15.96:0/2946203273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4248103380 msgr2=0x7f4248197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4248103380 0x7f4248197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 -- 172.21.15.96:0/2946203273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 msgr2=0x7f4248196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 0x7f4248196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.059+0000 7f423ffff640 1 -- 172.21.15.96:0/2946203273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42481985e0 con 0x7f42481029c0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f423ffff640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f423000c920 tx=0x7f423000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f423f7fe640 1 -- 172.21.15.96:0/2946203273 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230007c70 con 0x7f42481029c0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42481988d0 con 0x7f42481029c0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f423f7fe640 1 -- 172.21.15.96:0/2946203273 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4230007e10 con 0x7f42481029c0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f423f7fe640 1 -- 172.21.15.96:0/2946203273 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230018820 con 0x7f42481029c0 2024-09-10T22:54:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.060+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424819efa0 con 0x7f42481029c0 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 Processor -- start 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 -- start start 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 0x55f7469dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f7469446f0 con 0x55f7469d1110 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f746933b20 con 0x55f7469e0470 2024-09-10T22:54:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424fa25740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f746937930 con 0x55f7469d07d0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 0x55f7469dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.064+0000 7f424659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 0x55f7469dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38160/0 (socket says 172.21.15.96:38160) 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f424659c640 1 -- 172.21.15.96:0/3450990634 learned_addr learned my addr 172.21.15.96:0/3450990634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 -- 172.21.15.96:0/3450990634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 msgr2=0x55f7469e28a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e28a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 -- 172.21.15.96:0/3450990634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 msgr2=0x55f7469dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 0x55f7469dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.065+0000 7f4246d9d640 1 -- 172.21.15.96:0/3450990634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7468f9530 con 0x55f7469d07d0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.068+0000 7f4246d9d640 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469d0bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f424002ae10 tx=0x7f4240009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59588af022edda1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.072+0000 7f423f7fe640 1 -- 172.21.15.96:0/2946203273 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f423000f450 con 0x7f42481029c0 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.073+0000 7f423f7fe640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 0x7f42340b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.073+0000 7f423f7fe640 1 -- 172.21.15.96:0/2946203273 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f42300e5ca0 con 0x7f42481029c0 2024-09-10T22:54:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.074+0000 7f424659c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 0x7f42340b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.075+0000 7f423cff9640 1 -- 172.21.15.96:0/3450990634 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240030900 con 0x55f7469d07d0 2024-09-10T22:54:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.075+0000 7f423cff9640 1 -- 172.21.15.96:0/3450990634 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4240036030 con 0x55f7469d07d0 2024-09-10T22:54:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.075+0000 7f423cff9640 1 -- 172.21.15.96:0/3450990634 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240004520 con 0x55f7469d07d0 2024-09-10T22:54:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.075+0000 7f424659c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 0x7f42340b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42380045b0 tx=0x7f4238005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.076+0000 7f424fa25740 1 -- 172.21.15.96:0/3450990634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 msgr2=0x55f7469d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.076+0000 7f424fa25740 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469d0bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f424002ae10 tx=0x7f4240009a20 comp rx=0 tx=0).stop 2024-09-10T22:54:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.079+0000 7f424fa25740 1 -- 172.21.15.96:0/3450990634 shutdown_connections 2024-09-10T22:54:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.079+0000 7f424fa25740 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.079+0000 7f424fa25740 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d1110 0x55f7469dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.079+0000 7f424fa25740 1 --2- 172.21.15.96:0/3450990634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469d0bd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.079+0000 7f424fa25740 1 -- 172.21.15.96:0/3450990634 >> 172.21.15.96:0/3450990634 conn(0x55f7469cd930 msgr2=0x55f7469ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.085+0000 7f424fa25740 1 -- 172.21.15.96:0/3450990634 shutdown_connections 2024-09-10T22:54:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.089+0000 7f424fa25740 1 -- 172.21.15.96:0/3450990634 wait complete. 2024-09-10T22:54:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.089+0000 7f424fa25740 1 -- start start 2024-09-10T22:54:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.089+0000 7f424fa25740 1 Processor -- start 2024-09-10T22:54:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d07d0 0x55f7469e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 0x55f7469e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f74686bab0 con 0x55f7469d07d0 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f7468d9730 con 0x55f7469e0470 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f7469e30b0 con 0x55f7469d1110 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f424fa25740 10 client.0 _ll_register_callbacks cb 0x55f74692d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f4246d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f4246d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37430/0 (socket says 172.21.15.96:37430) 2024-09-10T22:54:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.094+0000 7f4246d9d640 1 -- 172.21.15.96:0/2828531197 learned_addr learned my addr 172.21.15.96:0/2828531197 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.095+0000 7f423ffff640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 0x55f7469e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.096+0000 7f423ffff640 1 -- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 msgr2=0x55f7469e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.096+0000 7f423ffff640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.096+0000 7f423ffff640 1 -- 172.21.15.96:0/2828531197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d07d0 msgr2=0x55f7469e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.096+0000 7f423ffff640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d07d0 0x55f7469e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.096+0000 7f423ffff640 1 -- 172.21.15.96:0/2828531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f746695790 con 0x55f7469d1110 2024-09-10T22:54:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.105+0000 7f423ffff640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 0x55f7469e4b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42300a96d0 tx=0x7f42300abc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b9e34ffb1112e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.108+0000 7f422dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.108+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300ac460 con 0x55f7469d1110 2024-09-10T22:54:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.108+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f42300b5560 con 0x55f7469d1110 2024-09-10T22:54:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.108+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4240036030 con 0x55f7469d1110 2024-09-10T22:54:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.109+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42300b57e0 con 0x55f7469d1110 2024-09-10T22:54:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.113+0000 7f424fa25740 10 client.26521 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.114+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f74670e620 con 0x55f7469d1110 2024-09-10T22:54:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42300b0f40 con 0x55f7469d1110 2024-09-10T22:54:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f42300a6b60 con 0x55f7469d1110 2024-09-10T22:54:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f424fa25740 10 client.26521 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f424fa25740 10 client.26521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f424fa25740 20 client.26521 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469f01e0 0x55f7469f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.115+0000 7f424fa25740 4 client.26521 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.116+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4240036030 con 0x55f7469f01e0 2024-09-10T22:54:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.116+0000 7f424659c640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469f01e0 0x55f7469f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.123+0000 7f424659c640 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469f01e0 0x55f7469f2620 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98f627ae2db2a44 server_cookie=ad125f6dee279e84 in_seq=0 out_seq=0 2024-09-10T22:54:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.123+0000 7f422dffb640 10 client.26521 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.127+0000 7f422dffb640 1 -- 172.21.15.96:0/2828531197 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4240036030 con 0x55f7469f01e0 2024-09-10T22:54:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.127+0000 7f422dffb640 10 client.26521 handle_command_reply: tid=0 2024-09-10T22:54:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.128+0000 7f424fa25740 1 client.26521 shutdown 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 msgr2=0x55f7469e4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 0x55f7469e4b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42300a96d0 tx=0x7f42300abc90 comp rx=0 tx=0).stop 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 shutdown_connections 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469f01e0 0x55f7469f2620 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469e0470 0x55f7469e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d1110 0x55f7469e4b70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 --2- 172.21.15.96:0/2828531197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469d07d0 0x55f7469e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.130+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 >> 172.21.15.96:0/2828531197 conn(0x55f7469cd930 msgr2=0x55f7469ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.131+0000 7f422dffb640 0 client.26521 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.131+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 shutdown_connections 2024-09-10T22:54:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.131+0000 7f424fa25740 1 -- 172.21.15.96:0/2828531197 wait complete. 2024-09-10T22:54:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.131+0000 7f424fa25740 20 client.26521 trim_cache size 0 max 16384 2024-09-10T22:54:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.157+0000 7f424fa25740 1 Processor -- start 2024-09-10T22:54:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.157+0000 7f424fa25740 1 -- start start 2024-09-10T22:54:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.157+0000 7f424fa25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424fa25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f424819f300 con 0x55f7469e0470 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424fa25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42481a0e40 con 0x55f7469d1110 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47476/0 (socket says 172.21.15.96:47476) 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 -- 172.21.15.96:0/388849235 learned_addr learned my addr 172.21.15.96:0/388849235 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f424819f9d0 con 0x55f7469d07d0 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f4246d9d640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f423ffff640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 -- 172.21.15.96:0/388849235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 msgr2=0x55f7469ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 -- 172.21.15.96:0/388849235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 msgr2=0x55f7469cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f424659c640 1 -- 172.21.15.96:0/388849235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7469d0bb0 con 0x55f7469d07d0 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.158+0000 7f423ffff640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.159+0000 7f424659c640 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4238004250 tx=0x7f423800f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620d40db20067d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.159+0000 7f422ffff640 1 -- 172.21.15.96:0/388849235 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238021070 con 0x55f7469d07d0 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.159+0000 7f422ffff640 1 -- 172.21.15.96:0/388849235 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4238022040 con 0x55f7469d07d0 2024-09-10T22:54:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.159+0000 7f422ffff640 1 -- 172.21.15.96:0/388849235 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423802e670 con 0x55f7469d07d0 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.159+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 msgr2=0x55f7469ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4238004250 tx=0x7f423800f050 comp rx=0 tx=0).stop 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 shutdown_connections 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 --2- 172.21.15.96:0/388849235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469ce190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 >> 172.21.15.96:0/388849235 conn(0x55f7469cd930 msgr2=0x55f7469df980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 shutdown_connections 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 -- 172.21.15.96:0/388849235 wait complete. 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 -- start start 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.160+0000 7f424fa25740 1 Processor -- start 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42481a0770 con 0x55f7469e0470 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42481a00a0 con 0x55f7469d1110 2024-09-10T22:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4248073500 con 0x55f7469d07d0 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424fa25740 10 client.0 _ll_register_callbacks cb 0x7f423000fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37454/0 (socket says 172.21.15.96:37454) 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 -- 172.21.15.96:0/3740619890 learned_addr learned my addr 172.21.15.96:0/3740619890 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f4246d9d640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424659c640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 -- 172.21.15.96:0/3740619890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 msgr2=0x55f7469f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 -- 172.21.15.96:0/3740619890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 msgr2=0x55f7469f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f423ffff640 1 -- 172.21.15.96:0/3740619890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7469f66e0 con 0x55f7469d1110 2024-09-10T22:54:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.161+0000 7f424659c640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f423ffff640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42300ac3f0 tx=0x7f42300abe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee973774593e52e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300b0f40 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4238022040 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f42300d3170 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42300b59f0 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 10 client.26480 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f7469d1880 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4238022040 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f42300ab130 con 0x55f7469d1110 2024-09-10T22:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 10 client.26480 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 20 client.26480 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469d2280 0x55f7469d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 4 client.26480 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.162+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f746946640 con 0x55f7469d2280 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.163+0000 7f424659c640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469d2280 0x55f7469d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.163+0000 7f424659c640 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469d2280 0x55f7469d46a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3df1bb669ea899 server_cookie=49f7a1309911bbfd in_seq=0 out_seq=0 2024-09-10T22:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.163+0000 7f422dffb640 10 client.26480 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f422dffb640 1 -- 172.21.15.96:0/3740619890 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f746946640 con 0x55f7469d2280 2024-09-10T22:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f422dffb640 10 client.26480 handle_command_reply: tid=0 2024-09-10T22:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f424fa25740 1 client.26480 shutdown 2024-09-10T22:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 msgr2=0x55f7469f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42300ac3f0 tx=0x7f42300abe80 comp rx=0 tx=0).stop 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.180+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 shutdown_connections 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f7469d2280 0x55f7469d46a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7469d1110 0x55f7469f5620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f422dffb640 0 client.26480 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f7469e0470 0x55f7469f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 --2- 172.21.15.96:0/3740619890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7469d07d0 0x55f7469f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 >> 172.21.15.96:0/3740619890 conn(0x55f7469cd930 msgr2=0x55f7469ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 shutdown_connections 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 1 -- 172.21.15.96:0/3740619890 wait complete. 2024-09-10T22:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424fa25740 20 client.26480 trim_cache size 0 max 16384 2024-09-10T22:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 msgr2=0x7f42340b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 0x7f42340b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42380045b0 tx=0x7f4238005f90 comp rx=0 tx=0).stop 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 msgr2=0x7f42481970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.181+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f423000c920 tx=0x7f423000cde0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 shutdown_connections 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f42340b5360 0x7f42340b7820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4248103380 0x7f4248197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42481029c0 0x7f42481970f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 --2- 172.21.15.96:0/2946203273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4248101870 0x7f4248196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 >> 172.21.15.96:0/2946203273 conn(0x7f42480fd150 msgr2=0x7f4248105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 shutdown_connections 2024-09-10T22:54:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:12.182+0000 7f424cb7c640 1 -- 172.21.15.96:0/2946203273 wait complete. 2024-09-10T22:54:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:13 smithi096 ceph-mon[30869]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 8.2 MiB/s wr, 748 op/s 2024-09-10T22:54:13.578 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:13 smithi117 ceph-mon[35081]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 8.2 MiB/s wr, 748 op/s 2024-09-10T22:54:13.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:13 smithi121 ceph-mon[35119]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 8.2 MiB/s wr, 748 op/s 2024-09-10T22:54:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:15 smithi096 ceph-mon[30869]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.1 MiB/s wr, 817 op/s 2024-09-10T22:54:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:15 smithi121 ceph-mon[35119]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.1 MiB/s wr, 817 op/s 2024-09-10T22:54:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:15 smithi117 ceph-mon[35081]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.1 MiB/s wr, 817 op/s 2024-09-10T22:54:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 400.0 s, 2782.6 tps, lat 35.735 ms stddev 38.335 2024-09-10T22:54:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:17 smithi096 ceph-mon[30869]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.8 MiB/s wr, 857 op/s 2024-09-10T22:54:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:17 smithi121 ceph-mon[35119]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.8 MiB/s wr, 857 op/s 2024-09-10T22:54:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:17 smithi117 ceph-mon[35081]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 9.8 MiB/s wr, 857 op/s 2024-09-10T22:54:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:19 smithi096 ceph-mon[30869]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-09-10T22:54:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:19 smithi121 ceph-mon[35119]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-09-10T22:54:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:19 smithi117 ceph-mon[35081]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-09-10T22:54:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 405.0 s, 2901.0 tps, lat 34.591 ms stddev 38.347 2024-09-10T22:54:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:21 smithi096 ceph-mon[30869]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-09-10T22:54:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:21 smithi121 ceph-mon[35119]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-09-10T22:54:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:21 smithi117 ceph-mon[35081]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 10 MiB/s wr, 888 op/s 2024-09-10T22:54:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:23 smithi096 ceph-mon[30869]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-09-10T22:54:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:23 smithi121 ceph-mon[35119]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-09-10T22:54:23.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:23 smithi117 ceph-mon[35081]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 760 op/s 2024-09-10T22:54:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- 172.21.15.96:0/4193882450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f225006b3d0 msgr2=0x7f22501055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 --2- 172.21.15.96:0/4193882450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f225006b3d0 0x7f22501055d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f224c009900 tx=0x7f224c02a410 comp rx=0 tx=0).stop 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- 172.21.15.96:0/4193882450 shutdown_connections 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 --2- 172.21.15.96:0/4193882450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 --2- 172.21.15.96:0/4193882450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2250108680 0x7f225010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 --2- 172.21.15.96:0/4193882450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f225006b3d0 0x7f22501055d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- 172.21.15.96:0/4193882450 >> 172.21.15.96:0/4193882450 conn(0x7f22500fd140 msgr2=0x7f22500ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- 172.21.15.96:0/4193882450 shutdown_connections 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- 172.21.15.96:0/4193882450 wait complete. 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 Processor -- start 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 -- start start 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.311+0000 7f22581a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f22581a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f22581a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 0x7f2250199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f22581a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f225010e5d0 con 0x7f225006b3d0 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f22581a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f225010e450 con 0x7f2250105b10 2024-09-10T22:54:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f22581a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f225010e750 con 0x7f2250108680 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48304/0 (socket says 172.21.15.96:48304) 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 -- 172.21.15.96:0/123352113 learned_addr learned my addr 172.21.15.96:0/123352113 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f225673f640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 0x7f2250199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f225573d640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 -- 172.21.15.96:0/123352113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 msgr2=0x7f2250199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 0x7f2250199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 -- 172.21.15.96:0/123352113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 msgr2=0x7f2250199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 -- 172.21.15.96:0/123352113 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225019a860 con 0x7f225006b3d0 2024-09-10T22:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f225573d640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f225673f640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 0x7f2250199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2255f3e640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f224c031500 tx=0x7f224c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.312+0000 7f2246ffd640 1 -- 172.21.15.96:0/123352113 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c03f030 con 0x7f225006b3d0 2024-09-10T22:54:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.313+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225019aaf0 con 0x7f225006b3d0 2024-09-10T22:54:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.313+0000 7f2246ffd640 1 -- 172.21.15.96:0/123352113 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224c003800 con 0x7f225006b3d0 2024-09-10T22:54:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.313+0000 7f2246ffd640 1 -- 172.21.15.96:0/123352113 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c035070 con 0x7f225006b3d0 2024-09-10T22:54:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.313+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a1220 con 0x7f225006b3d0 2024-09-10T22:54:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 Processor -- start 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 -- start start 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb612c1b0 0x558bb612e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb61310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558bb609b6f0 con 0x558bb612ebf0 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bb608ab20 con 0x558bb612c1b0 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.316+0000 7f225b04a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558bb608e930 con 0x558bb6129880 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48202/0 (socket says 172.21.15.96:48202) 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 -- 172.21.15.96:0/1890598674 learned_addr learned my addr 172.21.15.96:0/1890598674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 -- 172.21.15.96:0/1890598674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb612c1b0 msgr2=0x558bb612e5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb612c1b0 0x558bb612e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225673f640 1 -- 172.21.15.96:0/1890598674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 msgr2=0x558bb61310b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.317+0000 7f225573d640 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb61310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f225673f640 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb61310b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f225673f640 1 -- 172.21.15.96:0/1890598674 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb6050530 con 0x558bb6129880 2024-09-10T22:54:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f225573d640 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb61310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f225673f640 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2248015e70 tx=0x7f224800c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aafbfb4c5e718631 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f223bfff640 1 -- 172.21.15.96:0/1890598674 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224800ce10 con 0x558bb6129880 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f223bfff640 1 -- 172.21.15.96:0/1890598674 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224800a560 con 0x558bb6129880 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.318+0000 7f223bfff640 1 -- 172.21.15.96:0/1890598674 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224801f6d0 con 0x558bb6129880 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.319+0000 7f2246ffd640 1 -- 172.21.15.96:0/123352113 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f224c003cd0 con 0x7f225006b3d0 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.319+0000 7f225b04a740 1 -- 172.21.15.96:0/1890598674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 msgr2=0x558bb612bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.319+0000 7f225b04a740 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2248015e70 tx=0x7f224800c700 comp rx=0 tx=0).stop 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.319+0000 7f225b04a740 1 -- 172.21.15.96:0/1890598674 shutdown_connections 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.319+0000 7f225b04a740 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb612c1b0 0x558bb612e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f225b04a740 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb61310b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f225b04a740 1 --2- 172.21.15.96:0/1890598674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb6129880 0x558bb612bc70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f225b04a740 1 -- 172.21.15.96:0/1890598674 >> 172.21.15.96:0/1890598674 conn(0x558bb61249d0 msgr2=0x558bb6125760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f2246ffd640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 0x7f223c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f225b04a740 1 -- 172.21.15.96:0/1890598674 shutdown_connections 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f2246ffd640 1 -- 172.21.15.96:0/123352113 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f224c106460 con 0x7f225006b3d0 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.320+0000 7f225b04a740 1 -- 172.21.15.96:0/1890598674 wait complete. 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.321+0000 7f225b04a740 1 -- start start 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.321+0000 7f225b04a740 1 Processor -- start 2024-09-10T22:54:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.321+0000 7f225573d640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 0x7f223c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb61341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558bb5fc2ab0 con 0x558bb612c1b0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bb6030730 con 0x558bb6129880 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558bb6131bd0 con 0x558bb612ebf0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225b04a740 10 client.0 _ll_register_callbacks cb 0x558bb60846b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225573d640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 0x7f223c0b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22400058c0 tx=0x7f224000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225573d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb61341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb61341e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48208/0 (socket says 172.21.15.96:48208) 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225573d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48324/0 (socket says 172.21.15.96:48324) 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f225673f640 1 -- 172.21.15.96:0/1311214344 learned_addr learned my addr 172.21.15.96:0/1311214344 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.322+0000 7f2255f3e640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.323+0000 7f225573d640 1 -- 172.21.15.96:0/1311214344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 msgr2=0x558bb61341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.323+0000 7f225573d640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb61341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.323+0000 7f225573d640 1 -- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 msgr2=0x558bb6132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.323+0000 7f225573d640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.323+0000 7f225573d640 1 -- 172.21.15.96:0/1311214344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb5dec790 con 0x558bb612c1b0 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f225573d640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2240012e20 tx=0x7f22400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65ce8edc210eda4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224001f620 con 0x558bb612c1b0 2024-09-10T22:54:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f224001f7c0 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224800a560 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f224001fa00 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f225b04a740 10 client.16905 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.324+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bb5e65620 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224800a560 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f22400313b0 con 0x558bb612c1b0 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 10 client.16905 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 20 client.16905 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb613e860 0x558bb6140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 4 client.16905 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bb609d640 con 0x558bb613e860 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f2255f3e640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb613e860 0x558bb6140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f2255f3e640 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb613e860 0x558bb6140c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5600ad2937c801 server_cookie=f632ee97bad48e9d in_seq=0 out_seq=0 2024-09-10T22:54:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.325+0000 7f22397fa640 10 client.16905 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.326+0000 7f22397fa640 1 -- 172.21.15.96:0/1311214344 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bb609d640 con 0x558bb613e860 2024-09-10T22:54:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.326+0000 7f22397fa640 10 client.16905 handle_command_reply: tid=0 2024-09-10T22:54:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.327+0000 7f225b04a740 1 client.16905 shutdown 2024-09-10T22:54:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 msgr2=0x558bb6133610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2240012e20 tx=0x7f22400046c0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 shutdown_connections 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb613e860 0x558bb6140c50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb61341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6133610 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 --2- 172.21.15.96:0/1311214344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f22397fa640 0 client.16905 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 >> 172.21.15.96:0/1311214344 conn(0x558bb61249d0 msgr2=0x558bb6123bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 shutdown_connections 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 1 -- 172.21.15.96:0/1311214344 wait complete. 2024-09-10T22:54:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.328+0000 7f225b04a740 20 client.16905 trim_cache size 0 max 16384 2024-09-10T22:54:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.349+0000 7f225b04a740 1 Processor -- start 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.349+0000 7f225b04a740 1 -- start start 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 0x558bb613b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb6127810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a7b50 con 0x558bb612ebf0 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a9690 con 0x558bb6129880 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225b04a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a8220 con 0x558bb612c1b0 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225673f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb6127810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 0x558bb613b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225673f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb6127810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48334/0 (socket says 172.21.15.96:48334) 2024-09-10T22:54:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225673f640 1 -- 172.21.15.96:0/2906959483 learned_addr learned my addr 172.21.15.96:0/2906959483 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f2255f3e640 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 -- 172.21.15.96:0/2906959483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 msgr2=0x558bb6132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 -- 172.21.15.96:0/2906959483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 msgr2=0x558bb6127810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb6127810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.350+0000 7f225573d640 1 -- 172.21.15.96:0/2906959483 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb612e910 con 0x558bb612c1b0 2024-09-10T22:54:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.351+0000 7f225573d640 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 0x558bb613b220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f22400317f0 tx=0x7f2240005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649bdda0d778e156 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.351+0000 7f223b7fe640 1 -- 172.21.15.96:0/2906959483 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22400315b0 con 0x558bb612c1b0 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.351+0000 7f223b7fe640 1 -- 172.21.15.96:0/2906959483 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2240012e50 con 0x558bb612c1b0 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.351+0000 7f223b7fe640 1 -- 172.21.15.96:0/2906959483 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240032460 con 0x558bb612c1b0 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 -- 172.21.15.96:0/2906959483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 msgr2=0x558bb613b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 0x558bb613b220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f22400317f0 tx=0x7f2240005740 comp rx=0 tx=0).stop 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 -- 172.21.15.96:0/2906959483 shutdown_connections 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612ebf0 0x558bb6127810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612c1b0 0x558bb613b220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 --2- 172.21.15.96:0/2906959483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb6132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 -- 172.21.15.96:0/2906959483 >> 172.21.15.96:0/2906959483 conn(0x558bb61249d0 msgr2=0x558bb6132210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 -- 172.21.15.96:0/2906959483 shutdown_connections 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.352+0000 7f225b04a740 1 -- 172.21.15.96:0/2906959483 wait complete. 2024-09-10T22:54:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 -- start start 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 Processor -- start 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb613afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a8fc0 con 0x558bb612c1b0 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a88f0 con 0x558bb6129880 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f22501a5b70 con 0x558bb612ebf0 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225673f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48222/0 (socket says 172.21.15.96:48222) 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225673f640 1 -- 172.21.15.96:0/2223738935 learned_addr learned my addr 172.21.15.96:0/2223738935 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f225b04a740 10 client.0 _ll_register_callbacks cb 0x7f2240011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.353+0000 7f2255f3e640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb613afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 -- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 msgr2=0x558bb613afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225573d640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb613afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 -- 172.21.15.96:0/2223738935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 msgr2=0x558bb6139cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6139cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 -- 172.21.15.96:0/2223738935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb613aac0 con 0x558bb612ebf0 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225573d640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6139cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f225673f640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f224800a7c0 tx=0x7f224800a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95f31feac1c69fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.354+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.355+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224802f020 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.355+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f224801b040 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.355+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2240012e50 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.355+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f224800ce10 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.356+0000 7f225b04a740 10 client.26542 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.356+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bb61ab830 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224800aba0 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f224802b650 con 0x558bb612ebf0 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 10 client.26542 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 10 client.26542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 20 client.26542 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb61ac080 0x558bb61ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 4 client.26542 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2240012e50 con 0x558bb61ac080 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.357+0000 7f2255f3e640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb61ac080 0x558bb61ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.358+0000 7f22397fa640 10 client.26542 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.358+0000 7f2255f3e640 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb61ac080 0x558bb61ae4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a60f7d8bede19a0 server_cookie=61c11bfa13267c6 in_seq=0 out_seq=0 2024-09-10T22:54:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f22397fa640 1 -- 172.21.15.96:0/2223738935 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2240012e50 con 0x558bb61ac080 2024-09-10T22:54:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f22397fa640 10 client.26542 handle_command_reply: tid=0 2024-09-10T22:54:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 client.26542 shutdown 2024-09-10T22:54:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 msgr2=0x558bb613a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f224800a7c0 tx=0x7f224800a7f0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 shutdown_connections 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558bb61ac080 0x558bb61ae4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558bb612ebf0 0x558bb613a290 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bb612c1b0 0x558bb6139cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 --2- 172.21.15.96:0/2223738935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bb6129880 0x558bb613afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f22397fa640 0 client.26542 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 >> 172.21.15.96:0/2223738935 conn(0x558bb61249d0 msgr2=0x558bb61233d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 shutdown_connections 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 1 -- 172.21.15.96:0/2223738935 wait complete. 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.371+0000 7f225b04a740 20 client.26542 trim_cache size 0 max 16384 2024-09-10T22:54:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:54:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:54:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:54:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 msgr2=0x7f223c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 0x7f223c0b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22400058c0 tx=0x7f224000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 msgr2=0x7f2250198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f224c031500 tx=0x7f224c0043c0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 shutdown_connections 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2250105b10 0x7f2250199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f223c0b52e0 0x7f223c0b77a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2250108680 0x7f2250199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 --2- 172.21.15.96:0/123352113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f225006b3d0 0x7f2250198d20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 >> 172.21.15.96:0/123352113 conn(0x7f22500fd140 msgr2=0x7f2250103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 shutdown_connections 2024-09-10T22:54:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:25.372+0000 7f22581a2640 1 -- 172.21.15.96:0/123352113 wait complete. 2024-09-10T22:54:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:25 smithi096 ceph-mon[30869]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-09-10T22:54:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:25 smithi121 ceph-mon[35119]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-09-10T22:54:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:25 smithi117 ceph-mon[35081]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-09-10T22:54:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 410.0 s, 2943.6 tps, lat 33.977 ms stddev 37.277 2024-09-10T22:54:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:27 smithi096 ceph-mon[30869]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-09-10T22:54:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:27 smithi121 ceph-mon[35119]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-09-10T22:54:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:27 smithi117 ceph-mon[35081]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 812 op/s 2024-09-10T22:54:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:29 smithi096 ceph-mon[30869]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-10T22:54:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:29 smithi121 ceph-mon[35119]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-10T22:54:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:29 smithi117 ceph-mon[35081]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-10T22:54:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 415.0 s, 2699.6 tps, lat 36.657 ms stddev 39.702 2024-09-10T22:54:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:31 smithi096 ceph-mon[30869]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 863 op/s 2024-09-10T22:54:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:31 smithi121 ceph-mon[35119]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 863 op/s 2024-09-10T22:54:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:31 smithi117 ceph-mon[35081]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 863 op/s 2024-09-10T22:54:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:33 smithi096 ceph-mon[30869]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T22:54:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:33 smithi121 ceph-mon[35119]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T22:54:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:33 smithi117 ceph-mon[35081]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-09-10T22:54:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-10T22:54:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:54:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-10T22:54:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:54:35.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:54:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 420.0 s, 2288.2 tps, lat 44.136 ms stddev 50.891 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Unable to set osd_memory_target on smithi117 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Unable to set osd_memory_target on smithi121 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-10T22:54:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:36 smithi096 ceph-mon[30869]: Unable to set osd_memory_target on smithi096 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T22:54:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-10T22:54:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-10T22:54:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Unable to set osd_memory_target on smithi117 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-10T22:54:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Unable to set osd_memory_target on smithi121 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T22:54:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-10T22:54:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:36 smithi121 ceph-mon[35119]: Unable to set osd_memory_target on smithi096 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Unable to set osd_memory_target on smithi117 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Unable to set osd_memory_target on smithi121 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-10T22:54:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:36 smithi117 ceph-mon[35081]: Unable to set osd_memory_target on smithi096 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T22:54:37.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:37.623 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:37.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:37 smithi096 ceph-mon[30869]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 742 op/s 2024-09-10T22:54:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:37 smithi121 ceph-mon[35119]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 742 op/s 2024-09-10T22:54:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:37 smithi117 ceph-mon[35081]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 742 op/s 2024-09-10T22:54:38.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/1343297127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 msgr2=0x7f6e98102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/1343297127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e98102070 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009900 tx=0x7f6e9402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/1343297127 shutdown_connections 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/1343297127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e981076a0 0x7f6e98107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/1343297127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e98108020 0x7f6e9810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/1343297127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e98102070 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/1343297127 >> 172.21.15.96:0/1343297127 conn(0x7f6e980fd700 msgr2=0x7f6e980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/1343297127 shutdown_connections 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.130+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/1343297127 wait complete. 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 Processor -- start 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 -- start start 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 0x7f6e9819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98108020 0x7f6e9819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e98111ce0 con 0x7f6e98101c90 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e98111b60 con 0x7f6e98108020 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9f3a0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6e98111e60 con 0x7f6e981076a0 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50212/0 (socket says 172.21.15.96:50212) 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 -- 172.21.15.96:0/298154328 learned_addr learned my addr 172.21.15.96:0/298154328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98108020 0x7f6e9819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 0x7f6e9819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 -- 172.21.15.96:0/298154328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 msgr2=0x7f6e9819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 0x7f6e9819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 -- 172.21.15.96:0/298154328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98108020 msgr2=0x7f6e9819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98108020 0x7f6e9819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9d13c640 1 -- 172.21.15.96:0/298154328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e981a5490 con 0x7f6e98101c90 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.131+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 0x7f6e9819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6e94005940 tx=0x7f6e940040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e8e7fc640 1 -- 172.21.15.96:0/298154328 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9403f030 con 0x7f6e98101c90 2024-09-10T22:54:38.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e981a5720 con 0x7f6e98101c90 2024-09-10T22:54:38.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e8e7fc640 1 -- 172.21.15.96:0/298154328 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e940044c0 con 0x7f6e98101c90 2024-09-10T22:54:38.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e8e7fc640 1 -- 172.21.15.96:0/298154328 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9403f030 con 0x7f6e98101c90 2024-09-10T22:54:38.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.132+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e981a5c00 con 0x7f6e98101c90 2024-09-10T22:54:38.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.136+0000 7f6e8e7fc640 1 -- 172.21.15.96:0/298154328 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f6e9403b770 con 0x7f6e98101c90 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 Processor -- start 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e8e7fc640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 0x7f6e840b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 -- start start 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 0x7f6e840b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e8e7fc640 1 -- 172.21.15.96:0/298154328 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6e94040070 con 0x7f6e98101c90 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 0x55c14fcb6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 0x55c14fcc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c14fc2ce50 con 0x55c14fcc6d60 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c14fc1c280 con 0x55c14fcb69c0 2024-09-10T22:54:38.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6ea1775740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c14fc20090 con 0x55c14fcb60a0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e9c93b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 0x55c14fcc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e9d93d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e9d93d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50218/0 (socket says 172.21.15.96:50218) 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.137+0000 7f6e9d93d640 1 -- 172.21.15.96:0/557875873 learned_addr learned my addr 172.21.15.96:0/557875873 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 0x55c14fcb6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 0x7f6e840b7b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e9819b090 tx=0x7f6e90011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 -- 172.21.15.96:0/557875873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 msgr2=0x55c14fcb6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 0x55c14fcb6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 -- 172.21.15.96:0/557875873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 msgr2=0x55c14fcc9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 -- 172.21.15.96:0/557875873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14f97a650 con 0x55c14fcb69c0 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 0x55c14fcb6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 0x55c14fcc6820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009fa0 tx=0x7f6e880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0f7d73e7b014d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e837fe640 1 -- 172.21.15.96:0/557875873 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88004030 con 0x55c14fcb69c0 2024-09-10T22:54:38.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e837fe640 1 -- 172.21.15.96:0/557875873 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e880054d0 con 0x55c14fcb69c0 2024-09-10T22:54:38.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.138+0000 7f6e837fe640 1 -- 172.21.15.96:0/557875873 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88004c40 con 0x55c14fcb69c0 2024-09-10T22:54:38.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 -- 172.21.15.96:0/557875873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 msgr2=0x55c14fcc6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 0x55c14fcc6820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009fa0 tx=0x7f6e880118d0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 -- 172.21.15.96:0/557875873 shutdown_connections 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb69c0 0x55c14fcc6820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fcc9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 --2- 172.21.15.96:0/557875873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb60a0 0x55c14fcb6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 -- 172.21.15.96:0/557875873 >> 172.21.15.96:0/557875873 conn(0x55c14fcb3ed0 msgr2=0x55c14fcb5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 -- 172.21.15.96:0/557875873 shutdown_connections 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.139+0000 7f6ea1775740 1 -- 172.21.15.96:0/557875873 wait complete. 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.140+0000 7f6ea1775740 1 -- start start 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.140+0000 7f6ea1775740 1 Processor -- start 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.140+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcc6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcc6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcc6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50230/0 (socket says 172.21.15.96:50230) 2024-09-10T22:54:38.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6e9d13c640 1 -- 172.21.15.96:0/396266421 learned_addr learned my addr 172.21.15.96:0/396266421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcc3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c14fa7d700 con 0x55c14fcb60a0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c14fbb7110 con 0x55c14fcc6d60 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c14fcc9a10 con 0x55c14fcb69c0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcc3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.141+0000 7f6ea1775740 10 client.0 _ll_register_callbacks cb 0x55c14fbd0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 -- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 msgr2=0x55c14fcc4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 -- 172.21.15.96:0/396266421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 msgr2=0x55c14fcc6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcc6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14fbe4c40 con 0x55c14fcb69c0 2024-09-10T22:54:38.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcc3ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e8801b040 tx=0x7f6e8800fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97724a77c3275594 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.142+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88028020 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e880054d0 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6e88005b90 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e8801d070 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 10 client.26554 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c14fcd5360 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e880054d0 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6e88004940 con 0x55c14fcb69c0 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 10 client.26554 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 10 client.26554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 20 client.26554 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcd6770 0x55c14fcd8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 4 client.26554 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.143+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c14fc2eda0 con 0x55c14fcd6770 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.144+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcd6770 0x55c14fcd8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.144+0000 7f6e80ff9640 10 client.26554 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.144+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcd6770 0x55c14fcd8b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d129291e85f73ad server_cookie=93f28c6a61f97996 in_seq=0 out_seq=0 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6e80ff9640 1 -- 172.21.15.96:0/396266421 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c14fc2eda0 con 0x55c14fcd6770 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6e80ff9640 10 client.26554 handle_command_reply: tid=0 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 client.26554 shutdown 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 msgr2=0x55c14fcc3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcc3ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e8801b040 tx=0x7f6e8800fc20 comp rx=0 tx=0).stop 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 shutdown_connections 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcd6770 0x55c14fcd8b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcc3ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6e80ff9640 0 client.26554 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 --2- 172.21.15.96:0/396266421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcc6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 >> 172.21.15.96:0/396266421 conn(0x55c14fcb3ed0 msgr2=0x55c14fcb9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 shutdown_connections 2024-09-10T22:54:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 1 -- 172.21.15.96:0/396266421 wait complete. 2024-09-10T22:54:38.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.145+0000 7f6ea1775740 20 client.26554 trim_cache size 0 max 16384 2024-09-10T22:54:38.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 Processor -- start 2024-09-10T22:54:38.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 -- start start 2024-09-10T22:54:38.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981ac050 con 0x55c14fcb60a0 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981adb90 con 0x55c14fcc6d60 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6ea1775740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981ac720 con 0x55c14fcb69c0 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9d93d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9c93b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9d93d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35968/0 (socket says 172.21.15.96:35968) 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9d93d640 1 -- 172.21.15.96:0/2488341842 learned_addr learned my addr 172.21.15.96:0/2488341842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.171+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcd2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50232/0 (socket says 172.21.15.96:50232) 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 -- 172.21.15.96:0/2488341842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 msgr2=0x55c14fcc5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 -- 172.21.15.96:0/2488341842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 msgr2=0x55c14fcd2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcd2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 -- 172.21.15.96:0/2488341842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14fcc1fe0 con 0x55c14fcb69c0 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.172+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcb3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e88000c00 tx=0x7f6e88009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4d865e3a2c4609 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6e82ffd640 1 -- 172.21.15.96:0/2488341842 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88028040 con 0x55c14fcb69c0 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6e82ffd640 1 -- 172.21.15.96:0/2488341842 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e8801e040 con 0x55c14fcb69c0 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6e82ffd640 1 -- 172.21.15.96:0/2488341842 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88024480 con 0x55c14fcb69c0 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 -- 172.21.15.96:0/2488341842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 msgr2=0x55c14fcb3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcb3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e88000c00 tx=0x7f6e88009d00 comp rx=0 tx=0).stop 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 -- 172.21.15.96:0/2488341842 shutdown_connections 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcc6d60 0x55c14fcc5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fcb3710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 --2- 172.21.15.96:0/2488341842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcb60a0 0x55c14fcd2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.173+0000 7f6ea1775740 1 -- 172.21.15.96:0/2488341842 >> 172.21.15.96:0/2488341842 conn(0x55c14fcb3ed0 msgr2=0x55c14fcc5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- 172.21.15.96:0/2488341842 shutdown_connections 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- 172.21.15.96:0/2488341842 wait complete. 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- start start 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 Processor -- start 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb60a0 0x55c14fd3de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fd3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981ad4c0 con 0x55c14fcc6d60 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981acdf0 con 0x55c14fcb60a0 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.174+0000 7f6ea1775740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6e981aa050 con 0x55c14fcb69c0 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6ea1775740 10 client.0 _ll_register_callbacks cb 0x7f6e90009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9c93b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fd3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9c93b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35020/0 (socket says 172.21.15.96:35020) 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d13c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb60a0 0x55c14fd3de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9c93b640 1 -- 172.21.15.96:0/1979362816 learned_addr learned my addr 172.21.15.96:0/1979362816 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 -- 172.21.15.96:0/1979362816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 msgr2=0x55c14fd3e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 -- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb60a0 msgr2=0x55c14fd3de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb60a0 0x55c14fd3de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 -- 172.21.15.96:0/1979362816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14fcbf410 con 0x55c14fcc6d60 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9c93b640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e9d93d640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fd3e8a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009fd0 tx=0x7f6e90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eaba8ed1548954 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90017040 con 0x55c14fcc6d60 2024-09-10T22:54:38.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6e90004b20 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.175+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e8801e040 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e90004d20 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 10 client.16926 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c14fcce070 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e8801e040 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6e90031020 con 0x55c14fcc6d60 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 10 client.16926 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 20 client.16926 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcce7c0 0x55c14fccebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 4 client.16926 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c14fc2a850 con 0x55c14fcce7c0 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.176+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcce7c0 0x55c14fccebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.177+0000 7f6e80ff9640 10 client.16926 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.177+0000 7f6e9d13c640 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcce7c0 0x55c14fccebd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0947849047be903 server_cookie=715f271b76861cfc in_seq=0 out_seq=0 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.178+0000 7f6e80ff9640 1 -- 172.21.15.96:0/1979362816 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x55c14fc2a850 con 0x55c14fcce7c0 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.178+0000 7f6e80ff9640 10 client.16926 handle_command_reply: tid=0 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.178+0000 7f6ea1775740 1 client.16926 shutdown 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 msgr2=0x55c14fd3e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fd3e8a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009fd0 tx=0x7f6e90004ab0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 shutdown_connections 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c14fcce7c0 0x55c14fccebd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14fcc6d60 0x55c14fd3e8a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6e80ff9640 0 client.16926 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c14fcb69c0 0x55c14fd3e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 --2- 172.21.15.96:0/1979362816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c14fcb60a0 0x55c14fd3de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 >> 172.21.15.96:0/1979362816 conn(0x55c14fcb3ed0 msgr2=0x55c14fcb54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 shutdown_connections 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 1 -- 172.21.15.96:0/1979362816 wait complete. 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.179+0000 7f6ea1775740 20 client.16926 trim_cache size 0 max 16384 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 msgr2=0x7f6e840b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 0x7f6e840b7b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e9819b090 tx=0x7f6e90011810 comp rx=0 tx=0).stop 2024-09-10T22:54:38.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 msgr2=0x7f6e9819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6e94005940 tx=0x7f6e940040c0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 shutdown_connections 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98108020 0x7f6e9819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f6e840b56d0 0x7f6e840b7b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e981076a0 0x7f6e9819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 --2- 172.21.15.96:0/298154328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e98101c90 0x7f6e9819a0b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 >> 172.21.15.96:0/298154328 conn(0x7f6e980fd700 msgr2=0x7f6e980fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 shutdown_connections 2024-09-10T22:54:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:38.180+0000 7f6e9f3a0640 1 -- 172.21.15.96:0/298154328 wait complete. 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 -- 172.21.15.96:0/1005368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1844104ba0 msgr2=0x7f1844104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1005368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1844104ba0 0x7f1844104fa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1834009930 tx=0x7f183402a420 comp rx=0 tx=0).stop 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 -- 172.21.15.96:0/1005368039 shutdown_connections 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1005368039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18441066b0 0x7f184410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1005368039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844105cf0 0x7f1844106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1005368039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1844104ba0 0x7f1844104fa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 -- 172.21.15.96:0/1005368039 >> 172.21.15.96:0/1005368039 conn(0x7f1844100440 msgr2=0x7f1844102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 -- 172.21.15.96:0/1005368039 shutdown_connections 2024-09-10T22:54:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.498+0000 7f1848f9c640 1 -- 172.21.15.96:0/1005368039 wait complete. 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.499+0000 7f1848f9c640 1 Processor -- start 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.499+0000 7f1848f9c640 1 -- start start 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 0x7f18440779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 0x7f1844078000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18441066b0 0x7f1844078c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1844110810 con 0x7f18441066b0 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1844110690 con 0x7f1844105cf0 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1848f9c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1844110990 con 0x7f1844104ba0 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f184259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 0x7f18440779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 0x7f1844078000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f184259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 0x7f18440779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35040/0 (socket says 172.21.15.96:35040) 2024-09-10T22:54:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f184259c640 1 -- 172.21.15.96:0/1120928679 learned_addr learned my addr 172.21.15.96:0/1120928679 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1842d9d640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18441066b0 0x7f1844078c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 -- 172.21.15.96:0/1120928679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 msgr2=0x7f18440779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 0x7f18440779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 -- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18441066b0 msgr2=0x7f1844078c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18441066b0 0x7f1844078c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.500+0000 7f1841d9b640 1 -- 172.21.15.96:0/1120928679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18440794f0 con 0x7f1844105cf0 2024-09-10T22:54:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f1841d9b640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 0x7f1844078000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9f0 tx=0x7f183800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f18337fe640 1 -- 172.21.15.96:0/1120928679 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800ccc0 con 0x7f1844105cf0 2024-09-10T22:54:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18440797e0 con 0x7f1844105cf0 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1844079e80 con 0x7f1844105cf0 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f18337fe640 1 -- 172.21.15.96:0/1120928679 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1838005020 con 0x7f1844105cf0 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.501+0000 7f18337fe640 1 -- 172.21.15.96:0/1120928679 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004a00 con 0x7f1844105cf0 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.503+0000 7f184be42740 1 Processor -- start 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.503+0000 7f184be42740 1 -- start start 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.503+0000 7f18337fe640 1 -- 172.21.15.96:0/1120928679 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1838004ba0 con 0x7f1844105cf0 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56162fe9d6f0 con 0x56162ff2e190 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56162fe8cb20 con 0x56162ff2b860 2024-09-10T22:54:38.509 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184be42740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56162fe90930 con 0x56162ff30bd0 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36022/0 (socket says 172.21.15.96:36022) 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 -- 172.21.15.96:0/3370482568 learned_addr learned my addr 172.21.15.96:0/3370482568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1841d9b640 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 -- 172.21.15.96:0/3370482568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 msgr2=0x56162ff33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 -- 172.21.15.96:0/3370482568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 msgr2=0x56162ff30580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff30580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 -- 172.21.15.96:0/3370482568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162fe52530 con 0x56162ff2b860 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f184259c640 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff30580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.504+0000 7f1842d9d640 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f183c009f90 tx=0x7f183c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98ede8d5f321ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.505+0000 7f1830ff9640 1 -- 172.21.15.96:0/3370482568 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c004d20 con 0x56162ff2b860 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.505+0000 7f1830ff9640 1 -- 172.21.15.96:0/3370482568 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c004050 con 0x56162ff2b860 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.505+0000 7f18337fe640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 0x7f182c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.505+0000 7f184259c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 0x7f182c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.505+0000 7f18337fe640 1 -- 172.21.15.96:0/1120928679 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f18380e5a30 con 0x7f1844105cf0 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 -- 172.21.15.96:0/3370482568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 msgr2=0x56162ff2dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f183c009f90 tx=0x7f183c0118d0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 -- 172.21.15.96:0/3370482568 shutdown_connections 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff30580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 --2- 172.21.15.96:0/3370482568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2b860 0x56162ff2dc50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.506+0000 7f184be42740 1 -- 172.21.15.96:0/3370482568 >> 172.21.15.96:0/3370482568 conn(0x56162ff269b0 msgr2=0x56162ff27740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.507+0000 7f184be42740 1 -- 172.21.15.96:0/3370482568 shutdown_connections 2024-09-10T22:54:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.507+0000 7f184259c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 0x7f182c0b7820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1834004770 tx=0x7f18340314c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.507+0000 7f184be42740 1 -- 172.21.15.96:0/3370482568 wait complete. 2024-09-10T22:54:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 -- start start 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 Processor -- start 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff36180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56162fdc4ab0 con 0x56162ff2b860 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56162fe32730 con 0x56162ff2e190 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56162ff33c50 con 0x56162ff30bd0 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184be42740 10 client.0 _ll_register_callbacks cb 0x56162fe866b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50290/0 (socket says 172.21.15.96:50290) 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 -- 172.21.15.96:0/2656643197 learned_addr learned my addr 172.21.15.96:0/2656643197 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f1842d9d640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff36180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f1841d9b640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 -- 172.21.15.96:0/2656643197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 msgr2=0x56162ff36180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff36180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 -- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 msgr2=0x56162ff355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.508+0000 7f184259c640 1 -- 172.21.15.96:0/2656643197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162fbee790 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184259c640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1834031820 tx=0x7f1834009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194f96878b093548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834035040 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f18340037f0 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183c004050 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1834031910 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 10 client.16938 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56162fc67620 con 0x56162ff2b860 2024-09-10T22:54:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c004050 con 0x56162ff2b860 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1834044070 con 0x56162ff2b860 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 10 client.16938 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 20 client.16938 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ff407e0 0x56162ff42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 4 client.16938 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.509+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162fe9f640 con 0x56162ff407e0 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.510+0000 7f1841d9b640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ff407e0 0x56162ff42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.510+0000 7f1829ffb640 10 client.16938 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.510+0000 7f1841d9b640 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ff407e0 0x56162ff42c80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e416d67f1e60b9cf server_cookie=d40c5ce09f1d2464 in_seq=0 out_seq=0 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f1829ffb640 1 -- 172.21.15.96:0/2656643197 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56162fe9f640 con 0x56162ff407e0 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f1829ffb640 10 client.16938 handle_command_reply: tid=0 2024-09-10T22:54:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 client.16938 shutdown 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 msgr2=0x56162ff34f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1834031820 tx=0x7f1834009260 comp rx=0 tx=0).stop 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 shutdown_connections 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ff407e0 0x56162ff42c80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff36180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 --2- 172.21.15.96:0/2656643197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2b860 0x56162ff34f10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f1829ffb640 0 client.16938 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 >> 172.21.15.96:0/2656643197 conn(0x56162ff269b0 msgr2=0x56162ff25bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 shutdown_connections 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 1 -- 172.21.15.96:0/2656643197 wait complete. 2024-09-10T22:54:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.511+0000 7f184be42740 20 client.16938 trim_cache size 0 max 16384 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 Processor -- start 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 -- start start 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2a110 0x56162ff2b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f182400e370 con 0x56162ff2e190 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f182400feb0 con 0x56162ff2a110 2024-09-10T22:54:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184be42740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f182400ea40 con 0x56162ff30bd0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f1841d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50296/0 (socket says 172.21.15.96:50296) 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f184259c640 1 -- 172.21.15.96:0/3874075195 learned_addr learned my addr 172.21.15.96:0/3874075195 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.529+0000 7f1842d9d640 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2a110 0x56162ff2b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 -- 172.21.15.96:0/3874075195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2a110 msgr2=0x56162ff2b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2a110 0x56162ff2b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 -- 172.21.15.96:0/3874075195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 msgr2=0x56162ff355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 -- 172.21.15.96:0/3874075195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162ff2dfc0 con 0x56162ff30bd0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.530+0000 7f1841d9b640 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff2ac30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18380abef0 tx=0x7f18380096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b90a259fae56721 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.531+0000 7f182bfff640 1 -- 172.21.15.96:0/3874075195 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183801c040 con 0x56162ff30bd0 2024-09-10T22:54:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.531+0000 7f182bfff640 1 -- 172.21.15.96:0/3874075195 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18380abd00 con 0x56162ff30bd0 2024-09-10T22:54:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.531+0000 7f182bfff640 1 -- 172.21.15.96:0/3874075195 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380cf110 con 0x56162ff30bd0 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 -- 172.21.15.96:0/3874075195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 msgr2=0x56162ff2ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff2ac30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18380abef0 tx=0x7f18380096e0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 -- 172.21.15.96:0/3874075195 shutdown_connections 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2a110 0x56162ff2b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff2ac30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 --2- 172.21.15.96:0/3874075195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2e190 0x56162ff355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 -- 172.21.15.96:0/3874075195 >> 172.21.15.96:0/3874075195 conn(0x56162ff269b0 msgr2=0x56162ff29a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 -- 172.21.15.96:0/3874075195 shutdown_connections 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.532+0000 7f184be42740 1 -- 172.21.15.96:0/3874075195 wait complete. 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 -- start start 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 Processor -- start 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 0x56162ff3c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff3c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f182400f7e0 con 0x56162ff2a110 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f182400f110 con 0x56162ff2e190 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f182400c350 con 0x56162ff30bd0 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f184be42740 10 client.0 _ll_register_callbacks cb 0x7f1834009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35074/0 (socket says 172.21.15.96:35074) 2024-09-10T22:54:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.533+0000 7f1842d9d640 1 -- 172.21.15.96:0/1173937054 learned_addr learned my addr 172.21.15.96:0/1173937054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f184259c640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 0x56162ff3c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1841d9b640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff3c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 -- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 msgr2=0x56162ff3c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff3c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 -- 172.21.15.96:0/1173937054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 msgr2=0x56162ff3c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 0x56162ff3c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 -- 172.21.15.96:0/1173937054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162ff308f0 con 0x56162ff30bd0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1841d9b640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff3c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f184259c640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 0x56162ff3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1842d9d640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f183c009f90 tx=0x7f183c009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4db28cced7394d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c030060 con 0x56162ff30bd0 2024-09-10T22:54:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18380abd00 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f183c005630 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f183c005830 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.534+0000 7f184be42740 10 client.26578 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.535+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56162ffadfa0 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18380abd00 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f183c035070 con 0x56162ff30bd0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 10 client.26578 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 10 client.26578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 20 client.26578 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ffae7a0 0x56162ffb0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 4 client.26578 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.536+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56162fe9b0f0 con 0x56162ffae7a0 2024-09-10T22:54:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.537+0000 7f184259c640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ffae7a0 0x56162ffb0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.537+0000 7f184259c640 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ffae7a0 0x56162ffb0bc0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9218e66bf91e409f server_cookie=252cfe3ac51a15d7 in_seq=0 out_seq=0 2024-09-10T22:54:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.537+0000 7f1829ffb640 10 client.26578 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f1829ffb640 1 -- 172.21.15.96:0/1173937054 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56162fe9b0f0 con 0x56162ffae7a0 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f1829ffb640 10 client.26578 handle_command_reply: tid=0 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f184be42740 1 client.26578 shutdown 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 msgr2=0x56162ff3abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f183c009f90 tx=0x7f183c009d60 comp rx=0 tx=0).stop 2024-09-10T22:54:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 shutdown_connections 2024-09-10T22:54:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.559+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56162ffae7a0 0x56162ffb0bc0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56162ff2e190 0x56162ff3c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56162ff30bd0 0x56162ff3abe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 --2- 172.21.15.96:0/1173937054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56162ff2a110 0x56162ff3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f1829ffb640 0 client.26578 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 >> 172.21.15.96:0/1173937054 conn(0x56162ff269b0 msgr2=0x56162ff253d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 shutdown_connections 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 1 -- 172.21.15.96:0/1173937054 wait complete. 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.560+0000 7f184be42740 20 client.26578 trim_cache size 0 max 16384 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:54:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:54:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:54:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 msgr2=0x7f182c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 0x7f182c0b7820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1834004770 tx=0x7f18340314c0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 msgr2=0x7f1844078000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 0x7f1844078000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9f0 tx=0x7f183800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 shutdown_connections 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1844105cf0 0x7f1844078000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f182c0b5360 0x7f182c0b7820 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18441066b0 0x7f1844078c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 --2- 172.21.15.96:0/1120928679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1844104ba0 0x7f18440779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 >> 172.21.15.96:0/1120928679 conn(0x7f1844100440 msgr2=0x7f18441090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 shutdown_connections 2024-09-10T22:54:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:38.561+0000 7f1848f9c640 1 -- 172.21.15.96:0/1120928679 wait complete. 2024-09-10T22:54:39.282 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:54:39.282 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:54:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:39 smithi096 ceph-mon[30869]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T22:54:39.653 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:39 smithi121 ceph-mon[35119]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T22:54:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:39 smithi117 ceph-mon[35081]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.215+0000 7fb6b48b7640 1 -- 172.21.15.96:0/2271496732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b010b260 msgr2=0x7fb6b010b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.215+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/2271496732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b010b260 0x7fb6b010b640 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009900 tx=0x7fb6a002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.216+0000 7fb6b48b7640 1 -- 172.21.15.96:0/2271496732 shutdown_connections 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.216+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/2271496732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010bb80 0x7fb6b01142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.216+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/2271496732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b0114810 0x7fb6b0116c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.216+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/2271496732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b010b260 0x7fb6b010b640 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.216+0000 7fb6b48b7640 1 -- 172.21.15.96:0/2271496732 >> 172.21.15.96:0/2271496732 conn(0x7fb6b006cf20 msgr2=0x7fb6b006d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:40.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.218+0000 7fb6b48b7640 1 -- 172.21.15.96:0/2271496732 shutdown_connections 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.218+0000 7fb6b48b7640 1 -- 172.21.15.96:0/2271496732 wait complete. 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.218+0000 7fb6b48b7640 1 Processor -- start 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.219+0000 7fb6b48b7640 1 -- start start 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010b260 0x7fb6b0119450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 0x7fb6b0117fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b011b470 con 0x7fb6b0114810 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b011b2f0 con 0x7fb6b010b260 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6b48b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b011b5f0 con 0x7fb6b010bb80 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6ae59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6ae59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35086/0 (socket says 172.21.15.96:35086) 2024-09-10T22:54:40.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6ae59c640 1 -- 172.21.15.96:0/1580905523 learned_addr learned my addr 172.21.15.96:0/1580905523 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6aed9d640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010b260 0x7fb6b0119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.220+0000 7fb6af59e640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 0x7fb6b0117fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 -- 172.21.15.96:0/1580905523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010b260 msgr2=0x7fb6b0119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010b260 0x7fb6b0119450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 -- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 msgr2=0x7fb6b0117fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 0x7fb6b0117fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 -- 172.21.15.96:0/1580905523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b01b4220 con 0x7fb6b010bb80 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6af59e640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 0x7fb6b0117fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6ae59c640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9c0 tx=0x7fb6a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ccc0 con 0x7fb6b010bb80 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a40035f0 con 0x7fb6b010bb80 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ce60 con 0x7fb6b010bb80 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b01b43f0 con 0x7fb6b010bb80 2024-09-10T22:54:40.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.221+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01b48b0 con 0x7fb6b010bb80 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.222+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b010b640 con 0x7fb6b010bb80 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.225+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb6a4004000 con 0x7fb6b010bb80 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.226+0000 7fb697fff640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 0x7fb6980b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.226+0000 7fb6aed9d640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 0x7fb6980b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.226+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb6a40e5fc0 con 0x7fb6b010bb80 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.226+0000 7fb6aed9d640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 0x7fb6980b78d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0031440 tx=0x7fb6a0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:40.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.227+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6a40acf10 con 0x7fb6b010bb80 2024-09-10T22:54:40.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.566+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6b0118ca0 con 0x7fb6b010bb80 2024-09-10T22:54:40.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.567+0000 7fb697fff640 1 -- 172.21.15.96:0/1580905523 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fb6a40ac810 con 0x7fb6b010bb80 2024-09-10T22:54:40.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:54:40.570 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:54:40.570 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:54:40.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 msgr2=0x7fb6980b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 0x7fb6980b78d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0031440 tx=0x7fb6a0005b50 comp rx=0 tx=0).stop 2024-09-10T22:54:40.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 msgr2=0x7fb6b01179a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:40.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e9c0 tx=0x7fb6a400ee80 comp rx=0 tx=0).stop 2024-09-10T22:54:40.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 shutdown_connections 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb6980b5410 0x7fb6980b78d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6b0114810 0x7fb6b0117fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b010bb80 0x7fb6b01179a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 --2- 172.21.15.96:0/1580905523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b010b260 0x7fb6b0119450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 >> 172.21.15.96:0/1580905523 conn(0x7fb6b006cf20 msgr2=0x7fb6b00726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 shutdown_connections 2024-09-10T22:54:40.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:40.571+0000 7fb6b48b7640 1 -- 172.21.15.96:0/1580905523 wait complete. 2024-09-10T22:54:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 425.0 s, 2486.4 tps, lat 39.963 ms stddev 44.121 2024-09-10T22:54:41.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:54:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:41 smithi096 ceph-mon[30869]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 720 op/s 2024-09-10T22:54:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:41 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1580905523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:41 smithi121 ceph-mon[35119]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 720 op/s 2024-09-10T22:54:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:41 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1580905523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:41 smithi117 ceph-mon[35081]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 720 op/s 2024-09-10T22:54:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1580905523' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- 172.21.15.96:0/2171448843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794101c90 msgr2=0x7f9794102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 --2- 172.21.15.96:0/2171448843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794101c90 0x7f9794102070 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f978c009ca0 tx=0x7f978c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- 172.21.15.96:0/2171448843 shutdown_connections 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 --2- 172.21.15.96:0/2171448843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f9794107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 --2- 172.21.15.96:0/2171448843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794108020 0x7f979410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 --2- 172.21.15.96:0/2171448843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794101c90 0x7f9794102070 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- 172.21.15.96:0/2171448843 >> 172.21.15.96:0/2171448843 conn(0x7f97940fd700 msgr2=0x7f97940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- 172.21.15.96:0/2171448843 shutdown_connections 2024-09-10T22:54:42.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- 172.21.15.96:0/2171448843 wait complete. 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 Processor -- start 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.315+0000 7f979b876640 1 -- start start 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794101c90 0x7f979410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f979410f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9794112de0 con 0x7f9794108020 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9794112c60 con 0x7f97941076a0 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f979b876640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9794112f60 con 0x7f9794101c90 2024-09-10T22:54:42.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9799e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9799e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50340/0 (socket says 172.21.15.96:50340) 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9799e13640 1 -- 172.21.15.96:0/667797180 learned_addr learned my addr 172.21.15.96:0/667797180 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f979410f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9799612640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794101c90 0x7f979410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 -- 172.21.15.96:0/667797180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794101c90 msgr2=0x7f979410f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794101c90 0x7f979410f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 -- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 msgr2=0x7f979410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 -- 172.21.15.96:0/667797180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97941a76f0 con 0x7f97941076a0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9799e13640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f9798e11640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f979410f6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f978400e9f0 tx=0x7f978400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.316+0000 7f978a7fc640 1 -- 172.21.15.96:0/667797180 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978400ccc0 con 0x7f97941076a0 2024-09-10T22:54:42.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.317+0000 7f978a7fc640 1 -- 172.21.15.96:0/667797180 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97840108e0 con 0x7f97941076a0 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.317+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97941a79e0 con 0x7f97941076a0 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.317+0000 7f978a7fc640 1 -- 172.21.15.96:0/667797180 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784018880 con 0x7f97941076a0 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.317+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97941a7f70 con 0x7f97941076a0 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f978a7fc640 1 -- 172.21.15.96:0/667797180 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9784018a20 con 0x7f97941076a0 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 Processor -- start 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 -- start start 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd3eae5ea0 con 0x55fd3eb7bd80 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd3ead52d0 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.320+0000 7f979dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd3ead90e0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799612640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36088/0 (socket says 172.21.15.96:36088) 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 -- 172.21.15.96:0/2044565866 learned_addr learned my addr 172.21.15.96:0/2044565866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9798e11640 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 -- 172.21.15.96:0/2044565866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 msgr2=0x55fd3eb7b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 -- 172.21.15.96:0/2044565866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 msgr2=0x55fd3eb7e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb7e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 -- 172.21.15.96:0/2044565866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3e833650 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.321+0000 7f9799e13640 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9790011570 tx=0x7f9790011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2191e1ed5ac83b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f978a7fc640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 0x7f97800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f977f7fe640 1 -- 172.21.15.96:0/2044565866 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005020 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f977f7fe640 1 -- 172.21.15.96:0/2044565866 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97900047a0 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f977f7fe640 1 -- 172.21.15.96:0/2044565866 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000f900 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f978a7fc640 1 -- 172.21.15.96:0/667797180 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f97840e5e30 con 0x7f97941076a0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.322+0000 7f9799612640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 0x7f97800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 -- 172.21.15.96:0/2044565866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 msgr2=0x55fd3eb6f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9790011570 tx=0x7f9790011a30 comp rx=0 tx=0).stop 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 -- 172.21.15.96:0/2044565866 shutdown_connections 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb7e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2044565866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6f8b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f979dc4b740 1 -- 172.21.15.96:0/2044565866 >> 172.21.15.96:0/2044565866 conn(0x55fd3eb6d300 msgr2=0x55fd3eb6e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.323+0000 7f9799612640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 0x7f97800b7820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55fd3eb6e1e0 tx=0x7f978c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.324+0000 7f979dc4b740 1 -- 172.21.15.96:0/2044565866 shutdown_connections 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.324+0000 7f979dc4b740 1 -- 172.21.15.96:0/2044565866 wait complete. 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.324+0000 7f979dc4b740 1 -- start start 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.324+0000 7f979dc4b740 1 Processor -- start 2024-09-10T22:54:42.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd3e936700 con 0x55fd3eb7bd80 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fd3ea70160 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd3eb7ea00 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 10 client.0 _ll_register_callbacks cb 0x55fd3ea899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36094/0 (socket says 172.21.15.96:36094) 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 -- 172.21.15.96:0/798528716 learned_addr learned my addr 172.21.15.96:0/798528716 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:42.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9799612640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9799e13640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 -- 172.21.15.96:0/798528716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 msgr2=0x55fd3eb7b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 -- 172.21.15.96:0/798528716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 msgr2=0x55fd3eb78fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb78fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 -- 172.21.15.96:0/798528716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3ea9dc90 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f9798e11640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97840ac6d0 tx=0x7f978400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6867f647e115bc27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f977cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840ac510 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9784010430 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.325+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97900047a0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97840abeb0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 10 client.26546 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd3eb8a310 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97900047a0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f97840e9ae0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 10 client.26546 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 10 client.26546 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:54:42.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 20 client.26546 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3eb8dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 4 client.26546 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd3eae7df0 con 0x55fd3eb8b7d0 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.326+0000 7f9799612640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3eb8dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.327+0000 7f9799612640 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3eb8dbc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0b3f41089ebb04d server_cookie=e9ff9e5af1f01977 in_seq=0 out_seq=0 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.327+0000 7f977cff9640 10 client.26546 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.327+0000 7f977cff9640 1 -- 172.21.15.96:0/798528716 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd3eae7df0 con 0x55fd3eb8b7d0 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.327+0000 7f977cff9640 10 client.26546 handle_command_reply: tid=0 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.327+0000 7f979dc4b740 1 client.26546 shutdown 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 msgr2=0x55fd3eb78a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97840ac6d0 tx=0x7f978400cef0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 shutdown_connections 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3eb8dbc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f977cff9640 0 client.26546 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb78a30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb78fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 --2- 172.21.15.96:0/798528716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 >> 172.21.15.96:0/798528716 conn(0x55fd3eb6d300 msgr2=0x55fd3eb72e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 shutdown_connections 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 1 -- 172.21.15.96:0/798528716 wait complete. 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.328+0000 7f979dc4b740 20 client.26546 trim_cache size 0 max 16384 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 Processor -- start 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 -- start start 2024-09-10T22:54:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb75c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97941b0450 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97941b1f90 con 0x55fd3eb7bd80 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f979dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97941b0b20 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9798e11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9798e11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50362/0 (socket says 172.21.15.96:50362) 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9798e11640 1 -- 172.21.15.96:0/2628424521 learned_addr learned my addr 172.21.15.96:0/2628424521 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb75c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799612640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 -- 172.21.15.96:0/2628424521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 msgr2=0x55fd3eb6cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 -- 172.21.15.96:0/2628424521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 msgr2=0x55fd3eb7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799e13640 1 -- 172.21.15.96:0/2628424521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3eb7e4e0 con 0x55fd3eb7bd80 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.345+0000 7f9799612640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.346+0000 7f9799e13640 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb75c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f979001b040 tx=0x7f9790009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51609e6b770027a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.346+0000 7f977effd640 1 -- 172.21.15.96:0/2628424521 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005200 con 0x55fd3eb7bd80 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.346+0000 7f977effd640 1 -- 172.21.15.96:0/2628424521 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97900053a0 con 0x55fd3eb7bd80 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.346+0000 7f977effd640 1 -- 172.21.15.96:0/2628424521 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000f9c0 con 0x55fd3eb7bd80 2024-09-10T22:54:42.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 -- 172.21.15.96:0/2628424521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 msgr2=0x55fd3eb75c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb75c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f979001b040 tx=0x7f9790009c50 comp rx=0 tx=0).stop 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 -- 172.21.15.96:0/2628424521 shutdown_connections 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3eb75c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 --2- 172.21.15.96:0/2628424521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 -- 172.21.15.96:0/2628424521 >> 172.21.15.96:0/2628424521 conn(0x55fd3eb6d300 msgr2=0x55fd3eb7a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 -- 172.21.15.96:0/2628424521 shutdown_connections 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.347+0000 7f979dc4b740 1 -- 172.21.15.96:0/2628424521 wait complete. 2024-09-10T22:54:42.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.348+0000 7f979dc4b740 1 -- start start 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.348+0000 7f979dc4b740 1 Processor -- start 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.348+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.348+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3ebf75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9799612640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97941b18c0 con 0x55fd3eb6f4d0 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97941b11f0 con 0x55fd3eb7bd80 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97941ae450 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9799612640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50366/0 (socket says 172.21.15.96:50366) 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9799612640 1 -- 172.21.15.96:0/3171954406 learned_addr learned my addr 172.21.15.96:0/3171954406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:42.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 10 client.0 _ll_register_callbacks cb 0x7f97840e9640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9799e13640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3ebf75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 -- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 msgr2=0x55fd3ebf75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3ebf75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 -- 172.21.15.96:0/3171954406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 msgr2=0x55fd3eb7ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 -- 172.21.15.96:0/3171954406 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3ebf7b00 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9799e13640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3ebf75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:42.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f9798e11640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb804f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f978400e9c0 tx=0x7f97840106d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce7b2d84b7f8feb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f977cff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840ac090 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f97840e9ac0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.349+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97900053a0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97840ac510 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 10 client.26602 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd3ebf8bf0 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97840ac740 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f97840ab410 con 0x55fd3eb6fdf0 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 10 client.26602 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 10 client.26602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 20 client.26602 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:42.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3ebfb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 4 client.26602 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f97900053a0 con 0x55fd3eb8b7d0 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.350+0000 7f9799612640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3ebfb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f977cff9640 10 client.26602 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f9799612640 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3ebfb3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec8fdb4c7ed14e67 server_cookie=bd2897f37380561b in_seq=0 out_seq=0 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f977cff9640 1 -- 172.21.15.96:0/3171954406 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f97900053a0 con 0x55fd3eb8b7d0 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f977cff9640 10 client.26602 handle_command_reply: tid=0 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 client.26602 shutdown 2024-09-10T22:54:42.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 msgr2=0x55fd3eb804f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb804f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f978400e9c0 tx=0x7f97840106d0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 shutdown_connections 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55fd3eb8b7d0 0x55fd3ebfb3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3eb7bd80 0x55fd3ebf75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3eb6fdf0 0x55fd3eb804f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 --2- 172.21.15.96:0/3171954406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd3eb6f4d0 0x55fd3eb7ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f977cff9640 0 client.26602 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 >> 172.21.15.96:0/3171954406 conn(0x55fd3eb6d300 msgr2=0x55fd3eb7a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 shutdown_connections 2024-09-10T22:54:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 1 -- 172.21.15.96:0/3171954406 wait complete. 2024-09-10T22:54:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.351+0000 7f979dc4b740 20 client.26602 trim_cache size 0 max 16384 2024-09-10T22:54:42.621 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 msgr2=0x7f97800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 0x7f97800b7820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55fd3eb6e1e0 tx=0x7f978c009360 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 msgr2=0x7f979410f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f979410f6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f978400e9f0 tx=0x7f978400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 shutdown_connections 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97941076a0 0x7f979410f6c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f97800b5360 0x7f97800b7820 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9794108020 0x7f979410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 --2- 172.21.15.96:0/667797180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9794101c90 0x7f979410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.352+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 >> 172.21.15.96:0/667797180 conn(0x7f97940fd700 msgr2=0x7f97940ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.354+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 shutdown_connections 2024-09-10T22:54:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:42.354+0000 7f979b876640 1 -- 172.21.15.96:0/667797180 wait complete. 2024-09-10T22:54:43.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:54:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:43 smithi096 ceph-mon[30869]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-10T22:54:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:43 smithi121 ceph-mon[35119]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-10T22:54:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:43 smithi117 ceph-mon[35081]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-09-10T22:54:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:44.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.704+0000 7fad70c62640 1 -- 172.21.15.96:0/3665091731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 msgr2=0x7fad6c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.704+0000 7fad70c62640 1 --2- 172.21.15.96:0/3665091731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c107b00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fad54009900 tx=0x7fad5402a410 comp rx=0 tx=0).stop 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 -- 172.21.15.96:0/3665091731 shutdown_connections 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 --2- 172.21.15.96:0/3665091731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 --2- 172.21.15.96:0/3665091731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c107b00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 --2- 172.21.15.96:0/3665091731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 0x7fad6c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 -- 172.21.15.96:0/3665091731 >> 172.21.15.96:0/3665091731 conn(0x7fad6c0fd700 msgr2=0x7fad6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 -- 172.21.15.96:0/3665091731 shutdown_connections 2024-09-10T22:54:44.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 -- 172.21.15.96:0/3665091731 wait complete. 2024-09-10T22:54:44.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.705+0000 7fad70c62640 1 Processor -- start 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 -- start start 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 0x7fad6c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c111d00 con 0x7fad6c1076c0 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c111b80 con 0x7fad6c108040 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad70c62640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c111e80 con 0x7fad6c101cb0 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50392/0 (socket says 172.21.15.96:50392) 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad63fff640 1 -- 172.21.15.96:0/1595526917 learned_addr learned my addr 172.21.15.96:0/1595526917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:44.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6a59c640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 0x7fad6c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 -- 172.21.15.96:0/1595526917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 msgr2=0x7fad6c0fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 0x7fad6c0fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 -- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 msgr2=0x7fad6c0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad6ad9d640 1 -- 172.21.15.96:0/1595526917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c1a76a0 con 0x7fad6c108040 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.706+0000 7fad63fff640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c0fe680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fad64009fd0 tx=0x7fad6400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad637fe640 1 -- 172.21.15.96:0/1595526917 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6400ce20 con 0x7fad6c108040 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1a7990 con 0x7fad6c108040 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad637fe640 1 -- 172.21.15.96:0/1595526917 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad64005020 con 0x7fad6c108040 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad637fe640 1 -- 172.21.15.96:0/1595526917 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64010690 con 0x7fad6c108040 2024-09-10T22:54:44.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.707+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1a7e50 con 0x7fad6c108040 2024-09-10T22:54:44.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.710+0000 7fad73037740 1 Processor -- start 2024-09-10T22:54:44.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.711+0000 7fad637fe640 1 -- 172.21.15.96:0/1595526917 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fad640054d0 con 0x7fad6c108040 2024-09-10T22:54:44.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 -- start start 2024-09-10T22:54:44.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 0x5560dc6048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc613170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560dc57aea0 con 0x5560dc6044d0 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5560dc56a2d0 con 0x5560dc604df0 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad73037740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560dc56e0e0 con 0x5560dc610d80 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 0x5560dc6048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36130/0 (socket says 172.21.15.96:36130) 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 -- 172.21.15.96:0/4172972729 learned_addr learned my addr 172.21.15.96:0/4172972729 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 0x5560dc6048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50408/0 (socket says 172.21.15.96:50408) 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad63fff640 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc613170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 -- 172.21.15.96:0/4172972729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 msgr2=0x5560dc613170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc613170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 -- 172.21.15.96:0/4172972729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 msgr2=0x5560dc6048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:44.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 0x5560dc6048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad637fe640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 0x7fad580b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.712+0000 7fad6a59c640 1 -- 172.21.15.96:0/4172972729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dc2c8650 con 0x5560dc604df0 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad63fff640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 0x7fad580b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad637fe640 1 -- 172.21.15.96:0/1595526917 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fad640e61c0 con 0x7fad6c108040 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad6a59c640 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009cf0 tx=0x7fad5c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a421178a609f216f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad60ff9640 1 -- 172.21.15.96:0/4172972729 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c01e070 con 0x5560dc604df0 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad60ff9640 1 -- 172.21.15.96:0/4172972729 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c009de0 con 0x5560dc604df0 2024-09-10T22:54:44.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.713+0000 7fad60ff9640 1 -- 172.21.15.96:0/4172972729 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0196b0 con 0x5560dc604df0 2024-09-10T22:54:44.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- 172.21.15.96:0/4172972729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 msgr2=0x5560dc610840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad63fff640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 0x7fad580b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad54031440 tx=0x7fad54041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009cf0 tx=0x7fad5c011810 comp rx=0 tx=0).stop 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- 172.21.15.96:0/4172972729 shutdown_connections 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc604df0 0x5560dc610840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc613170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 --2- 172.21.15.96:0/4172972729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc6044d0 0x5560dc6048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- 172.21.15.96:0/4172972729 >> 172.21.15.96:0/4172972729 conn(0x5560dc602300 msgr2=0x5560dc6034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- 172.21.15.96:0/4172972729 shutdown_connections 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- 172.21.15.96:0/4172972729 wait complete. 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 -- start start 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.714+0000 7fad73037740 1 Processor -- start 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc610650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc60daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560dc3cb700 con 0x5560dc604df0 2024-09-10T22:54:44.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5560dc505160 con 0x5560dc6044d0 2024-09-10T22:54:44.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5560dc613a00 con 0x5560dc610d80 2024-09-10T22:54:44.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35150/0 (socket says 172.21.15.96:35150) 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 -- 172.21.15.96:0/2448810206 learned_addr learned my addr 172.21.15.96:0/2448810206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad73037740 10 client.0 _ll_register_callbacks cb 0x5560dc51e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad63fff640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc60daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 -- 172.21.15.96:0/2448810206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 msgr2=0x5560dc610650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc610650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 -- 172.21.15.96:0/2448810206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 msgr2=0x5560dc60daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc60daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad6ad9d640 1 -- 172.21.15.96:0/2448810206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dc532c90 con 0x5560dc610d80 2024-09-10T22:54:44.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.715+0000 7fad63fff640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc60daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad640a8ff0 tx=0x7fad640ac530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fbade7acbdda2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640b1300 con 0x5560dc610d80 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fad640b5b20 con 0x5560dc610d80 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5c009de0 con 0x5560dc610d80 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad640aca20 con 0x5560dc610d80 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad73037740 10 client.26614 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560dc61f370 con 0x5560dc610d80 2024-09-10T22:54:44.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c009de0 con 0x5560dc610d80 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.716+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fad640ab6f0 con 0x5560dc610d80 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 10 client.26614 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 10 client.26614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 20 client.26614 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc622c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 4 client.26614 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560dc57cdf0 con 0x5560dc6207d0 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad6a59c640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc622c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad6a59c640 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc622c30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee66b5f478d07cda server_cookie=816495fcdfd5e0c1 in_seq=0 out_seq=0 2024-09-10T22:54:44.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.717+0000 7fad51ffb640 10 client.26614 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:54:44.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad51ffb640 1 -- 172.21.15.96:0/2448810206 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560dc57cdf0 con 0x5560dc6207d0 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad51ffb640 10 client.26614 handle_command_reply: tid=0 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 client.26614 shutdown 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 msgr2=0x5560dc60e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad640a8ff0 tx=0x7fad640ac530 comp rx=0 tx=0).stop 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 shutdown_connections 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc622c30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc610d80 0x5560dc60e080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad51ffb640 0 client.26614 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc60daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 --2- 172.21.15.96:0/2448810206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc610650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.718+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 >> 172.21.15.96:0/2448810206 conn(0x5560dc602300 msgr2=0x5560dc607ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.719+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 shutdown_connections 2024-09-10T22:54:44.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.719+0000 7fad73037740 1 -- 172.21.15.96:0/2448810206 wait complete. 2024-09-10T22:54:44.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.719+0000 7fad73037740 20 client.26614 trim_cache size 0 max 16384 2024-09-10T22:54:44.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.742+0000 7fad73037740 1 Processor -- start 2024-09-10T22:54:44.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 -- start start 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc603920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc604df0 0x5560dc601c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b0390 con 0x5560dc610d80 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b1ed0 con 0x5560dc6044d0 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad73037740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b0a60 con 0x5560dc604df0 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad6ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50412/0 (socket says 172.21.15.96:50412) 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.743+0000 7fad6ad9d640 1 -- 172.21.15.96:0/3552524465 learned_addr learned my addr 172.21.15.96:0/3552524465 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:44.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.744+0000 7fad6ad9d640 1 -- 172.21.15.96:0/3552524465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc604df0 msgr2=0x5560dc601c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:44.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.744+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc604df0 0x5560dc601c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.744+0000 7fad6ad9d640 1 -- 172.21.15.96:0/3552524465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 msgr2=0x5560dc603920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:44.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.744+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc603920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.744+0000 7fad6ad9d640 1 -- 172.21.15.96:0/3552524465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560dc604c20 con 0x5560dc610d80 2024-09-10T22:54:44.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.747+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fad640ac720 tx=0x7fad640b2750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4a8f1e2acd7c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.749+0000 7fad53fff640 1 -- 172.21.15.96:0/3552524465 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640b1500 con 0x5560dc610d80 2024-09-10T22:54:44.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.749+0000 7fad53fff640 1 -- 172.21.15.96:0/3552524465 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad640cc070 con 0x5560dc610d80 2024-09-10T22:54:44.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.749+0000 7fad53fff640 1 -- 172.21.15.96:0/3552524465 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad640ab6f0 con 0x5560dc610d80 2024-09-10T22:54:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.759+0000 7fad73037740 1 -- 172.21.15.96:0/3552524465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 msgr2=0x5560dc612170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.759+0000 7fad73037740 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fad640ac720 tx=0x7fad640b2750 comp rx=0 tx=0).stop 2024-09-10T22:54:44.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.764+0000 7fad73037740 1 -- 172.21.15.96:0/3552524465 shutdown_connections 2024-09-10T22:54:44.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.764+0000 7fad73037740 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc610d80 0x5560dc612170 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.764+0000 7fad73037740 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc604df0 0x5560dc601c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.764+0000 7fad73037740 1 --2- 172.21.15.96:0/3552524465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc6044d0 0x5560dc603920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.764+0000 7fad73037740 1 -- 172.21.15.96:0/3552524465 >> 172.21.15.96:0/3552524465 conn(0x5560dc602300 msgr2=0x5560dc603210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.772+0000 7fad73037740 1 -- 172.21.15.96:0/3552524465 shutdown_connections 2024-09-10T22:54:44.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.772+0000 7fad73037740 1 -- 172.21.15.96:0/3552524465 wait complete. 2024-09-10T22:54:44.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.773+0000 7fad73037740 1 -- start start 2024-09-10T22:54:44.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.773+0000 7fad73037740 1 Processor -- start 2024-09-10T22:54:44.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc619640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc610d80 0x5560dc613ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad6a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35162/0 (socket says 172.21.15.96:35162) 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad6a59c640 1 -- 172.21.15.96:0/914849126 learned_addr learned my addr 172.21.15.96:0/914849126 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b1800 con 0x5560dc604df0 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1b1130 con 0x5560dc610d80 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1ae390 con 0x5560dc6044d0 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.774+0000 7fad6ad9d640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc610d80 0x5560dc613ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad73037740 10 client.0 _ll_register_callbacks cb 0x7fad640ac190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 -- 172.21.15.96:0/914849126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc610d80 msgr2=0x5560dc613ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc610d80 0x5560dc613ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 -- 172.21.15.96:0/914849126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 msgr2=0x5560dc619640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc619640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560dc6146d0 con 0x5560dc6044d0 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad6a59c640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad5c003bf0 tx=0x7fad5c0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc7f0ac81f52cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:44.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.775+0000 7fad51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:44.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c01e040 con 0x5560dc6044d0 2024-09-10T22:54:44.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad640cc070 con 0x5560dc6044d0 2024-09-10T22:54:44.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fad5c0035f0 con 0x5560dc6044d0 2024-09-10T22:54:44.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad5c0037f0 con 0x5560dc6044d0 2024-09-10T22:54:44.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad73037740 10 client.26617 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:44.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.776+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560dc696bf0 con 0x5560dc6044d0 2024-09-10T22:54:44.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad640cc070 con 0x5560dc6044d0 2024-09-10T22:54:44.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fad5c035050 con 0x5560dc6044d0 2024-09-10T22:54:44.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 10 client.26617 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:44.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 10 client.26617 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:54:44.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 20 client.26617 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc699320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 4 client.26617 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560dc5788a0 con 0x5560dc6207d0 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.780+0000 7fad63fff640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc699320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.781+0000 7fad63fff640 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc699320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6a36e1dac9a9b54 server_cookie=5ed04306fa815ce0 in_seq=0 out_seq=0 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.781+0000 7fad51ffb640 10 client.26617 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:54:44.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.781+0000 7fad51ffb640 1 -- 172.21.15.96:0/914849126 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560dc5788a0 con 0x5560dc6207d0 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.781+0000 7fad51ffb640 10 client.26617 handle_command_reply: tid=0 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.781+0000 7fad73037740 1 client.26617 shutdown 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 msgr2=0x5560dc6190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad5c003bf0 tx=0x7fad5c0096d0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 shutdown_connections 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5560dc6207d0 0x5560dc699320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5560dc610d80 0x5560dc613ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560dc604df0 0x5560dc619640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 --2- 172.21.15.96:0/914849126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5560dc6044d0 0x5560dc6190c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 >> 172.21.15.96:0/914849126 conn(0x5560dc602300 msgr2=0x5560dc60f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad51ffb640 0 client.26617 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 shutdown_connections 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 1 -- 172.21.15.96:0/914849126 wait complete. 2024-09-10T22:54:44.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.787+0000 7fad73037740 20 client.26617 trim_cache size 0 max 16384 2024-09-10T22:54:44.794 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:44.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 msgr2=0x7fad580b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 0x7fad580b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fad54031440 tx=0x7fad54041040 comp rx=0 tx=0).stop 2024-09-10T22:54:44.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 msgr2=0x7fad6c0fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:44.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c0fe680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fad64009fd0 tx=0x7fad6400ef10 comp rx=0 tx=0).stop 2024-09-10T22:54:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 shutdown_connections 2024-09-10T22:54:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6c108040 0x7fad6c0fe680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fad580b52e0 0x7fad580b77a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad6c1076c0 0x7fad6c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 --2- 172.21.15.96:0/1595526917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad6c101cb0 0x7fad6c0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:44.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 >> 172.21.15.96:0/1595526917 conn(0x7fad6c0fd700 msgr2=0x7fad6c06b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:44.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 shutdown_connections 2024-09-10T22:54:44.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:44.796+0000 7fad70c62640 1 -- 172.21.15.96:0/1595526917 wait complete. 2024-09-10T22:54:45.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:54:45.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:45 smithi096 ceph-mon[30869]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-10T22:54:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:45 smithi121 ceph-mon[35119]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-10T22:54:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:45 smithi117 ceph-mon[35081]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-10T22:54:45.931 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 430.0 s, 1916.2 tps, lat 52.110 ms stddev 56.642 2024-09-10T22:54:46.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 -- 172.21.15.96:0/3887289940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 msgr2=0x7fc75c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 --2- 172.21.15.96:0/3887289940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c10f940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc754009ca0 tx=0x7fc75402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 -- 172.21.15.96:0/3887289940 shutdown_connections 2024-09-10T22:54:46.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 --2- 172.21.15.96:0/3887289940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 --2- 172.21.15.96:0/3887289940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c10f940 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 --2- 172.21.15.96:0/3887289940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 -- 172.21.15.96:0/3887289940 >> 172.21.15.96:0/3887289940 conn(0x7fc75c0fe860 msgr2=0x7fc75c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 -- 172.21.15.96:0/3887289940 shutdown_connections 2024-09-10T22:54:46.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.404+0000 7fc762491640 1 -- 172.21.15.96:0/3887289940 wait complete. 2024-09-10T22:54:46.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 Processor -- start 2024-09-10T22:54:46.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 -- start start 2024-09-10T22:54:46.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c112e00 con 0x7fc75c109140 2024-09-10T22:54:46.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c112c80 con 0x7fc75c1087c0 2024-09-10T22:54:46.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.405+0000 7fc762491640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c112f80 con 0x7fc75c102db0 2024-09-10T22:54:46.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc753fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38476/0 (socket says 172.21.15.96:38476) 2024-09-10T22:54:46.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35658/0 (socket says 172.21.15.96:35658) 2024-09-10T22:54:46.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 -- 172.21.15.96:0/670864053 learned_addr learned my addr 172.21.15.96:0/670864053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:46.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 -- 172.21.15.96:0/670864053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 msgr2=0x7fc75c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 -- 172.21.15.96:0/670864053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 msgr2=0x7fc75c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 -- 172.21.15.96:0/670864053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c19f2e0 con 0x7fc75c109140 2024-09-10T22:54:46.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc753fff640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:46.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc760a2e640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc754005cc0 tx=0x7fc754031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc759ffb640 1 -- 172.21.15.96:0/670864053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754003b70 con 0x7fc75c109140 2024-09-10T22:54:46.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc759ffb640 1 -- 172.21.15.96:0/670864053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7540318f0 con 0x7fc75c109140 2024-09-10T22:54:46.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c19f570 con 0x7fc75c109140 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.406+0000 7fc759ffb640 1 -- 172.21.15.96:0/670864053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75402f950 con 0x7fc75c109140 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.407+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1a5c90 con 0x7fc75c109140 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc759ffb640 1 -- 172.21.15.96:0/670864053 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc75402fb30 con 0x7fc75c109140 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 Processor -- start 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 -- start start 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fb44d0 0x564a41fb48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fc0d80 0x564a41fc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564a41f2aea0 con 0x564a41fb4df0 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a41f1a2d0 con 0x564a41fb44d0 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc764866740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a41f1e0e0 con 0x564a41fc0d80 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc75bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38480/0 (socket says 172.21.15.96:38480) 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc75bfff640 1 -- 172.21.15.96:0/263563103 learned_addr learned my addr 172.21.15.96:0/263563103 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.409+0000 7fc753fff640 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fb44d0 0x564a41fb48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc753fff640 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fc0d80 0x564a41fc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 -- 172.21.15.96:0/263563103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fc0d80 msgr2=0x564a41fc3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fc0d80 0x564a41fc3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 -- 172.21.15.96:0/263563103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fb44d0 msgr2=0x564a41fb48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fb44d0 0x564a41fb48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 -- 172.21.15.96:0/263563103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a41c78650 con 0x564a41fb4df0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc759ffb640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 0x7fc7480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc75bfff640 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d20 tx=0x7fc74c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f36ca232f4e7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc753fff640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 0x7fc7480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc759ffb640 1 -- 172.21.15.96:0/670864053 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc754106c30 con 0x7fc75c109140 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc7527fc640 1 -- 172.21.15.96:0/263563103 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c01e070 con 0x564a41fb4df0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc7527fc640 1 -- 172.21.15.96:0/263563103 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74c009e10 con 0x564a41fb4df0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.410+0000 7fc7527fc640 1 -- 172.21.15.96:0/263563103 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0187a0 con 0x564a41fb4df0 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 -- 172.21.15.96:0/263563103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 msgr2=0x564a41fc0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d20 tx=0x7fc74c011810 comp rx=0 tx=0).stop 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 -- 172.21.15.96:0/263563103 shutdown_connections 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fc0d80 0x564a41fc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fc0840 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 --2- 172.21.15.96:0/263563103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fb44d0 0x564a41fb48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.411+0000 7fc764866740 1 -- 172.21.15.96:0/263563103 >> 172.21.15.96:0/263563103 conn(0x564a41fb2300 msgr2=0x564a41fb34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.412+0000 7fc753fff640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 0x7fc7480b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc75c108280 tx=0x7fc744004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.412+0000 7fc764866740 1 -- 172.21.15.96:0/263563103 shutdown_connections 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.412+0000 7fc764866740 1 -- 172.21.15.96:0/263563103 wait complete. 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.412+0000 7fc764866740 1 -- start start 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 Processor -- start 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fc05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fb99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564a41d7b700 con 0x564a41fb44d0 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a41eb5160 con 0x564a41fc0d80 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a41fc3880 con 0x564a41fb4df0 2024-09-10T22:54:46.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc764866740 10 client.0 _ll_register_callbacks cb 0x564a41ece9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc753fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fb99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50134/0 (socket says 172.21.15.96:50134) 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 -- 172.21.15.96:0/1776840421 learned_addr learned my addr 172.21.15.96:0/1776840421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc75bfff640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fc05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 -- 172.21.15.96:0/1776840421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 msgr2=0x564a41fb99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fb99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 -- 172.21.15.96:0/1776840421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 msgr2=0x564a41fc05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fc05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.413+0000 7fc760a2e640 1 -- 172.21.15.96:0/1776840421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a41ee2c90 con 0x564a41fc0d80 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc7540047f0 tx=0x7fc7540ccfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ed14d2f04780 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc73b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:46.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540cd490 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc7540cd6f0 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74c009e10 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7540d1f70 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc764866740 10 client.26576 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564a41fcf470 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.414+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc754036070 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc7540d6070 con 0x564a41fc0d80 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 10 client.26576 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 10 client.26576 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 20 client.26576 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a41fd0860 0x564a41fd2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 4 client.26576 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:46.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc74c009e10 con 0x564a41fd0860 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc75bfff640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a41fd0860 0x564a41fd2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc73b7fe640 10 client.26576 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.415+0000 7fc75bfff640 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a41fd0860 0x564a41fd2c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d8827bc35b25887 server_cookie=79b84342f9b07f77 in_seq=0 out_seq=0 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1776840421 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc74c009e10 con 0x564a41fd0860 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc73b7fe640 10 client.26576 handle_command_reply: tid=0 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 client.26576 shutdown 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 msgr2=0x564a41fb9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc7540047f0 tx=0x7fc7540ccfa0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 shutdown_connections 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a41fd0860 0x564a41fd2c90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fb9f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fb99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc73b7fe640 0 client.26576 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:54:46.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 --2- 172.21.15.96:0/1776840421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fc05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 >> 172.21.15.96:0/1776840421 conn(0x564a41fb2300 msgr2=0x564a41fb7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 shutdown_connections 2024-09-10T22:54:46.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 1 -- 172.21.15.96:0/1776840421 wait complete. 2024-09-10T22:54:46.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.416+0000 7fc764866740 20 client.26576 trim_cache size 0 max 16384 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 Processor -- start 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 -- start start 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb44d0 0x564a41fb8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fb8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b4760 con 0x564a41fb4df0 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b62a0 con 0x564a41fc0d80 2024-09-10T22:54:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc764866740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b4e30 con 0x564a41fb44d0 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50144/0 (socket says 172.21.15.96:50144) 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.440+0000 7fc760a2e640 1 -- 172.21.15.96:0/2717064078 learned_addr learned my addr 172.21.15.96:0/2717064078 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fb8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 -- 172.21.15.96:0/2717064078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb44d0 msgr2=0x564a41fb8780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb44d0 0x564a41fb8780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 -- 172.21.15.96:0/2717064078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 msgr2=0x564a41fc5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.441+0000 7fc753fff640 1 -- 172.21.15.96:0/2717064078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a41fc3ac0 con 0x564a41fb4df0 2024-09-10T22:54:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.442+0000 7fc753fff640 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fb8cc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f950 tx=0x7fc744018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9562a60b297e04f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.450+0000 7fc751ffb640 1 -- 172.21.15.96:0/2717064078 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74400fd10 con 0x564a41fb4df0 2024-09-10T22:54:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.450+0000 7fc751ffb640 1 -- 172.21.15.96:0/2717064078 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74402e070 con 0x564a41fb4df0 2024-09-10T22:54:46.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.450+0000 7fc751ffb640 1 -- 172.21.15.96:0/2717064078 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744003d40 con 0x564a41fb4df0 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.451+0000 7fc764866740 1 -- 172.21.15.96:0/2717064078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 msgr2=0x564a41fb8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.451+0000 7fc764866740 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fb8cc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f950 tx=0x7fc744018000 comp rx=0 tx=0).stop 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.452+0000 7fc764866740 1 -- 172.21.15.96:0/2717064078 shutdown_connections 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.452+0000 7fc764866740 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.452+0000 7fc764866740 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb4df0 0x564a41fb8cc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.452+0000 7fc764866740 1 --2- 172.21.15.96:0/2717064078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb44d0 0x564a41fb8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.452+0000 7fc764866740 1 -- 172.21.15.96:0/2717064078 >> 172.21.15.96:0/2717064078 conn(0x564a41fb2300 msgr2=0x564a41fc2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.454+0000 7fc764866740 1 -- 172.21.15.96:0/2717064078 shutdown_connections 2024-09-10T22:54:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.455+0000 7fc764866740 1 -- 172.21.15.96:0/2717064078 wait complete. 2024-09-10T22:54:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.455+0000 7fc764866740 1 -- start start 2024-09-10T22:54:46.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.456+0000 7fc764866740 1 Processor -- start 2024-09-10T22:54:46.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fcd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fc7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b5bd0 con 0x564a41fb44d0 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b5500 con 0x564a41fc0d80 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b2780 con 0x564a41fb4df0 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc764866740 10 client.0 _ll_register_callbacks cb 0x7fc754009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50156/0 (socket says 172.21.15.96:50156) 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 -- 172.21.15.96:0/1080756337 learned_addr learned my addr 172.21.15.96:0/1080756337 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc753fff640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fc7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 -- 172.21.15.96:0/1080756337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 msgr2=0x564a41fc7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fc7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 -- 172.21.15.96:0/1080756337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 msgr2=0x564a41fcd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fcd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.457+0000 7fc760a2e640 1 -- 172.21.15.96:0/1080756337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a41fc8a60 con 0x564a41fc0d80 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc760a2e640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7540ca700 tx=0x7fc7540d67c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13a4bebefb62724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc73b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540d6360 con 0x564a41fc0d80 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc754004b40 con 0x564a41fc0d80 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74402e070 con 0x564a41fc0d80 2024-09-10T22:54:46.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7540cd490 con 0x564a41fc0d80 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc764866740 10 client.26582 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.458+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564a4204f830 con 0x564a41fc0d80 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.459+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74402e070 con 0x564a41fc0d80 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.459+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc7540d6040 con 0x564a41fc0d80 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 10 client.26582 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 10 client.26582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 20 client.26582 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a4204fdf0 0x564a42052210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:46.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 4 client.26582 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:46.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a41f2cdf0 con 0x564a4204fdf0 2024-09-10T22:54:46.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.460+0000 7fc75bfff640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a4204fdf0 0x564a42052210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.462+0000 7fc75bfff640 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a4204fdf0 0x564a42052210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62f7833d43b5d464 server_cookie=9ecf828e17cd1723 in_seq=0 out_seq=0 2024-09-10T22:54:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.462+0000 7fc73b7fe640 10 client.26582 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:54:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.462+0000 7fc73b7fe640 1 -- 172.21.15.96:0/1080756337 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a41f2cdf0 con 0x564a4204fdf0 2024-09-10T22:54:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.462+0000 7fc73b7fe640 10 client.26582 handle_command_reply: tid=0 2024-09-10T22:54:46.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.462+0000 7fc764866740 1 client.26582 shutdown 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 msgr2=0x564a41fc8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7540ca700 tx=0x7fc7540d67c0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 shutdown_connections 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x564a4204fdf0 0x564a42052210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a41fc0d80 0x564a41fc8230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a41fb4df0 0x564a41fc7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 --2- 172.21.15.96:0/1080756337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564a41fb44d0 0x564a41fcd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 >> 172.21.15.96:0/1080756337 conn(0x564a41fb2300 msgr2=0x564a41fb8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc73b7fe640 0 client.26582 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 shutdown_connections 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.463+0000 7fc764866740 1 -- 172.21.15.96:0/1080756337 wait complete. 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc764866740 20 client.26582 trim_cache size 0 max 16384 2024-09-10T22:54:46.466 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 msgr2=0x7fc7480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 0x7fc7480b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc75c108280 tx=0x7fc744004730 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 msgr2=0x7fc75c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc754005cc0 tx=0x7fc754031700 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 shutdown_connections 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.464+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc75c1087c0 0x7fc75c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7480b52e0 0x7fc7480b77a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc75c109140 0x7fc75c19eb10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 --2- 172.21.15.96:0/670864053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc75c102db0 0x7fc75c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:46.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 >> 172.21.15.96:0/670864053 conn(0x7fc75c0fe860 msgr2=0x7fc75c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 shutdown_connections 2024-09-10T22:54:46.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:46.465+0000 7fc762491640 1 -- 172.21.15.96:0/670864053 wait complete. 2024-09-10T22:54:47.393 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:54:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:47 smithi096 ceph-mon[30869]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 741 op/s 2024-09-10T22:54:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:47 smithi121 ceph-mon[35119]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 741 op/s 2024-09-10T22:54:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:47 smithi117 ceph-mon[35081]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 741 op/s 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.998+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/480532777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 msgr2=0x7ff7f41053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.998+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/480532777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 0x7ff7f41053a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e802a440 comp rx=0 tx=0).stop 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.999+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/480532777 shutdown_connections 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.999+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/480532777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410b350 0x7ff7f4111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.999+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/480532777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410a9d0 0x7ff7f410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.999+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/480532777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 0x7ff7f41053a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:48.999+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/480532777 >> 172.21.15.96:0/480532777 conn(0x7ff7f4100a10 msgr2=0x7ff7f4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.003+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/480532777 shutdown_connections 2024-09-10T22:54:49.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.004+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/480532777 wait complete. 2024-09-10T22:54:49.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.005+0000 7ff7f8f6a640 1 Processor -- start 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 -- start start 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 0x7ff7f4110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 0x7ff7f411bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 0x7ff7f4110bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4078770 con 0x7ff7f4104fc0 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40785f0 con 0x7ff7f410a9d0 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.006+0000 7ff7f8f6a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40788f0 con 0x7ff7f410b350 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.008+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 0x7ff7f4110bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.008+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 0x7ff7f4110bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35706/0 (socket says 172.21.15.96:35706) 2024-09-10T22:54:49.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.008+0000 7ff7f2d9d640 1 -- 172.21.15.96:0/2698800944 learned_addr learned my addr 172.21.15.96:0/2698800944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.009+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 0x7ff7f411bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.010+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/2698800944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 msgr2=0x7ff7f4110bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.010+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 0x7ff7f4110bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.010+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 msgr2=0x7ff7f4110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.010+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 0x7ff7f4110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.010+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/2698800944 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f41a33a0 con 0x7ff7f410a9d0 2024-09-10T22:54:49.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.011+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 0x7ff7f411bed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00cce0 tx=0x7ff7ec0075f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.011+0000 7ff7e37fe640 1 -- 172.21.15.96:0/2698800944 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004b20 con 0x7ff7f410a9d0 2024-09-10T22:54:49.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.011+0000 7ff7e37fe640 1 -- 172.21.15.96:0/2698800944 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec004cc0 con 0x7ff7f410a9d0 2024-09-10T22:54:49.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.011+0000 7ff7e37fe640 1 -- 172.21.15.96:0/2698800944 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00f6b0 con 0x7ff7f410a9d0 2024-09-10T22:54:49.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.012+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41a3630 con 0x7ff7f410a9d0 2024-09-10T22:54:49.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.013+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41a3ba0 con 0x7ff7f410a9d0 2024-09-10T22:54:49.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.016+0000 7ff7fb33f740 1 Processor -- start 2024-09-10T22:54:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.019+0000 7ff7fb33f740 1 -- start start 2024-09-10T22:54:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x56323538c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563235302ea0 con 0x56323538cdf0 2024-09-10T22:54:49.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5632352f22d0 con 0x563235398d80 2024-09-10T22:54:49.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.020+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5632352f60e0 con 0x56323538c4d0 2024-09-10T22:54:49.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.021+0000 7ff7e37fe640 1 -- 172.21.15.96:0/2698800944 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff7ec00f850 con 0x7ff7f410a9d0 2024-09-10T22:54:49.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.021+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.021+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38534/0 (socket says 172.21.15.96:38534) 2024-09-10T22:54:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.021+0000 7ff7f259c640 1 -- 172.21.15.96:0/1190912032 learned_addr learned my addr 172.21.15.96:0/1190912032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.022+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.022+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x56323538c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.023+0000 7ff7f259c640 1 -- 172.21.15.96:0/1190912032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 msgr2=0x56323538c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.023+0000 7ff7f259c640 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x56323538c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.023+0000 7ff7f259c640 1 -- 172.21.15.96:0/1190912032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 msgr2=0x56323539b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.023+0000 7ff7f259c640 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.023+0000 7ff7f259c640 1 -- 172.21.15.96:0/1190912032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563235050650 con 0x56323538cdf0 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.024+0000 7ff7e37fe640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 0x7ff7dc0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.024+0000 7ff7f259c640 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8004310 tx=0x7ff7e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d057ec1fb741c5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.024+0000 7ff7e37fe640 1 -- 172.21.15.96:0/2698800944 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff7ec0e5e10 con 0x7ff7f410a9d0 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.025+0000 7ff7e0ff9640 1 -- 172.21.15.96:0/1190912032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e803e030 con 0x56323538cdf0 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.025+0000 7ff7e0ff9640 1 -- 172.21.15.96:0/1190912032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e8004470 con 0x56323538cdf0 2024-09-10T22:54:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.025+0000 7ff7e0ff9640 1 -- 172.21.15.96:0/1190912032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8035040 con 0x56323538cdf0 2024-09-10T22:54:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.026+0000 7ff7fb33f740 1 -- 172.21.15.96:0/1190912032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 msgr2=0x563235398840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.026+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8004310 tx=0x7ff7e8000bc0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.028+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 0x7ff7dc0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.029+0000 7ff7fb33f740 1 -- 172.21.15.96:0/1190912032 shutdown_connections 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.029+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.029+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235398840 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.029+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/1190912032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x56323538c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.029+0000 7ff7fb33f740 1 -- 172.21.15.96:0/1190912032 >> 172.21.15.96:0/1190912032 conn(0x56323538a300 msgr2=0x56323538b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.030+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 0x7ff7dc0b79b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006fd0 tx=0x7ff7e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.031+0000 7ff7fb33f740 1 -- 172.21.15.96:0/1190912032 shutdown_connections 2024-09-10T22:54:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.031+0000 7ff7fb33f740 1 -- 172.21.15.96:0/1190912032 wait complete. 2024-09-10T22:54:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.031+0000 7ff7fb33f740 1 -- start start 2024-09-10T22:54:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 Processor -- start 2024-09-10T22:54:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235395a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563235398d80 0x563235396040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563235153700 con 0x56323538cdf0 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56323528d160 con 0x56323538c4d0 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56323539b9a0 con 0x563235398d80 2024-09-10T22:54:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7fb33f740 10 client.0 _ll_register_callbacks cb 0x5632352a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:49.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50190/0 (socket says 172.21.15.96:50190) 2024-09-10T22:54:49.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7f259c640 1 -- 172.21.15.96:0/199607420 learned_addr learned my addr 172.21.15.96:0/199607420 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:49.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.032+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563235398d80 0x563235396040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 -- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563235398d80 msgr2=0x563235396040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563235398d80 0x563235396040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 -- 172.21.15.96:0/199607420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 msgr2=0x563235395a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235395a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235395a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 -- 172.21.15.96:0/199607420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632352bac90 con 0x56323538c4d0 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235395a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.033+0000 7ff7f259c640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8031550 tx=0x7ff7e8031980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9ff4edd693fb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8035070 con 0x56323538c4d0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e8004470 con 0x56323538c4d0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff7e8031ca0 con 0x56323538c4d0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7e803e040 con 0x56323538c4d0 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7fb33f740 10 client.26591 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.034+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632353a7310 con 0x56323538c4d0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e8004470 con 0x56323538c4d0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e8043410 con 0x56323538c4d0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 10 client.26591 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 10 client.26591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 20 client.26591 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5632353a8810 0x5632353aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 4 client.26591 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563235304df0 con 0x5632353a8810 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.035+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5632353a8810 0x5632353aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.036+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5632353a8810 0x5632353aac90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6a7df5bb3d673bd server_cookie=e3c244babf57092b in_seq=0 out_seq=0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.036+0000 7ff7d9ffb640 10 client.26591 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.036+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/199607420 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563235304df0 con 0x5632353a8810 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.036+0000 7ff7d9ffb640 10 client.26591 handle_command_reply: tid=0 2024-09-10T22:54:49.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.036+0000 7ff7fb33f740 1 client.26591 shutdown 2024-09-10T22:54:49.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 msgr2=0x563235398630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8031550 tx=0x7ff7e8031980 comp rx=0 tx=0).stop 2024-09-10T22:54:49.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 shutdown_connections 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563235398d80 0x563235396040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x563235395a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5632353a8810 0x5632353aac90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/199607420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56323538c4d0 0x563235398630 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 >> 172.21.15.96:0/199607420 conn(0x56323538a300 msgr2=0x56323538fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7d9ffb640 0 client.26591 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 shutdown_connections 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 1 -- 172.21.15.96:0/199607420 wait complete. 2024-09-10T22:54:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.038+0000 7ff7fb33f740 20 client.26591 trim_cache size 0 max 16384 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.059+0000 7ff7fb33f740 1 Processor -- start 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.059+0000 7ff7fb33f740 1 -- start start 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538c4d0 0x5632353a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ac200 con 0x56323538c4d0 2024-09-10T22:54:49.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41add40 con 0x563235398d80 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ac8d0 con 0x56323538cdf0 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35724/0 (socket says 172.21.15.96:35724) 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/742097544 learned_addr learned my addr 172.21.15.96:0/742097544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/742097544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 msgr2=0x56323539a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/742097544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538c4d0 msgr2=0x5632353a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538c4d0 0x5632353a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/742097544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563235395830 con 0x56323538cdf0 2024-09-10T22:54:49.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:49.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.060+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0a98c0 tx=0x7ff7ec007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3aa79794b73612b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.062+0000 7ff7dbfff640 1 -- 172.21.15.96:0/742097544 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0acd50 con 0x56323538cdf0 2024-09-10T22:54:49.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.062+0000 7ff7dbfff640 1 -- 172.21.15.96:0/742097544 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec0ac650 con 0x56323538cdf0 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.062+0000 7ff7dbfff640 1 -- 172.21.15.96:0/742097544 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0acb10 con 0x56323538cdf0 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 -- 172.21.15.96:0/742097544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 msgr2=0x56323539a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0a98c0 tx=0x7ff7ec007f90 comp rx=0 tx=0).stop 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 -- 172.21.15.96:0/742097544 shutdown_connections 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x56323539a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538cdf0 0x56323539a170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/742097544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538c4d0 0x5632353a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 -- 172.21.15.96:0/742097544 >> 172.21.15.96:0/742097544 conn(0x56323538a300 msgr2=0x563235390cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 -- 172.21.15.96:0/742097544 shutdown_connections 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.063+0000 7ff7fb33f740 1 -- 172.21.15.96:0/742097544 wait complete. 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.064+0000 7ff7fb33f740 1 -- start start 2024-09-10T22:54:49.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.064+0000 7ff7fb33f740 1 Processor -- start 2024-09-10T22:54:49.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x5632353a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x5632353b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41ad670 con 0x56323538cdf0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41acfa0 con 0x563235398d80 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41aa1e0 con 0x56323538c4d0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38558/0 (socket says 172.21.15.96:38558) 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/168915757 learned_addr learned my addr 172.21.15.96:0/168915757 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7fb33f740 10 client.0 _ll_register_callbacks cb 0x7ff7e8004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f2d9d640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x5632353b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f259c640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x5632353a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 msgr2=0x5632353a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x5632353a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/168915757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 msgr2=0x5632353b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x5632353b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f1d9b640 1 -- 172.21.15.96:0/168915757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632353b10b0 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.065+0000 7ff7f259c640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x5632353a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7f1d9b640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000f80 tx=0x7ff7ec0ac290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ffb6cb609d850 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec097020 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ec0ac650 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff7ec0acab0 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec0b1130 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 10 client.17007 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563235415920 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec0ac650 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff7ec00fb70 con 0x56323538cdf0 2024-09-10T22:54:49.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 10 client.17007 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 10 client.17007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 20 client.17007 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563235416030 0x563235418450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 4 client.17007 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.066+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632353008a0 con 0x563235416030 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7f259c640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563235416030 0x563235418450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7f259c640 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563235416030 0x563235418450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ff3518d7f11c04c server_cookie=2735cd9dddc5b38b in_seq=0 out_seq=0 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7d9ffb640 10 client.17007 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7d9ffb640 1 -- 172.21.15.96:0/168915757 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632353008a0 con 0x563235416030 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7d9ffb640 10 client.17007 handle_command_reply: tid=0 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7fb33f740 1 client.17007 shutdown 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 msgr2=0x5632353b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000f80 tx=0x7ff7ec0ac290 comp rx=0 tx=0).stop 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 shutdown_connections 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.067+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563235416030 0x563235418450 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563235398d80 0x5632353b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56323538cdf0 0x5632353b0270 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7d9ffb640 0 client.17007 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 --2- 172.21.15.96:0/168915757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56323538c4d0 0x5632353a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 >> 172.21.15.96:0/168915757 conn(0x56323538a300 msgr2=0x563235388c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 shutdown_connections 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 1 -- 172.21.15.96:0/168915757 wait complete. 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7fb33f740 20 client.17007 trim_cache size 0 max 16384 2024-09-10T22:54:49.074 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 msgr2=0x7ff7dc0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 0x7ff7dc0b79b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4006fd0 tx=0x7ff7e4008040 comp rx=0 tx=0).stop 2024-09-10T22:54:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 msgr2=0x7ff7f411bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.068+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 0x7ff7f411bed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00cce0 tx=0x7ff7ec0075f0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.069+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 shutdown_connections 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7f410a9d0 0x7ff7f411bed0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff7dc0b54f0 0x7ff7dc0b79b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7f410b350 0x7ff7f4110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 --2- 172.21.15.96:0/2698800944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f4104fc0 0x7ff7f4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 >> 172.21.15.96:0/2698800944 conn(0x7ff7f4100a10 msgr2=0x7ff7f4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 shutdown_connections 2024-09-10T22:54:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:49.070+0000 7ff7f8f6a640 1 -- 172.21.15.96:0/2698800944 wait complete. 2024-09-10T22:54:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:49 smithi096 ceph-mon[30869]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T22:54:49.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:49 smithi121 ceph-mon[35119]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T22:54:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:49 smithi117 ceph-mon[35081]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 559 op/s 2024-09-10T22:54:49.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:54:50.207 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 435.0 s, 2478.6 tps, lat 40.349 ms stddev 46.367 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 -- 172.21.15.96:0/2843942864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 msgr2=0x7f358010b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 --2- 172.21.15.96:0/2843942864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f358010b640 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3578009900 tx=0x7f357802a480 comp rx=0 tx=0).stop 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 -- 172.21.15.96:0/2843942864 shutdown_connections 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 --2- 172.21.15.96:0/2843942864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3580114810 0x7f3580116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 --2- 172.21.15.96:0/2843942864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f358010bb80 0x7f35801142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 --2- 172.21.15.96:0/2843942864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f358010b640 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 -- 172.21.15.96:0/2843942864 >> 172.21.15.96:0/2843942864 conn(0x7f358006e2e0 msgr2=0x7f358006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 -- 172.21.15.96:0/2843942864 shutdown_connections 2024-09-10T22:54:51.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.056+0000 7f3586fc0640 1 -- 172.21.15.96:0/2843942864 wait complete. 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 Processor -- start 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 -- start start 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 0x7f35801a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3580114810 0x7f35801a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f358011b550 con 0x7f358010b260 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f358011b3d0 con 0x7f358010bb80 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3586fc0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f358011b6d0 con 0x7f3580114810 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38574/0 (socket says 172.21.15.96:38574) 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.057+0000 7f3584d5c640 1 -- 172.21.15.96:0/1648726937 learned_addr learned my addr 172.21.15.96:0/1648726937 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 0x7f35801a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f358555d640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3580114810 0x7f35801a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 -- 172.21.15.96:0/1648726937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3580114810 msgr2=0x7f35801a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3580114810 0x7f35801a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 -- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 msgr2=0x7f35801a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f357ffff640 1 -- 172.21.15.96:0/1648726937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35801a7b70 con 0x7f358010bb80 2024-09-10T22:54:51.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.058+0000 7f3584d5c640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f357ffff640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 0x7f35801a2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9f0 tx=0x7f357400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f357dffb640 1 -- 172.21.15.96:0/1648726937 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400ccc0 con 0x7f358010bb80 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801a7e60 con 0x7f358010bb80 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f357dffb640 1 -- 172.21.15.96:0/1648726937 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35740108e0 con 0x7f358010bb80 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f357dffb640 1 -- 172.21.15.96:0/1648726937 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574018880 con 0x7f358010bb80 2024-09-10T22:54:51.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.059+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801179a0 con 0x7f358010bb80 2024-09-10T22:54:51.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.062+0000 7f3589395740 1 Processor -- start 2024-09-10T22:54:51.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.063+0000 7f357dffb640 1 -- 172.21.15.96:0/1648726937 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3574010430 con 0x7f358010bb80 2024-09-10T22:54:51.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.064+0000 7f357dffb640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 0x7f35680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f358555d640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 0x7f35680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f357dffb640 1 -- 172.21.15.96:0/1648726937 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35740e5ef0 con 0x7f358010bb80 2024-09-10T22:54:51.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 -- start start 2024-09-10T22:54:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 0x55ac1b43e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b441170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b3a8ea0 con 0x55ac1b4324d0 2024-09-10T22:54:51.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b3982d0 con 0x55ac1b43ed80 2024-09-10T22:54:51.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3589395740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b39c0e0 con 0x55ac1b432df0 2024-09-10T22:54:51.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38582/0 (socket says 172.21.15.96:38582) 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.065+0000 7f3584d5c640 1 -- 172.21.15.96:0/978411601 learned_addr learned my addr 172.21.15.96:0/978411601 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 -- 172.21.15.96:0/978411601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 msgr2=0x55ac1b43e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f357ffff640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 0x55ac1b43e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 0x55ac1b43e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f358555d640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 0x7f35680b7820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3570005fd0 tx=0x7f3570009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 -- 172.21.15.96:0/978411601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 msgr2=0x55ac1b441170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f358555d640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b441170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b441170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 -- 172.21.15.96:0/978411601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1b0f6650 con 0x55ac1b4324d0 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f357ffff640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 0x55ac1b43e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f358555d640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b441170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3584d5c640 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c480cfeebb0673 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3566ffd640 1 -- 172.21.15.96:0/978411601 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578035040 con 0x55ac1b4324d0 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.066+0000 7f3566ffd640 1 -- 172.21.15.96:0/978411601 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35780317a0 con 0x55ac1b4324d0 2024-09-10T22:54:51.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3566ffd640 1 -- 172.21.15.96:0/978411601 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578005650 con 0x55ac1b4324d0 2024-09-10T22:54:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 -- 172.21.15.96:0/978411601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 msgr2=0x55ac1b4328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578032ed0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 -- 172.21.15.96:0/978411601 shutdown_connections 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b441170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b432df0 0x55ac1b43e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.067+0000 7f3589395740 1 --2- 172.21.15.96:0/978411601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b4324d0 0x55ac1b4328b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.068+0000 7f3589395740 1 -- 172.21.15.96:0/978411601 >> 172.21.15.96:0/978411601 conn(0x55ac1b430300 msgr2=0x55ac1b4314a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.068+0000 7f3589395740 1 -- 172.21.15.96:0/978411601 shutdown_connections 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.068+0000 7f3589395740 1 -- 172.21.15.96:0/978411601 wait complete. 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.068+0000 7f3589395740 1 -- start start 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.068+0000 7f3589395740 1 Processor -- start 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b43e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b43bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b1f9700 con 0x55ac1b432df0 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b333160 con 0x55ac1b43ed80 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ac1b441a40 con 0x55ac1b4324d0 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f3589395740 10 client.0 _ll_register_callbacks cb 0x55ac1b34c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f357ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f357ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38592/0 (socket says 172.21.15.96:38592) 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.069+0000 7f357ffff640 1 -- 172.21.15.96:0/933057571 learned_addr learned my addr 172.21.15.96:0/933057571 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f3584d5c640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b43e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 -- 172.21.15.96:0/933057571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 msgr2=0x55ac1b43e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b43e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 -- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 msgr2=0x55ac1b43bee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b43bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 -- 172.21.15.96:0/933057571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1b360c90 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f3584d5c640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b43e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f357ffff640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f35740a99d0 tx=0x7f35740abf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25baf5d493d776ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.070+0000 7f355ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740a6e60 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35740ab9c0 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35780317a0 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35740b1190 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f3589395740 10 client.17019 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.071+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac1b44d1c0 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35780317a0 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f35740ab640 con 0x55ac1b432df0 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 10 client.17019 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 20 client.17019 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:51.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b44eb20 0x55ac1b450fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 4 client.17019 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:51.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1b3aadf0 con 0x55ac1b44eb20 2024-09-10T22:54:51.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.072+0000 7f3584d5c640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b44eb20 0x55ac1b450fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.082+0000 7f3584d5c640 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b44eb20 0x55ac1b450fe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e110bcee9fac484 server_cookie=15c77a4b5f3e5451 in_seq=0 out_seq=0 2024-09-10T22:54:51.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.082+0000 7f355ffff640 10 client.17019 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.086+0000 7f355ffff640 1 -- 172.21.15.96:0/933057571 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac1b3aadf0 con 0x55ac1b44eb20 2024-09-10T22:54:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.086+0000 7f355ffff640 10 client.17019 handle_command_reply: tid=0 2024-09-10T22:54:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.086+0000 7f3589395740 1 client.17019 shutdown 2024-09-10T22:54:51.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.087+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 msgr2=0x55ac1b43b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.087+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f35740a99d0 tx=0x7f35740abf50 comp rx=0 tx=0).stop 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 shutdown_connections 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b44eb20 0x55ac1b450fe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b43ed80 0x55ac1b43bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b432df0 0x55ac1b43b9a0 secure :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f35740a99d0 tx=0x7f35740abf50 comp rx=0 tx=0).stop 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 --2- 172.21.15.96:0/933057571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b43e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 >> 172.21.15.96:0/933057571 conn(0x55ac1b430300 msgr2=0x55ac1b435e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.091+0000 7f355ffff640 0 client.17019 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.098+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 shutdown_connections 2024-09-10T22:54:51.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.098+0000 7f3589395740 1 -- 172.21.15.96:0/933057571 wait complete. 2024-09-10T22:54:51.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.098+0000 7f3589395740 20 client.17019 trim_cache size 0 max 16384 2024-09-10T22:54:51.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.118+0000 7f3589395740 1 Processor -- start 2024-09-10T22:54:51.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.118+0000 7f3589395740 1 -- start start 2024-09-10T22:54:51.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b42fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b436bf0 0x55ac1b437080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38608/0 (socket says 172.21.15.96:38608) 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 -- 172.21.15.96:0/4259524334 learned_addr learned my addr 172.21.15.96:0/4259524334 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35801b8d90 con 0x55ac1b43ed80 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35801ba8d0 con 0x55ac1b436bf0 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35801b9460 con 0x55ac1b4324d0 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f3584d5c640 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b42fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f358555d640 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b436bf0 0x55ac1b437080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 -- 172.21.15.96:0/4259524334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 msgr2=0x55ac1b42fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b42fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 -- 172.21.15.96:0/4259524334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b436bf0 msgr2=0x55ac1b437080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b436bf0 0x55ac1b437080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.119+0000 7f357ffff640 1 -- 172.21.15.96:0/4259524334 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1b451350 con 0x55ac1b43ed80 2024-09-10T22:54:51.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.120+0000 7f357ffff640 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f35740a8d30 tx=0x7f3574005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39d708edc1e29b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.120+0000 7f35667fc640 1 -- 172.21.15.96:0/4259524334 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740a6e60 con 0x55ac1b43ed80 2024-09-10T22:54:51.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.120+0000 7f35667fc640 1 -- 172.21.15.96:0/4259524334 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35740b5c40 con 0x55ac1b43ed80 2024-09-10T22:54:51.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.120+0000 7f35667fc640 1 -- 172.21.15.96:0/4259524334 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740a80b0 con 0x55ac1b43ed80 2024-09-10T22:54:51.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 msgr2=0x55ac1b42b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f35740a8d30 tx=0x7f3574005b40 comp rx=0 tx=0).stop 2024-09-10T22:54:51.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 shutdown_connections 2024-09-10T22:54:51.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b436bf0 0x55ac1b437080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b43ed80 0x55ac1b42b480 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 --2- 172.21.15.96:0/4259524334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b4324d0 0x55ac1b42fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 >> 172.21.15.96:0/4259524334 conn(0x55ac1b430300 msgr2=0x55ac1b43b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 shutdown_connections 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- 172.21.15.96:0/4259524334 wait complete. 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 -- start start 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.121+0000 7f3589395740 1 Processor -- start 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b436bf0 0x55ac1b442a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b43ed80 0x55ac1b443070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35801ba200 con 0x55ac1b436bf0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35801b9b30 con 0x55ac1b4324d0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35801b6d70 con 0x55ac1b43ed80 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3589395740 10 client.0 _ll_register_callbacks cb 0x7f35740a8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50272/0 (socket says 172.21.15.96:50272) 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 -- 172.21.15.96:0/3173971784 learned_addr learned my addr 172.21.15.96:0/3173971784 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f357ffff640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b436bf0 0x55ac1b442a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f358555d640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b43ed80 0x55ac1b443070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 -- 172.21.15.96:0/3173971784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b43ed80 msgr2=0x55ac1b443070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b43ed80 0x55ac1b443070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 -- 172.21.15.96:0/3173971784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b436bf0 msgr2=0x55ac1b442a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b436bf0 0x55ac1b442a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.122+0000 7f3584d5c640 1 -- 172.21.15.96:0/3173971784 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1b4baaf0 con 0x55ac1b4324d0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.123+0000 7f3584d5c640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3578009a20 tx=0x7f3578005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23eeec2225e64ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.123+0000 7f355ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.123+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578035040 con 0x55ac1b4324d0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.123+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3578004040 con 0x55ac1b4324d0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.124+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35740b5c40 con 0x55ac1b4324d0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.124+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f357805e050 con 0x55ac1b4324d0 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f3589395740 10 client.26609 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:51.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac1b4bbca0 con 0x55ac1b4324d0 2024-09-10T22:54:51.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3578031910 con 0x55ac1b4324d0 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f357804f070 con 0x55ac1b4324d0 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f3589395740 10 client.26609 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.125+0000 7f3589395740 10 client.26609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f3589395740 20 client.26609 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b4bbfe0 0x55ac1b4be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f3589395740 4 client.26609 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f35740b5c40 con 0x55ac1b4bbfe0 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f357ffff640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b4bbfe0 0x55ac1b4be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f355ffff640 10 client.26609 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f357ffff640 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b4bbfe0 0x55ac1b4be400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21ee715cd32d8da server_cookie=f90a96b768334922 in_seq=0 out_seq=0 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f355ffff640 1 -- 172.21.15.96:0/3173971784 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f35740b5c40 con 0x55ac1b4bbfe0 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f355ffff640 10 client.26609 handle_command_reply: tid=0 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.126+0000 7f3589395740 1 client.26609 shutdown 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 msgr2=0x55ac1b447570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3578009a20 tx=0x7f3578005e50 comp rx=0 tx=0).stop 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 shutdown_connections 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ac1b4bbfe0 0x55ac1b4be400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ac1b43ed80 0x55ac1b443070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f355ffff640 0 client.26609 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac1b436bf0 0x55ac1b442a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 --2- 172.21.15.96:0/3173971784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac1b4324d0 0x55ac1b447570 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 >> 172.21.15.96:0/3173971784 conn(0x55ac1b430300 msgr2=0x55ac1b43cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 shutdown_connections 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 1 -- 172.21.15.96:0/3173971784 wait complete. 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.127+0000 7f3589395740 20 client.26609 trim_cache size 0 max 16384 2024-09-10T22:54:51.135 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:54:51.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 msgr2=0x7f35680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 0x7f35680b7820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3570005fd0 tx=0x7f3570009500 comp rx=0 tx=0).stop 2024-09-10T22:54:51.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 msgr2=0x7f35801a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 0x7f35801a2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9f0 tx=0x7f357400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 shutdown_connections 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f358010bb80 0x7f35801a2000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f35680b5360 0x7f35680b7820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3580114810 0x7f35801a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 --2- 172.21.15.96:0/1648726937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f358010b260 0x7f35801a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 >> 172.21.15.96:0/1648726937 conn(0x7f358006e2e0 msgr2=0x7f3580071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 shutdown_connections 2024-09-10T22:54:51.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:51.128+0000 7f3586fc0640 1 -- 172.21.15.96:0/1648726937 wait complete. 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 -- 172.21.15.96:0/3622954185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324103380 msgr2=0x7f2324109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 --2- 172.21.15.96:0/3622954185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324103380 0x7f2324109d00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f231c009cd0 tx=0x7f231c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 -- 172.21.15.96:0/3622954185 shutdown_connections 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 --2- 172.21.15.96:0/3622954185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324103380 0x7f2324109d00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 --2- 172.21.15.96:0/3622954185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23241029c0 0x7f2324102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 --2- 172.21.15.96:0/3622954185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2324101870 0x7f2324101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 -- 172.21.15.96:0/3622954185 >> 172.21.15.96:0/3622954185 conn(0x7f23240fd130 msgr2=0x7f23240ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 -- 172.21.15.96:0/3622954185 shutdown_connections 2024-09-10T22:54:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.674+0000 7f232bef7640 1 -- 172.21.15.96:0/3622954185 wait complete. 2024-09-10T22:54:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.675+0000 7f232bef7640 1 Processor -- start 2024-09-10T22:54:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.675+0000 7f232bef7640 1 -- start start 2024-09-10T22:54:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.675+0000 7f232bef7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324101870 0x7f23241056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.675+0000 7f232bef7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23241029c0 0x7f2324103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.675+0000 7f232bef7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232bef7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d4e0 con 0x7f2324101870 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f2329492640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23241029c0 0x7f2324103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35814/0 (socket says 172.21.15.96:35814) 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 -- 172.21.15.96:0/4220947331 learned_addr learned my addr 172.21.15.96:0/4220947331 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d360 con 0x7f23241029c0 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d660 con 0x7f2324103380 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 -- 172.21.15.96:0/4220947331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23241029c0 msgr2=0x7f2324103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23241029c0 0x7f2324103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 -- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324101870 msgr2=0x7f23241056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324101870 0x7f23241056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 -- 172.21.15.96:0/4220947331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23241049a0 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.676+0000 7f232a494640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f231c033040 tx=0x7f231c003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.677+0000 7f231affd640 1 -- 172.21.15.96:0/4220947331 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c004240 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.677+0000 7f231affd640 1 -- 172.21.15.96:0/4220947331 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c0305a0 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.677+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2324104c30 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.677+0000 7f231affd640 1 -- 172.21.15.96:0/4220947331 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0466e0 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.677+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23241a7770 con 0x7f2324103380 2024-09-10T22:54:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.680+0000 7f231affd640 1 -- 172.21.15.96:0/4220947331 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f231c04e070 con 0x7f2324103380 2024-09-10T22:54:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f232ecc2740 1 Processor -- start 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f232ecc2740 1 -- start start 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f231affd640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 0x7f23100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f2329492640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.681+0000 7f2329c93640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 0x7f23100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35830/0 (socket says 172.21.15.96:35830) 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 -- 172.21.15.96:0/2496823547 learned_addr learned my addr 172.21.15.96:0/2496823547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a098ab86f0 con 0x55a098b4bbd0 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a098aa7b20 con 0x55a098b49190 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a098aab930 con 0x55a098b46860 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f231affd640 1 -- 172.21.15.96:0/4220947331 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f231c02e030 con 0x7f2324103380 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f232a494640 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 -- 172.21.15.96:0/2496823547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 msgr2=0x55a098b4b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 -- 172.21.15.96:0/2496823547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 msgr2=0x55a098b4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 -- 172.21.15.96:0/2496823547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a098a6d530 con 0x55a098b46860 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329c93640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 0x7f23100b7820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23140058c0 tx=0x7f2314005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f2329492640 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2320009710 tx=0x7f23200097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db8975c6cd27327 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f230ffff640 1 -- 172.21.15.96:0/2496823547 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320013170 con 0x55a098b46860 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f230ffff640 1 -- 172.21.15.96:0/2496823547 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f232000fc70 con 0x55a098b46860 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.682+0000 7f230ffff640 1 -- 172.21.15.96:0/2496823547 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320010930 con 0x55a098b46860 2024-09-10T22:54:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.683+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 msgr2=0x55a098b48c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.683+0000 7f232ecc2740 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2320009710 tx=0x7f23200097a0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 shutdown_connections 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 --2- 172.21.15.96:0/2496823547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b48c50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 >> 172.21.15.96:0/2496823547 conn(0x55a098b419b0 msgr2=0x55a098b42740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 shutdown_connections 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.684+0000 7f232ecc2740 1 -- 172.21.15.96:0/2496823547 wait complete. 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 -- start start 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 Processor -- start 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b4ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a0989dfab0 con 0x55a098b4bbd0 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a098a4d730 con 0x55a098b49190 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.685+0000 7f232ecc2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a098b4ebb0 con 0x55a098b46860 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50314/0 (socket says 172.21.15.96:50314) 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 -- 172.21.15.96:0/3067175148 learned_addr learned my addr 172.21.15.96:0/3067175148 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329c93640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b4ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f232a494640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f232ecc2740 10 client.0 _ll_register_callbacks cb 0x55a098aa16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 -- 172.21.15.96:0/3067175148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 msgr2=0x55a098b4ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b4ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 -- 172.21.15.96:0/3067175148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 msgr2=0x55a098b51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 -- 172.21.15.96:0/3067175148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a098809790 con 0x55a098b49190 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f232a494640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f2329492640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23200104e0 tx=0x7f232000fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d465faf994eba2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f230d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320005260 con 0x55a098b49190 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f232000fc70 con 0x55a098b49190 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.686+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2320005400 con 0x55a098b49190 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2320005600 con 0x55a098b49190 2024-09-10T22:54:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 10 client.26621 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a098882620 con 0x55a098b49190 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f232000fc70 con 0x55a098b49190 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f23200109c0 con 0x55a098b49190 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 10 client.26621 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 20 client.26621 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098b5b570 0x55a098b5d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 4 client.26621 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.687+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a098aba640 con 0x55a098b5b570 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.688+0000 7f2329c93640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098b5b570 0x55a098b5d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.688+0000 7f2329c93640 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098b5b570 0x55a098b5d960 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8880a01b471e705d server_cookie=148910b3932b051a in_seq=0 out_seq=0 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.688+0000 7f230d7fa640 10 client.26621 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.689+0000 7f230d7fa640 1 -- 172.21.15.96:0/3067175148 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a098aba640 con 0x55a098b5b570 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.689+0000 7f230d7fa640 10 client.26621 handle_command_reply: tid=0 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.689+0000 7f232ecc2740 1 client.26621 shutdown 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.689+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 msgr2=0x55a098b505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.689+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23200104e0 tx=0x7f232000fa50 comp rx=0 tx=0).stop 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 shutdown_connections 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098b5b570 0x55a098b5d960 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b505a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3067175148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b4ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 >> 172.21.15.96:0/3067175148 conn(0x55a098b419b0 msgr2=0x55a098b40bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f230d7fa640 0 client.26621 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.690+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 shutdown_connections 2024-09-10T22:54:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.691+0000 7f232ecc2740 1 -- 172.21.15.96:0/3067175148 wait complete. 2024-09-10T22:54:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.691+0000 7f232ecc2740 20 client.26621 trim_cache size 0 max 16384 2024-09-10T22:54:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.715+0000 7f232ecc2740 1 Processor -- start 2024-09-10T22:54:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.715+0000 7f232ecc2740 1 -- start start 2024-09-10T22:54:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.715+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.715+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.716+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.716+0000 7f232ecc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23241b0310 con 0x55a098b4bbd0 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.716+0000 7f232ecc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23241b1e50 con 0x55a098b49190 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.716+0000 7f232ecc2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23241b09e0 con 0x55a098b46860 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.717+0000 7f232a494640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.717+0000 7f232a494640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38664/0 (socket says 172.21.15.96:38664) 2024-09-10T22:54:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.717+0000 7f232a494640 1 -- 172.21.15.96:0/1286469177 learned_addr learned my addr 172.21.15.96:0/1286469177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.718+0000 7f232a494640 1 -- 172.21.15.96:0/1286469177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 msgr2=0x55a098b413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.718+0000 7f232a494640 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.718+0000 7f232a494640 1 -- 172.21.15.96:0/1286469177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 msgr2=0x55a098b4bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.718+0000 7f232a494640 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.718+0000 7f232a494640 1 -- 172.21.15.96:0/1286469177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a098b514e0 con 0x55a098b4bbd0 2024-09-10T22:54:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.719+0000 7f232a494640 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a5a5be63b3848 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.724+0000 7f230f7fe640 1 -- 172.21.15.96:0/1286469177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0c7e60 con 0x55a098b4bbd0 2024-09-10T22:54:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.724+0000 7f230f7fe640 1 -- 172.21.15.96:0/1286469177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c036040 con 0x55a098b4bbd0 2024-09-10T22:54:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.725+0000 7f232ecc2740 1 -- 172.21.15.96:0/1286469177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 msgr2=0x55a098b4c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.725+0000 7f232ecc2740 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.729+0000 7f232ecc2740 1 -- 172.21.15.96:0/1286469177 shutdown_connections 2024-09-10T22:54:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.729+0000 7f232ecc2740 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b49190 0x55a098b4bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.729+0000 7f232ecc2740 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b4bbd0 0x55a098b4c4f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.729+0000 7f232ecc2740 1 --2- 172.21.15.96:0/1286469177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b46860 0x55a098b413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.729+0000 7f232ecc2740 1 -- 172.21.15.96:0/1286469177 >> 172.21.15.96:0/1286469177 conn(0x55a098b419b0 msgr2=0x55a098b45d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.732+0000 7f232ecc2740 1 -- 172.21.15.96:0/1286469177 shutdown_connections 2024-09-10T22:54:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.733+0000 7f232ecc2740 1 -- 172.21.15.96:0/1286469177 wait complete. 2024-09-10T22:54:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.734+0000 7f232ecc2740 1 -- start start 2024-09-10T22:54:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.734+0000 7f232ecc2740 1 Processor -- start 2024-09-10T22:54:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b46860 0x55a098b54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b4bbd0 0x55a098b55c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23241b1780 con 0x55a098b46860 2024-09-10T22:54:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23241b10b0 con 0x55a098b4bbd0 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23241ae2f0 con 0x55a098b49190 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232ecc2740 10 client.0 _ll_register_callbacks cb 0x7f231c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35850/0 (socket says 172.21.15.96:35850) 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 -- 172.21.15.96:0/3108337201 learned_addr learned my addr 172.21.15.96:0/3108337201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f232a494640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b4bbd0 0x55a098b55c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 -- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b4bbd0 msgr2=0x55a098b55c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b4bbd0 0x55a098b55c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 -- 172.21.15.96:0/3108337201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b46860 msgr2=0x55a098b54a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b46860 0x55a098b54a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.735+0000 7f2329492640 1 -- 172.21.15.96:0/3108337201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a098b56470 con 0x55a098b49190 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.736+0000 7f2329492640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2320009a40 tx=0x7f23200295b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de17e8a26f0ead8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.736+0000 7f230d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.736+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320029a30 con 0x55a098b49190 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.736+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c036040 con 0x55a098b49190 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.736+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2320029bd0 con 0x55a098b49190 2024-09-10T22:54:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.737+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f232000fc60 con 0x55a098b49190 2024-09-10T22:54:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.738+0000 7f232ecc2740 10 client.26674 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.738+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a098b57810 con 0x55a098b49190 2024-09-10T22:54:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.739+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c036040 con 0x55a098b49190 2024-09-10T22:54:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.740+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f23200035f0 con 0x55a098b49190 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 10 client.26674 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 20 client.26674 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098bd9420 0x55a098bdb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 4 client.26674 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:54:51.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.741+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a098ab60f0 con 0x55a098bd9420 2024-09-10T22:54:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.742+0000 7f2329c93640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098bd9420 0x55a098bdb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.745+0000 7f2329c93640 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098bd9420 0x55a098bdb810 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56f6fd11872a71 server_cookie=bf985cc659873699 in_seq=0 out_seq=0 2024-09-10T22:54:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.745+0000 7f230d7fa640 10 client.26674 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:51 smithi096 ceph-mon[30869]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 739 op/s 2024-09-10T22:54:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:51 smithi121 ceph-mon[35119]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 739 op/s 2024-09-10T22:54:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.764+0000 7f230d7fa640 1 -- 172.21.15.96:0/3108337201 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a098ab60f0 con 0x55a098bd9420 2024-09-10T22:54:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f230d7fa640 10 client.26674 handle_command_reply: tid=0 2024-09-10T22:54:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 client.26674 shutdown 2024-09-10T22:54:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 msgr2=0x55a098b550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2320009a40 tx=0x7f23200295b0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 shutdown_connections 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55a098bd9420 0x55a098bdb810 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a098b4bbd0 0x55a098b55c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a098b49190 0x55a098b550c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f230d7fa640 0 client.26674 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 --2- 172.21.15.96:0/3108337201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a098b46860 0x55a098b54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.765+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 >> 172.21.15.96:0/3108337201 conn(0x55a098b419b0 msgr2=0x55a098b4f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 shutdown_connections 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232ecc2740 1 -- 172.21.15.96:0/3108337201 wait complete. 2024-09-10T22:54:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232ecc2740 20 client.26674 trim_cache size 0 max 16384 2024-09-10T22:54:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:54:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:54:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:54:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 msgr2=0x7f23100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 0x7f23100b7820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23140058c0 tx=0x7f2314005850 comp rx=0 tx=0).stop 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 msgr2=0x7f2324104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f231c033040 tx=0x7f231c003b80 comp rx=0 tx=0).stop 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 shutdown_connections 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23241029c0 0x7f2324103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.766+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23100b5360 0x7f23100b7820 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.767+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2324103380 0x7f2324104460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.767+0000 7f232bef7640 1 --2- 172.21.15.96:0/4220947331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2324101870 0x7f23241056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.767+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 >> 172.21.15.96:0/4220947331 conn(0x7f23240fd130 msgr2=0x7f232406b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.767+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 shutdown_connections 2024-09-10T22:54:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:54:51.767+0000 7f232bef7640 1 -- 172.21.15.96:0/4220947331 wait complete. 2024-09-10T22:54:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:51 smithi117 ceph-mon[35081]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 739 op/s 2024-09-10T22:54:53.005 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:54:53.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:54:53.363 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:53.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:53 smithi096 ceph-mon[30869]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-10T22:54:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:53 smithi121 ceph-mon[35119]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-10T22:54:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:53 smithi117 ceph-mon[35081]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 728 op/s 2024-09-10T22:54:53.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 -- 172.21.15.96:0/975569615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 msgr2=0x7fa268108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 --2- 172.21.15.96:0/975569615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa268108bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa250009900 tx=0x7fa25002a400 comp rx=0 tx=0).stop 2024-09-10T22:54:53.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 -- 172.21.15.96:0/975569615 shutdown_connections 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 --2- 172.21.15.96:0/975569615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa268108bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 --2- 172.21.15.96:0/975569615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268109130 0x7fa26810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 --2- 172.21.15.96:0/975569615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268102da0 0x7fa268103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 -- 172.21.15.96:0/975569615 >> 172.21.15.96:0/975569615 conn(0x7fa2680fe850 msgr2=0x7fa268100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.918+0000 7fa26e36f640 1 -- 172.21.15.96:0/975569615 shutdown_connections 2024-09-10T22:54:53.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.919+0000 7fa26e36f640 1 -- 172.21.15.96:0/975569615 wait complete. 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.919+0000 7fa26e36f640 1 Processor -- start 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.919+0000 7fa26e36f640 1 -- start start 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.919+0000 7fa26e36f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26e36f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa2681995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26e36f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268109130 0x7fa26819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26e36f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa268112d80 con 0x7fa268102da0 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26e36f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa268112c00 con 0x7fa2681087b0 2024-09-10T22:54:53.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26e36f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa268112f00 con 0x7fa268109130 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38690/0 (socket says 172.21.15.96:38690) 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 -- 172.21.15.96:0/1475933321 learned_addr learned my addr 172.21.15.96:0/1475933321 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa25ffff640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa2681995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa26c90c640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268109130 0x7fa26819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 -- 172.21.15.96:0/1475933321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268109130 msgr2=0x7fa26819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268109130 0x7fa26819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 -- 172.21.15.96:0/1475933321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 msgr2=0x7fa2681995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa2681995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 -- 172.21.15.96:0/1475933321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26819f140 con 0x7fa268102da0 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa25ffff640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa2681995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.920+0000 7fa267fff640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa25800ea60 tx=0x7fa25800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.921+0000 7fa265ffb640 1 -- 172.21.15.96:0/1475933321 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800cde0 con 0x7fa268102da0 2024-09-10T22:54:53.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.921+0000 7fa265ffb640 1 -- 172.21.15.96:0/1475933321 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2580035f0 con 0x7fa268102da0 2024-09-10T22:54:53.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.921+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26819f430 con 0x7fa268102da0 2024-09-10T22:54:53.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.921+0000 7fa265ffb640 1 -- 172.21.15.96:0/1475933321 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258010690 con 0x7fa268102da0 2024-09-10T22:54:53.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.921+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2681a5cc0 con 0x7fa268102da0 2024-09-10T22:54:53.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa270744740 1 Processor -- start 2024-09-10T22:54:53.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa270744740 1 -- start start 2024-09-10T22:54:53.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa270744740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa270744740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edc800 0x55ec35eec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa265ffb640 1 -- 172.21.15.96:0/1475933321 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa258004000 con 0x7fa268102da0 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.925+0000 7fa270744740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa270744740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec35e52e60 con 0x55ec35edbee0 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa270744740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec35e42290 con 0x55ec35edc800 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa270744740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec35e460a0 con 0x55ec35eecba0 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35876/0 (socket says 172.21.15.96:35876) 2024-09-10T22:54:53.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa26c90c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 -- 172.21.15.96:0/2477014662 learned_addr learned my addr 172.21.15.96:0/2477014662 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa267fff640 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edc800 0x55ec35eec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 -- 172.21.15.96:0/2477014662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edc800 msgr2=0x55ec35eec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edc800 0x55ec35eec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 -- 172.21.15.96:0/2477014662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 msgr2=0x55ec35edc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 -- 172.21.15.96:0/2477014662 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec35ba0930 con 0x55ec35eecba0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.926+0000 7fa25ffff640 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa250005c20 tx=0x7fa250031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8aa8e37081701b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa25e7fc640 1 -- 172.21.15.96:0/2477014662 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250040070 con 0x55ec35eecba0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa265ffb640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 0x7fa2540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa25e7fc640 1 -- 172.21.15.96:0/2477014662 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25000b0b0 con 0x55ec35eecba0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa25e7fc640 1 -- 172.21.15.96:0/2477014662 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250035040 con 0x55ec35eecba0 2024-09-10T22:54:53.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa265ffb640 1 -- 172.21.15.96:0/1475933321 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa2580e5e20 con 0x7fa268102da0 2024-09-10T22:54:53.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.927+0000 7fa26c90c640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 0x7fa2540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 -- 172.21.15.96:0/2477014662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 msgr2=0x55ec35eeef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa250005c20 tx=0x7fa250031400 comp rx=0 tx=0).stop 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 -- 172.21.15.96:0/2477014662 shutdown_connections 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edc800 0x55ec35eec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35eecba0 0x55ec35eeef90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 --2- 172.21.15.96:0/2477014662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.928+0000 7fa270744740 1 -- 172.21.15.96:0/2477014662 >> 172.21.15.96:0/2477014662 conn(0x55ec35ed9d10 msgr2=0x55ec35edaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.929+0000 7fa26c90c640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 0x7fa2540b7820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.929+0000 7fa270744740 1 -- 172.21.15.96:0/2477014662 shutdown_connections 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.929+0000 7fa270744740 1 -- 172.21.15.96:0/2477014662 wait complete. 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.930+0000 7fa270744740 1 -- start start 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.930+0000 7fa270744740 1 Processor -- start 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.930+0000 7fa270744740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.930+0000 7fa270744740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35eea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35eea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ec35ca3890 con 0x55ec35eecba0 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec35ddcd40 con 0x55ec35edbee0 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec35eef7f0 con 0x55ec35edc800 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50368/0 (socket says 172.21.15.96:50368) 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 -- 172.21.15.96:0/394829516 learned_addr learned my addr 172.21.15.96:0/394829516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa25ffff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35eea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa26c90c640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35eea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 -- 172.21.15.96:0/394829516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 msgr2=0x55ec35eea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35eea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 -- 172.21.15.96:0/394829516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 msgr2=0x55ec35eea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35eea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 -- 172.21.15.96:0/394829516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec35ef8c00 con 0x55ec35edbee0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa26c90c640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35eea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 10 client.0 _ll_register_callbacks cb 0x55ec35e40c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa267fff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2580ac610 tx=0x7fa258003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716317bffbcdda0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa2477fe640 10 client.26633 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:53.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa25000b0b0 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa2477fe640 1 -- 172.21.15.96:0/394829516 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580abdf0 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa2477fe640 1 -- 172.21.15.96:0/394829516 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa2580a88f0 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa2477fe640 1 -- 172.21.15.96:0/394829516 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2580ac190 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 10 client.26633 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.931+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ec35efb480 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa2477fe640 1 -- 172.21.15.96:0/394829516 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa2580ab8e0 con 0x55ec35edbee0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 10 client.26633 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 20 client.26633 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35efc820 0x55ec35efece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 4 client.26633 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec35e54db0 con 0x55ec35efc820 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa25ffff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35efc820 0x55ec35efece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa2477fe640 10 client.26633 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.932+0000 7fa25ffff640 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35efc820 0x55ec35efece0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d2f49fea2aff94 server_cookie=e889bc9c335819c1 in_seq=0 out_seq=0 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa2477fe640 1 -- 172.21.15.96:0/394829516 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec35e54db0 con 0x55ec35efc820 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa2477fe640 10 client.26633 handle_command_reply: tid=0 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 client.26633 shutdown 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 msgr2=0x55ec35ee9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2580ac610 tx=0x7fa258003df0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 shutdown_connections 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35efc820 0x55ec35efece0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35eea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35eea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 --2- 172.21.15.96:0/394829516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ee9b30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 >> 172.21.15.96:0/394829516 conn(0x55ec35ed9d10 msgr2=0x55ec35ed8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.933+0000 7fa2477fe640 0 client.26633 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:53.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.934+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 shutdown_connections 2024-09-10T22:54:53.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.934+0000 7fa270744740 1 -- 172.21.15.96:0/394829516 wait complete. 2024-09-10T22:54:53.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.934+0000 7fa270744740 20 client.26633 trim_cache size 0 max 16384 2024-09-10T22:54:53.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 Processor -- start 2024-09-10T22:54:53.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 -- start start 2024-09-10T22:54:53.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35f045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35eecba0 0x55ec35f04b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681abfb0 con 0x55ec35edbee0 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681adaf0 con 0x55ec35eecba0 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa270744740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681ac680 con 0x55ec35edc800 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38722/0 (socket says 172.21.15.96:38722) 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 -- 172.21.15.96:0/1488632942 learned_addr learned my addr 172.21.15.96:0/1488632942 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa25ffff640 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35f045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa26c90c640 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35eecba0 0x55ec35f04b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 -- 172.21.15.96:0/1488632942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 msgr2=0x55ec35f045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35f045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 -- 172.21.15.96:0/1488632942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35eecba0 msgr2=0x55ec35f04b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35eecba0 0x55ec35f04b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.957+0000 7fa267fff640 1 -- 172.21.15.96:0/1488632942 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec35ee8030 con 0x55ec35edbee0 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa267fff640 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d010 tx=0x7fa2580cf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a508f094c7b43ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa25dffb640 1 -- 172.21.15.96:0/1488632942 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580abdf0 con 0x55ec35edbee0 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa25dffb640 1 -- 172.21.15.96:0/1488632942 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25000b0b0 con 0x55ec35edbee0 2024-09-10T22:54:53.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa25dffb640 1 -- 172.21.15.96:0/1488632942 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580a8cd0 con 0x55ec35edbee0 2024-09-10T22:54:53.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa270744740 1 -- 172.21.15.96:0/1488632942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 msgr2=0x55ec35edb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.958+0000 7fa270744740 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa25800d010 tx=0x7fa2580cf000 comp rx=0 tx=0).stop 2024-09-10T22:54:53.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 -- 172.21.15.96:0/1488632942 shutdown_connections 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35eecba0 0x55ec35f04b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35f045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 --2- 172.21.15.96:0/1488632942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35edbee0 0x55ec35edb470 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 -- 172.21.15.96:0/1488632942 >> 172.21.15.96:0/1488632942 conn(0x55ec35ed9d10 msgr2=0x55ec35edad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 -- 172.21.15.96:0/1488632942 shutdown_connections 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 -- 172.21.15.96:0/1488632942 wait complete. 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 -- start start 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.959+0000 7fa270744740 1 Processor -- start 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.960+0000 7fa270744740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ef4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.960+0000 7fa270744740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35ef5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.960+0000 7fa270744740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35ef16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.960+0000 7fa270744740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681ad420 con 0x55ec35eecba0 2024-09-10T22:54:53.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa26c90c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35ef16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa270744740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681acd50 con 0x55ec35edbee0 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa26c90c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35ef16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38726/0 (socket says 172.21.15.96:38726) 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa26c90c640 1 -- 172.21.15.96:0/645789724 learned_addr learned my addr 172.21.15.96:0/645789724 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681a9fb0 con 0x55ec35edc800 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35ef5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa267fff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ef4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa270744740 10 client.0 _ll_register_callbacks cb 0x7fa25800e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 -- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 msgr2=0x55ec35ef4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ef4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 -- 172.21.15.96:0/645789724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 msgr2=0x55ec35ef16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35ef16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 -- 172.21.15.96:0/645789724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec35ef1f00 con 0x55ec35edc800 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa25000b0b0 con 0x55ec35edc800 2024-09-10T22:54:53.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa25ffff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35ef5520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa250009900 tx=0x7fa250000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630d3e5b68be4526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.961+0000 7fa2477fe640 10 client.26692 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa2477fe640 1 -- 172.21.15.96:0/645789724 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250040040 con 0x55ec35edc800 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa2477fe640 1 -- 172.21.15.96:0/645789724 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa250032040 con 0x55ec35edc800 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa2477fe640 1 -- 172.21.15.96:0/645789724 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa250035040 con 0x55ec35edc800 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 10 client.26692 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ec35f6a4d0 con 0x55ec35edc800 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa2477fe640 1 -- 172.21.15.96:0/645789724 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa250032280 con 0x55ec35edc800 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 10 client.26692 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 20 client.26692 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35f6a6a0 0x55ec35f6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 4 client.26692 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:54:53.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.962+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec35eed760 con 0x55ec35f6a6a0 2024-09-10T22:54:53.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.963+0000 7fa267fff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35f6a6a0 0x55ec35f6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:53.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.963+0000 7fa267fff640 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35f6a6a0 0x55ec35f6ca90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81939f417f2cdde4 server_cookie=3ea4e34d7bfb2075 in_seq=0 out_seq=0 2024-09-10T22:54:53.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.963+0000 7fa2477fe640 10 client.26692 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa2477fe640 1 -- 172.21.15.96:0/645789724 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec35eed760 con 0x55ec35f6a6a0 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa2477fe640 10 client.26692 handle_command_reply: tid=0 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 client.26692 shutdown 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 msgr2=0x55ec35ef5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35ef5520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa250009900 tx=0x7fa250000bc0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 shutdown_connections 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ec35f6a6a0 0x55ec35f6ca90 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ec35eecba0 0x55ec35ef16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec35edc800 0x55ec35ef5520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.965+0000 7fa270744740 1 --2- 172.21.15.96:0/645789724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec35edbee0 0x55ec35ef4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 >> 172.21.15.96:0/645789724 conn(0x55ec35ed9d10 msgr2=0x55ec35ef7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa2477fe640 0 client.26692 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 shutdown_connections 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa270744740 1 -- 172.21.15.96:0/645789724 wait complete. 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa270744740 20 client.26692 trim_cache size 0 max 16384 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:54:53.970 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "496acac3-8d58-4071-a703-8099cbf09bdf", 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 msgr2=0x7fa2540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 0x7fa2540b7820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260009500 comp rx=0 tx=0).stop 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 msgr2=0x7fa268199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa25800ea60 tx=0x7fa25800ef20 comp rx=0 tx=0).stop 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 shutdown_connections 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2681087b0 0x7fa2681995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2540b5360 0x7fa2540b7820 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa268109130 0x7fa26819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 --2- 172.21.15.96:0/1475933321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa268102da0 0x7fa268199090 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:53.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.966+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 >> 172.21.15.96:0/1475933321 conn(0x7fa2680fe850 msgr2=0x7fa268106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:53.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.967+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 shutdown_connections 2024-09-10T22:54:53.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:53.967+0000 7fa26e36f640 1 -- 172.21.15.96:0/1475933321 wait complete. 2024-09-10T22:54:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:54 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:54:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:54 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:54 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:54:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:54 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:54:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:54 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:54 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:54:54.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:54 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:54 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:54:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:54 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:54:54.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:54 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:54 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:54:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:54 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:54 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:54:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:54 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:54:54.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:54 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:55.533 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:54:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:55 smithi096 ceph-mon[30869]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 733 op/s 2024-09-10T22:54:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:55 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:55 smithi121 ceph-mon[35119]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 733 op/s 2024-09-10T22:54:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:55 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:55 smithi117 ceph-mon[35081]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 733 op/s 2024-09-10T22:54:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:55 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:54:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:54:55.953 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 440.0 s, 1915.2 tps, lat 52.266 ms stddev 56.538 2024-09-10T22:54:56.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 -- 172.21.15.96:0/2563611164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc102dd0 msgr2=0x7f71cc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2563611164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc102dd0 0x7f71cc1031b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 -- 172.21.15.96:0/2563611164 shutdown_connections 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2563611164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 0x7f71cc108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2563611164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc109160 0x7f71cc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2563611164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc102dd0 0x7f71cc1031b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 -- 172.21.15.96:0/2563611164 >> 172.21.15.96:0/2563611164 conn(0x7f71cc0fe860 msgr2=0x7f71cc100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 -- 172.21.15.96:0/2563611164 shutdown_connections 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.668+0000 7f71d1f44640 1 -- 172.21.15.96:0/2563611164 wait complete. 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.669+0000 7f71d1f44640 1 Processor -- start 2024-09-10T22:54:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.669+0000 7f71d1f44640 1 -- start start 2024-09-10T22:54:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 0x7f71cc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc109160 0x7f71cc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc112e20 con 0x7f71cc109160 2024-09-10T22:54:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc112ca0 con 0x7f71cc1087e0 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71d1f44640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc112fa0 con 0x7f71cc102dd0 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc109160 0x7f71cc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52344/0 (socket says 172.21.15.96:52344) 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 -- 172.21.15.96:0/2674359365 learned_addr learned my addr 172.21.15.96:0/2674359365 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71c3fff640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 0x7f71cc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 -- 172.21.15.96:0/2674359365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 msgr2=0x7f71cc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 0x7f71cc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 -- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc109160 msgr2=0x7f71cc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc109160 0x7f71cc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 -- 172.21.15.96:0/2674359365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc19f340 con 0x7f71cc102dd0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.670+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005940 tx=0x7f71bc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.671+0000 7f71c97fa640 1 -- 172.21.15.96:0/2674359365 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc03f030 con 0x7f71cc102dd0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.671+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc19f5d0 con 0x7f71cc102dd0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.671+0000 7f71c97fa640 1 -- 172.21.15.96:0/2674359365 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc003aa0 con 0x7f71cc102dd0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.671+0000 7f71c97fa640 1 -- 172.21.15.96:0/2674359365 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc035070 con 0x7f71cc102dd0 2024-09-10T22:54:56.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.671+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a5cf0 con 0x7f71cc102dd0 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.674+0000 7f71c97fa640 1 -- 172.21.15.96:0/2674359365 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f71bc0035f0 con 0x7f71cc102dd0 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.675+0000 7f71d4319740 1 Processor -- start 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 -- start start 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71c97fa640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 0x7f71b80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x5569595f2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556959568e50 con 0x5569595fe950 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556959558280 con 0x5569595f20a0 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71d4319740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55695955c090 con 0x5569595f29c0 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53220/0 (socket says 172.21.15.96:53220) 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 -- 172.21.15.96:0/2385481265 learned_addr learned my addr 172.21.15.96:0/2385481265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71c97fa640 1 -- 172.21.15.96:0/2674359365 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f71bc1064c0 con 0x7f71cc102dd0 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71c3fff640 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 -- 172.21.15.96:0/2385481265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 msgr2=0x5569595fe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 -- 172.21.15.96:0/2385481265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 msgr2=0x5569595f2480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:54:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x5569595f2480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 -- 172.21.15.96:0/2385481265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569592b6650 con 0x5569595fe950 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 0x7f71b80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.676+0000 7f71cbfff640 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f71c4011410 tx=0x7f71c40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45153efed761be86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71c1ffb640 1 -- 172.21.15.96:0/2385481265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4012980 con 0x5569595fe950 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71c1ffb640 1 -- 172.21.15.96:0/2385481265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4016070 con 0x5569595fe950 2024-09-10T22:54:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71c1ffb640 1 -- 172.21.15.96:0/2385481265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c400f670 con 0x5569595fe950 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71cbfff640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 0x7f71b80b78a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71c401cfd0 tx=0x7f71c400e780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71d4319740 1 -- 172.21.15.96:0/2385481265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 msgr2=0x556959600d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.677+0000 7f71d4319740 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f71c4011410 tx=0x7f71c40118d0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 -- 172.21.15.96:0/2385481265 shutdown_connections 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959600d40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 --2- 172.21.15.96:0/2385481265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x5569595f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 -- 172.21.15.96:0/2385481265 >> 172.21.15.96:0/2385481265 conn(0x5569595efed0 msgr2=0x5569595f1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 -- 172.21.15.96:0/2385481265 shutdown_connections 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 -- 172.21.15.96:0/2385481265 wait complete. 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 -- start start 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 Processor -- start 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.678+0000 7f71d4319740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x5569595fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569593b9700 con 0x5569595fe950 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569594f3110 con 0x5569595f29c0 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5569596059e0 con 0x5569595f20a0 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52362/0 (socket says 172.21.15.96:52362) 2024-09-10T22:54:56.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 -- 172.21.15.96:0/1162439738 learned_addr learned my addr 172.21.15.96:0/1162439738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cbfff640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71c3fff640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x5569595fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 10 client.0 _ll_register_callbacks cb 0x55695950c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 -- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 msgr2=0x5569595fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x5569595fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 -- 172.21.15.96:0/1162439738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 msgr2=0x5569595fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 -- 172.21.15.96:0/1162439738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556959520c40 con 0x5569595f20a0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cca30 tx=0x7f71bc009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2dd60a67c75e84a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc049070 con 0x5569595f20a0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f71bc003c50 con 0x5569595f20a0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.679+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4016070 con 0x5569595f20a0 2024-09-10T22:54:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71bc0cc680 con 0x5569595f20a0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 10 client.26701 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556959611380 con 0x5569595f20a0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4016070 con 0x5569595f20a0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f71bc0c8fa0 con 0x5569595f20a0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 10 client.26701 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 20 client.26701 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959614b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 4 client.26701 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55695956ada0 con 0x556959612770 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.680+0000 7f71c3fff640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959614b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.681+0000 7f71aaffd640 10 client.26701 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.681+0000 7f71c3fff640 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959614b80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0dc40a734f28b server_cookie=3ef07a58c01fac96 in_seq=0 out_seq=0 2024-09-10T22:54:56.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.681+0000 7f71aaffd640 1 -- 172.21.15.96:0/1162439738 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55695956ada0 con 0x556959612770 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.681+0000 7f71aaffd640 10 client.26701 handle_command_reply: tid=0 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.681+0000 7f71d4319740 1 client.26701 shutdown 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 msgr2=0x5569595fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cca30 tx=0x7f71bc009620 comp rx=0 tx=0).stop 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 shutdown_connections 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959614b80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71aaffd640 0 client.26701 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x5569595fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 --2- 172.21.15.96:0/1162439738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x5569595fe2c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 >> 172.21.15.96:0/1162439738 conn(0x5569595efed0 msgr2=0x5569595f5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 shutdown_connections 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 1 -- 172.21.15.96:0/1162439738 wait complete. 2024-09-10T22:54:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.682+0000 7f71d4319740 20 client.26701 trim_cache size 0 max 16384 2024-09-10T22:54:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.706+0000 7f71d4319740 1 Processor -- start 2024-09-10T22:54:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.706+0000 7f71d4319740 1 -- start start 2024-09-10T22:54:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.706+0000 7f71d4319740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.706+0000 7f71d4319740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.706+0000 7f71d4319740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44124/0 (socket says 172.21.15.96:44124) 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:52368/0 (socket says 172.21.15.96:52368) 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71d4319740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1abfa0 con 0x5569595fe950 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71d4319740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1adae0 con 0x5569595f20a0 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71d4319740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ac670 con 0x5569595f29c0 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 -- 172.21.15.96:0/542431541 learned_addr learned my addr 172.21.15.96:0/542431541 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cbfff640 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 -- 172.21.15.96:0/542431541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 msgr2=0x5569595fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 -- 172.21.15.96:0/542431541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 msgr2=0x5569595faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 -- 172.21.15.96:0/542431541 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569595eb790 con 0x5569595f20a0 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71c3fff640 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71bc03b580 tx=0x7f71bc0c9440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fd329e8a1a8842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.707+0000 7f71c17fa640 1 -- 172.21.15.96:0/542431541 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc049050 con 0x5569595f20a0 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71c17fa640 1 -- 172.21.15.96:0/542431541 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc0b41b0 con 0x5569595f20a0 2024-09-10T22:54:56.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71c17fa640 1 -- 172.21.15.96:0/542431541 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0cc4a0 con 0x5569595f20a0 2024-09-10T22:54:56.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 -- 172.21.15.96:0/542431541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 msgr2=0x55695960ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71bc03b580 tx=0x7f71bc0c9440 comp rx=0 tx=0).stop 2024-09-10T22:54:56.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 -- 172.21.15.96:0/542431541 shutdown_connections 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x5569595faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f29c0 0x5569595fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 --2- 172.21.15.96:0/542431541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f20a0 0x55695960ec40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 -- 172.21.15.96:0/542431541 >> 172.21.15.96:0/542431541 conn(0x5569595efed0 msgr2=0x5569595f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.708+0000 7f71d4319740 1 -- 172.21.15.96:0/542431541 shutdown_connections 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 -- 172.21.15.96:0/542431541 wait complete. 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 -- start start 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 Processor -- start 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x55695960b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959609230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ad410 con 0x5569595fe950 2024-09-10T22:54:56.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1acd40 con 0x5569595f29c0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.709+0000 7f71d4319740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a9fa0 con 0x5569595f20a0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71d4319740 10 client.0 _ll_register_callbacks cb 0x7f71bc005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959609230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44126/0 (socket says 172.21.15.96:44126) 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 -- 172.21.15.96:0/3758096727 learned_addr learned my addr 172.21.15.96:0/3758096727 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x55695960b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 -- 172.21.15.96:0/3758096727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 msgr2=0x55695960b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x55695960b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 -- 172.21.15.96:0/3758096727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 msgr2=0x556959609230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959609230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 -- 172.21.15.96:0/3758096727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556959609870 con 0x5569595f29c0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71c3fff640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5569596153f0 tx=0x7f71b40039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ea9977205b9ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:54:56.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4029060 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f71b40106f0 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.710+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc0b41b0 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71b40108f0 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 10 client.26654 fetch_fsmap learned FSMap version 32 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556959680520 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc0b41b0 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f71b402e070 con 0x5569595f29c0 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 10 client.26654 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 20 client.26654 populate_metadata read hostname 'smithi096' 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959682cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 4 client.26654 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:54:56.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556959566850 con 0x556959612770 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.711+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959682cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71cb7fe640 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959682cc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13e3b1034399955 server_cookie=d1f0dad3ef9d251e in_seq=0 out_seq=0 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71aaffd640 10 client.26654 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71aaffd640 1 -- 172.21.15.96:0/3758096727 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x556959566850 con 0x556959612770 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71aaffd640 10 client.26654 handle_command_reply: tid=0 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 client.26654 shutdown 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 msgr2=0x556959608cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5569596153f0 tx=0x7f71b40039c0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 shutdown_connections 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556959612770 0x556959682cc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569595f29c0 0x556959608cf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569595fe950 0x556959609230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71aaffd640 0 client.26654 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:54:56.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 --2- 172.21.15.96:0/3758096727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5569595f20a0 0x55695960b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 >> 172.21.15.96:0/3758096727 conn(0x5569595efed0 msgr2=0x5569595fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 shutdown_connections 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 1 -- 172.21.15.96:0/3758096727 wait complete. 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.712+0000 7f71d4319740 20 client.26654 trim_cache size 0 max 16384 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "496acac3-8d58-4071-a703-8099cbf09bdf", 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:54:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:54:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:54:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:54:56.726 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:54:56.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 msgr2=0x7f71b80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 0x7f71b80b78a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71c401cfd0 tx=0x7f71c400e780 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 msgr2=0x7f71cc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005940 tx=0x7f71bc004270 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 shutdown_connections 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71cc1087e0 0x7f71cc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f71b80b53e0 0x7f71b80b78a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71cc109160 0x7f71cc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 --2- 172.21.15.96:0/2674359365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71cc102dd0 0x7f71cc199290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 >> 172.21.15.96:0/2674359365 conn(0x7f71cc0fe860 msgr2=0x7f71cc1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 shutdown_connections 2024-09-10T22:54:56.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:54:56.713+0000 7f71d1f44640 1 -- 172.21.15.96:0/2674359365 wait complete. 2024-09-10T22:54:57.456 INFO:tasks.cephfs.filesystem:scrub status for tag:496acac3-8d58-4071-a703-8099cbf09bdf - {'path': '/', 'tag': '496acac3-8d58-4071-a703-8099cbf09bdf', 'options': 'recursive,force'} 2024-09-10T22:54:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:57 smithi096 ceph-mon[30869]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 722 op/s 2024-09-10T22:54:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:57 smithi121 ceph-mon[35119]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 722 op/s 2024-09-10T22:54:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:57 smithi117 ceph-mon[35081]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 722 op/s 2024-09-10T22:54:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:59 smithi096 ceph-mon[30869]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 600 op/s 2024-09-10T22:54:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:54:59 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:54:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:59 smithi121 ceph-mon[35119]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 600 op/s 2024-09-10T22:54:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:54:59 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:54:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:59 smithi117 ceph-mon[35081]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 600 op/s 2024-09-10T22:54:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:54:59 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:55:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:00 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:55:00.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:00 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:55:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:00 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:55:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 445.0 s, 1772.2 tps, lat 56.628 ms stddev 60.895 2024-09-10T22:55:01.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:01 smithi096 ceph-mon[30869]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 824 op/s 2024-09-10T22:55:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:01 smithi121 ceph-mon[35119]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 824 op/s 2024-09-10T22:55:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:01 smithi117 ceph-mon[35081]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.9 MiB/s wr, 824 op/s 2024-09-10T22:55:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:03 smithi096 ceph-mon[30869]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.2 MiB/s wr, 744 op/s 2024-09-10T22:55:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:03 smithi121 ceph-mon[35119]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.2 MiB/s wr, 744 op/s 2024-09-10T22:55:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:03 smithi117 ceph-mon[35081]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.2 MiB/s wr, 744 op/s 2024-09-10T22:55:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.897+0000 7ff284700640 1 -- 172.21.15.96:0/4129659319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c1044a0 msgr2=0x7ff27c10ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.897+0000 7ff284700640 1 --2- 172.21.15.96:0/4129659319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c1044a0 0x7ff27c10ae20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff274009cd0 tx=0x7ff27402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 -- 172.21.15.96:0/4129659319 shutdown_connections 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 --2- 172.21.15.96:0/4129659319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c1044a0 0x7ff27c10ae20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 --2- 172.21.15.96:0/4129659319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 --2- 172.21.15.96:0/4129659319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c102990 0x7ff27c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 -- 172.21.15.96:0/4129659319 >> 172.21.15.96:0/4129659319 conn(0x7ff27c0fe250 msgr2=0x7ff27c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.898+0000 7ff284700640 1 -- 172.21.15.96:0/4129659319 shutdown_connections 2024-09-10T22:55:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.899+0000 7ff284700640 1 -- 172.21.15.96:0/4129659319 wait complete. 2024-09-10T22:55:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 Processor -- start 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 -- start start 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c1044a0 0x7ff27c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10e650 con 0x7ff27c102990 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10e4d0 con 0x7ff27c1044a0 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.900+0000 7ff284700640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10e7d0 con 0x7ff27c103ae0 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53258/0 (socket says 172.21.15.96:53258) 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff28249c640 1 -- 172.21.15.96:0/2098216001 learned_addr learned my addr 172.21.15.96:0/2098216001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff282c9d640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c1044a0 0x7ff27c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 -- 172.21.15.96:0/2098216001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c1044a0 msgr2=0x7ff27c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c1044a0 0x7ff27c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 -- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 msgr2=0x7ff27c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 -- 172.21.15.96:0/2098216001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c19a870 con 0x7ff27c103ae0 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff28249c640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff281c9b640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c199380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e9f0 tx=0x7ff26c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff2737fe640 1 -- 172.21.15.96:0/2098216001 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00ccc0 con 0x7ff27c103ae0 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c19ab60 con 0x7ff27c103ae0 2024-09-10T22:55:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff2737fe640 1 -- 172.21.15.96:0/2098216001 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff26c005020 con 0x7ff27c103ae0 2024-09-10T22:55:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff2737fe640 1 -- 172.21.15.96:0/2098216001 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c004a00 con 0x7ff27c103ae0 2024-09-10T22:55:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.901+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c1a1230 con 0x7ff27c103ae0 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2737fe640 1 -- 172.21.15.96:0/2098216001 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff26c004ba0 con 0x7ff27c103ae0 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 Processor -- start 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 -- start start 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feae9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c1fea526f0 con 0x55c1feae4c70 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1fea41b20 con 0x55c1feae9fe0 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff2874cc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c1fea45930 con 0x55c1feae75a0 2024-09-10T22:55:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff282c9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff281c9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff28249c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feae9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff282c9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53268/0 (socket says 172.21.15.96:53268) 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.904+0000 7ff282c9d640 1 -- 172.21.15.96:0/756540866 learned_addr learned my addr 172.21.15.96:0/756540866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 -- 172.21.15.96:0/756540866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 msgr2=0x55c1feaec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 -- 172.21.15.96:0/756540866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 msgr2=0x55c1feae7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 -- 172.21.15.96:0/756540866 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1fea07530 con 0x55c1feae75a0 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff282c9d640 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff28249c640 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feae9990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2780114f0 tx=0x7ff2780119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fac9aef5b2eb00e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff270ff9640 1 -- 172.21.15.96:0/756540866 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27801e070 con 0x55c1feae75a0 2024-09-10T22:55:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff270ff9640 1 -- 172.21.15.96:0/756540866 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780035f0 con 0x55c1feae75a0 2024-09-10T22:55:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.905+0000 7ff270ff9640 1 -- 172.21.15.96:0/756540866 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27800f820 con 0x55c1feae75a0 2024-09-10T22:55:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2737fe640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 0x7ff2680b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff282c9d640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 0x7ff2680b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 -- 172.21.15.96:0/756540866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 msgr2=0x55c1feae9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feae9990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2780114f0 tx=0x7ff2780119b0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2737fe640 1 -- 172.21.15.96:0/2098216001 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff26c0e5ad0 con 0x7ff27c103ae0 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 -- 172.21.15.96:0/756540866 shutdown_connections 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feae9990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 --2- 172.21.15.96:0/756540866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feae7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff2874cc740 1 -- 172.21.15.96:0/756540866 >> 172.21.15.96:0/756540866 conn(0x55c1feadb9b0 msgr2=0x55c1feadc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.906+0000 7ff282c9d640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 0x7ff2680b78a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff274005e30 tx=0x7ff274031850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 -- 172.21.15.96:0/756540866 shutdown_connections 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 -- 172.21.15.96:0/756540866 wait complete. 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 -- start start 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 Processor -- start 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 0x55c1feaee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feaef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff2874cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe979ab0 con 0x55c1feae4c70 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff2874cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe9e7730 con 0x55c1feae75a0 2024-09-10T22:55:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff2874cc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c1feaecfc0 con 0x55c1feae9fe0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.907+0000 7ff281c9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 0x55c1feaee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53284/0 (socket says 172.21.15.96:53284) 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff28249c640 1 -- 172.21.15.96:0/636576034 learned_addr learned my addr 172.21.15.96:0/636576034 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff2874cc740 10 client.0 _ll_register_callbacks cb 0x55c1fea3b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff282c9d640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feaef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 -- 172.21.15.96:0/636576034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 msgr2=0x55c1feaef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feaef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 -- 172.21.15.96:0/636576034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 msgr2=0x55c1feaee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 -- 172.21.15.96:0/636576034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1fe7a3790 con 0x55c1feae75a0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff28249c640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff281c9b640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 0x55c1feaee8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0a9500 tx=0x7ff26c0a88e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3fedc9952b98ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c01c070 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2780035f0 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.908+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff26c0054f0 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.909+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff26c01c070 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.909+0000 7ff2874cc740 10 client.26666 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.909+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1fe81c620 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.910+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780035f0 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.910+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff26c0b5390 con 0x55c1feae75a0 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 10 client.26666 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 10 client.26666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 20 client.26666 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feaf9be0 0x55c1feafbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 4 client.26666 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1fea54640 con 0x55c1feaf9be0 2024-09-10T22:55:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.911+0000 7ff28249c640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feaf9be0 0x55c1feafbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff28249c640 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feaf9be0 0x55c1feafbfd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e5b9d3f680940a server_cookie=77f433e9a32982ee in_seq=0 out_seq=0 2024-09-10T22:55:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff265ffb640 10 client.26666 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff265ffb640 1 -- 172.21.15.96:0/636576034 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1fea54640 con 0x55c1feaf9be0 2024-09-10T22:55:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff265ffb640 10 client.26666 handle_command_reply: tid=0 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff2874cc740 1 client.26666 shutdown 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.912+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 msgr2=0x55c1feaee8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 0x55c1feaee8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0a9500 tx=0x7ff26c0a88e0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 shutdown_connections 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feaf9be0 0x55c1feafbfd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae75a0 0x55c1feaee8f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feaef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff265ffb640 0 client.26666 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 --2- 172.21.15.96:0/636576034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 >> 172.21.15.96:0/636576034 conn(0x55c1feadb9b0 msgr2=0x55c1feadabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 shutdown_connections 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 1 -- 172.21.15.96:0/636576034 wait complete. 2024-09-10T22:55:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.913+0000 7ff2874cc740 20 client.26666 trim_cache size 0 max 16384 2024-09-10T22:55:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.930+0000 7ff2874cc740 1 Processor -- start 2024-09-10T22:55:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.930+0000 7ff2874cc740 1 -- start start 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 0x55c1feadfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 0x55c1feae0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feb008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff28249c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 0x55c1feadfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff28249c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 0x55c1feadfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44186/0 (socket says 172.21.15.96:44186) 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff281c9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 0x55c1feae0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff28249c640 1 -- 172.21.15.96:0/11575930 learned_addr learned my addr 172.21.15.96:0/11575930 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a7bb0 con 0x55c1feae75a0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff282c9d640 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feb008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a96f0 con 0x55c1feae4c70 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.931+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a8280 con 0x55c1feae9fe0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 -- 172.21.15.96:0/11575930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 msgr2=0x55c1feb008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feb008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 -- 172.21.15.96:0/11575930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 msgr2=0x55c1feadfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 0x55c1feadfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 -- 172.21.15.96:0/11575930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1feaef830 con 0x55c1feae75a0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff281c9b640 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 0x55c1feae0200 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ac220 tx=0x7ff26c0e3650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f673a63e6d14eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff267fff640 1 -- 172.21.15.96:0/11575930 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0aba00 con 0x55c1feae75a0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff267fff640 1 -- 172.21.15.96:0/11575930 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff26c0abda0 con 0x55c1feae75a0 2024-09-10T22:55:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.932+0000 7ff267fff640 1 -- 172.21.15.96:0/11575930 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0cf0c0 con 0x55c1feae75a0 2024-09-10T22:55:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 msgr2=0x55c1feae0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 0x55c1feae0200 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ac220 tx=0x7ff26c0e3650 comp rx=0 tx=0).stop 2024-09-10T22:55:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 shutdown_connections 2024-09-10T22:55:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae9fe0 0x55c1feb008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae75a0 0x55c1feae0200 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 --2- 172.21.15.96:0/11575930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae4c70 0x55c1feadfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 >> 172.21.15.96:0/11575930 conn(0x55c1feadb9b0 msgr2=0x55c1feadea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 shutdown_connections 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- 172.21.15.96:0/11575930 wait complete. 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.933+0000 7ff2874cc740 1 -- start start 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff2874cc740 1 Processor -- start 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feaf4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff2874cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaf53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff281c9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feaf4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff28249c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53296/0 (socket says 172.21.15.96:53296) 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff28249c640 1 -- 172.21.15.96:0/3830520739 learned_addr learned my addr 172.21.15.96:0/3830520739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a9020 con 0x55c1feae4c70 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.934+0000 7ff282c9d640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaf53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a8950 con 0x55c1feae9fe0 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1a5bb0 con 0x55c1feae75a0 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff2874cc740 10 client.0 _ll_register_callbacks cb 0x7ff26c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 -- 172.21.15.96:0/3830520739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 msgr2=0x55c1feaf4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feaf4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 -- 172.21.15.96:0/3830520739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 msgr2=0x55c1feaf5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1feae9d00 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff28249c640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff281c9b640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feaf4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff282c9d640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaf53f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff27402a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6638e764aadd01a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.935+0000 7ff265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274004250 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff2740043f0 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff26c0abda0 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2740045f0 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 10 client.26672 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1feb66c20 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274056070 con 0x55c1feae9fe0 2024-09-10T22:55:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff274030070 con 0x55c1feae9fe0 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 10 client.26672 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 20 client.26672 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feb67600 0x55c1feb69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 4 client.26672 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.936+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff26c0abda0 con 0x55c1feb67600 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.937+0000 7ff28249c640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feb67600 0x55c1feb69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.937+0000 7ff28249c640 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feb67600 0x55c1feb69a20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d6dc742cedcddb server_cookie=2995b1871f1e13fa in_seq=0 out_seq=0 2024-09-10T22:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.937+0000 7ff265ffb640 10 client.26672 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.954+0000 7ff265ffb640 1 -- 172.21.15.96:0/3830520739 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff26c0abda0 con 0x55c1feb67600 2024-09-10T22:55:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.954+0000 7ff265ffb640 10 client.26672 handle_command_reply: tid=0 2024-09-10T22:55:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.954+0000 7ff2874cc740 1 client.26672 shutdown 2024-09-10T22:55:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 msgr2=0x55c1feaf53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaf53f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff27402a8e0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 shutdown_connections 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c1feb67600 0x55c1feb69a20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1feae9fe0 0x55c1feaf53f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff265ffb640 0 client.26672 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1feae75a0 0x55c1feaf4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 --2- 172.21.15.96:0/3830520739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c1feae4c70 0x55c1feaf5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 >> 172.21.15.96:0/3830520739 conn(0x55c1feadb9b0 msgr2=0x55c1feadf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 shutdown_connections 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 1 -- 172.21.15.96:0/3830520739 wait complete. 2024-09-10T22:55:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.955+0000 7ff2874cc740 20 client.26672 trim_cache size 0 max 16384 2024-09-10T22:55:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:55:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:55:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:55:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 msgr2=0x7ff2680b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 0x7ff2680b78a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff274005e30 tx=0x7ff274031850 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 msgr2=0x7ff27c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c199380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e9f0 tx=0x7ff26c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 shutdown_connections 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff27c1044a0 0x7ff27c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2680b53e0 0x7ff2680b78a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff27c103ae0 0x7ff27c199380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 --2- 172.21.15.96:0/2098216001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c102990 0x7ff27c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 >> 172.21.15.96:0/2098216001 conn(0x7ff27c0fe250 msgr2=0x7ff27c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 shutdown_connections 2024-09-10T22:55:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:04.956+0000 7ff284700640 1 -- 172.21.15.96:0/2098216001 wait complete. 2024-09-10T22:55:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:05 smithi096 ceph-mon[30869]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 773 op/s 2024-09-10T22:55:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:05 smithi121 ceph-mon[35119]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 773 op/s 2024-09-10T22:55:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:05 smithi117 ceph-mon[35081]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 773 op/s 2024-09-10T22:55:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 450.0 s, 2133.6 tps, lat 46.772 ms stddev 52.375 2024-09-10T22:55:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:07 smithi096 ceph-mon[30869]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-09-10T22:55:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:07 smithi121 ceph-mon[35119]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-09-10T22:55:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:07 smithi117 ceph-mon[35081]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-09-10T22:55:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:09 smithi096 ceph-mon[30869]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.8 MiB/s wr, 602 op/s 2024-09-10T22:55:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:09 smithi121 ceph-mon[35119]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.8 MiB/s wr, 602 op/s 2024-09-10T22:55:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:09 smithi117 ceph-mon[35081]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.8 MiB/s wr, 602 op/s 2024-09-10T22:55:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 455.0 s, 2652.6 tps, lat 37.609 ms stddev 42.567 2024-09-10T22:55:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:11 smithi096 ceph-mon[30869]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-09-10T22:55:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:11 smithi121 ceph-mon[35119]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-09-10T22:55:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:11 smithi117 ceph-mon[35081]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.0 MiB/s wr, 820 op/s 2024-09-10T22:55:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:13 smithi096 ceph-mon[30869]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:14.008 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:13 smithi117 ceph-mon[35081]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:14.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:13 smithi121 ceph-mon[35119]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:15 smithi096 ceph-mon[30869]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:15 smithi121 ceph-mon[35119]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:15 smithi117 ceph-mon[35081]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-09-10T22:55:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 460.0 s, 2066.4 tps, lat 48.682 ms stddev 54.648 2024-09-10T22:55:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:17 smithi096 ceph-mon[30869]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T22:55:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:17 smithi121 ceph-mon[35119]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T22:55:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:17 smithi117 ceph-mon[35081]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 671 op/s 2024-09-10T22:55:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55e5d9b640 1 -- 172.21.15.96:0/1835686868 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8035020 con 0x7f55e8103fa0 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 -- 172.21.15.96:0/1835686868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 msgr2=0x7f55e81043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 --2- 172.21.15.96:0/1835686868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e81043a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009930 tx=0x7f55d802a420 comp rx=0 tx=0).stop 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 -- 172.21.15.96:0/1835686868 shutdown_connections 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 --2- 172.21.15.96:0/1835686868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e81048e0 0x7f55e8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 --2- 172.21.15.96:0/1835686868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e8101130 0x7f55e810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 --2- 172.21.15.96:0/1835686868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e81043a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.068+0000 7f55ece52640 1 -- 172.21.15.96:0/1835686868 >> 172.21.15.96:0/1835686868 conn(0x7f55e80fc880 msgr2=0x7f55e80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.069+0000 7f55ece52640 1 -- 172.21.15.96:0/1835686868 shutdown_connections 2024-09-10T22:55:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.069+0000 7f55ece52640 1 -- 172.21.15.96:0/1835686868 wait complete. 2024-09-10T22:55:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.069+0000 7f55ece52640 1 Processor -- start 2024-09-10T22:55:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.069+0000 7f55ece52640 1 -- start start 2024-09-10T22:55:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e8101130 0x7f55e819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 0x7f55e819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81108f0 con 0x7f55e8103fa0 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8103af0 con 0x7f55e8101130 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55ece52640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8110a70 con 0x7f55e81048e0 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55e659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e8101130 0x7f55e819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45812/0 (socket says 172.21.15.96:45812) 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 -- 172.21.15.96:0/3840298211 learned_addr learned my addr 172.21.15.96:0/3840298211 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 0x7f55e819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 -- 172.21.15.96:0/3840298211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 msgr2=0x7f55e819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 0x7f55e819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 -- 172.21.15.96:0/3840298211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e8101130 msgr2=0x7f55e819af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e8101130 0x7f55e819af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 -- 172.21.15.96:0/3840298211 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e819cab0 con 0x7f55e8103fa0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 0x7f55e819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.070+0000 7f55dffff640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9c0 tx=0x7f55d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.071+0000 7f55df7fe640 1 -- 172.21.15.96:0/3840298211 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000cd80 con 0x7f55e8103fa0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.071+0000 7f55df7fe640 1 -- 172.21.15.96:0/3840298211 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d00035f0 con 0x7f55e8103fa0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.071+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e819cda0 con 0x7f55e8103fa0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.071+0000 7f55df7fe640 1 -- 172.21.15.96:0/3840298211 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0010690 con 0x7f55e8103fa0 2024-09-10T22:55:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.071+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e81a3470 con 0x7f55e8103fa0 2024-09-10T22:55:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.073+0000 7f55df7fe640 1 -- 172.21.15.96:0/3840298211 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f55d0004000 con 0x7f55e8103fa0 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.073+0000 7f55efc22740 1 Processor -- start 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 -- start start 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629c50056f0 con 0x5629c5093860 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629c4ff4b20 con 0x5629c5098bd0 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55efc22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5629c4ff8930 con 0x5629c5096190 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55dffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55e659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55e659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45160/0 (socket says 172.21.15.96:45160) 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55e659c640 1 -- 172.21.15.96:0/634454294 learned_addr learned my addr 172.21.15.96:0/634454294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.074+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45828/0 (socket says 172.21.15.96:45828) 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55df7fe640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 0x7f55d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 -- 172.21.15.96:0/634454294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 msgr2=0x5629c509b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 -- 172.21.15.96:0/634454294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 msgr2=0x5629c5095c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 -- 172.21.15.96:0/634454294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c4fba530 con 0x5629c5096190 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55dffff640 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55df7fe640 1 -- 172.21.15.96:0/3840298211 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55d00e5ef0 con 0x7f55e8103fa0 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 0x7f55d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e659c640 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009930 tx=0x7f55d8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ab3f49bce37d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55dcff9640 1 -- 172.21.15.96:0/634454294 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8035030 con 0x5629c5096190 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55dcff9640 1 -- 172.21.15.96:0/634454294 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d8040070 con 0x5629c5096190 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55dcff9640 1 -- 172.21.15.96:0/634454294 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8003970 con 0x5629c5096190 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.075+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 0x7f55d40b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 -- 172.21.15.96:0/634454294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 msgr2=0x5629c5098580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009930 tx=0x7f55d8005bd0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 -- 172.21.15.96:0/634454294 shutdown_connections 2024-09-10T22:55:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c5098580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 --2- 172.21.15.96:0/634454294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c5095c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.076+0000 7f55efc22740 1 -- 172.21.15.96:0/634454294 >> 172.21.15.96:0/634454294 conn(0x5629c508e9b0 msgr2=0x5629c508f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.078+0000 7f55efc22740 1 -- 172.21.15.96:0/634454294 shutdown_connections 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.078+0000 7f55efc22740 1 -- 172.21.15.96:0/634454294 wait complete. 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 -- start start 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 Processor -- start 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c509d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5629c4f2cab0 con 0x5629c5093860 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629c4f9a730 con 0x5629c5098bd0 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5629c509bb50 con 0x5629c5096190 2024-09-10T22:55:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55efc22740 10 client.0 _ll_register_callbacks cb 0x5629c4fee6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55dffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c509d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55e659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45836/0 (socket says 172.21.15.96:45836) 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.079+0000 7f55e659c640 1 -- 172.21.15.96:0/983259159 learned_addr learned my addr 172.21.15.96:0/983259159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 -- 172.21.15.96:0/983259159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 msgr2=0x5629c509d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c509d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 -- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 msgr2=0x5629c509e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 -- 172.21.15.96:0/983259159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c4d56790 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55e659c640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f55d8000c00 tx=0x7f55d8004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474c4b442692dd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8035070 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55d8031ae0 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d8040070 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55d8009520 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55efc22740 10 client.17100 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5629c4dcf620 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.080+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d8040070 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f55d8045770 con 0x5629c5093860 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 10 client.17100 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 20 client.17100 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c50a8310 0x5629c50aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 4 client.17100 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c5007640 con 0x5629c50a8310 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55dffff640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c50a8310 0x5629c50aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55cdffb640 10 client.17100 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.081+0000 7f55dffff640 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c50a8310 0x5629c50aa700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7f5c83d529ad95 server_cookie=8640d531fbd9d40 in_seq=0 out_seq=0 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55cdffb640 1 -- 172.21.15.96:0/983259159 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629c5007640 con 0x5629c50a8310 2024-09-10T22:55:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55cdffb640 10 client.17100 handle_command_reply: tid=0 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 client.17100 shutdown 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 msgr2=0x5629c509ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f55d8000c00 tx=0x7f55d8004330 comp rx=0 tx=0).stop 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 shutdown_connections 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c50a8310 0x5629c50aa700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5098bd0 0x5629c509e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5096190 0x5629c509d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55cdffb640 0 client.17100 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 --2- 172.21.15.96:0/983259159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5093860 0x5629c509ce70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 >> 172.21.15.96:0/983259159 conn(0x5629c508e9b0 msgr2=0x5629c508dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 shutdown_connections 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 1 -- 172.21.15.96:0/983259159 wait complete. 2024-09-10T22:55:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.082+0000 7f55efc22740 20 client.17100 trim_cache size 0 max 16384 2024-09-10T22:55:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.107+0000 7f55efc22740 1 Processor -- start 2024-09-10T22:55:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 -- start start 2024-09-10T22:55:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5093860 0x5629c5098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5096190 0x5629c50994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81abfb0 con 0x5629c5096190 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81adaf0 con 0x5629c5093860 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.111+0000 7f55efc22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81ac680 con 0x5629c5098bd0 2024-09-10T22:55:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.113+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.113+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45180/0 (socket says 172.21.15.96:45180) 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.113+0000 7f55e6d9d640 1 -- 172.21.15.96:0/2722742146 learned_addr learned my addr 172.21.15.96:0/2722742146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.113+0000 7f55e659c640 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5093860 0x5629c5098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.114+0000 7f55dffff640 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5096190 0x5629c50994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.115+0000 7f55e6d9d640 1 -- 172.21.15.96:0/2722742146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5093860 msgr2=0x5629c5098fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.115+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5093860 0x5629c5098fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.115+0000 7f55e6d9d640 1 -- 172.21.15.96:0/2722742146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5096190 msgr2=0x5629c50994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.115+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5096190 0x5629c50994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.115+0000 7f55e6d9d640 1 -- 172.21.15.96:0/2722742146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c509e450 con 0x5629c5098bd0 2024-09-10T22:55:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.120+0000 7f55e6d9d640 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55e0003cd0 tx=0x7f55e001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c839204d1e9870 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.121+0000 7f55cffff640 1 -- 172.21.15.96:0/2722742146 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e001daa0 con 0x5629c5098bd0 2024-09-10T22:55:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.122+0000 7f55cffff640 1 -- 172.21.15.96:0/2722742146 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e0050070 con 0x5629c5098bd0 2024-09-10T22:55:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.122+0000 7f55cffff640 1 -- 172.21.15.96:0/2722742146 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000f3c0 con 0x5629c5098bd0 2024-09-10T22:55:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.124+0000 7f55efc22740 1 -- 172.21.15.96:0/2722742146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 msgr2=0x5629c508d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.124+0000 7f55efc22740 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55e0003cd0 tx=0x7f55e001a000 comp rx=0 tx=0).stop 2024-09-10T22:55:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.125+0000 7f55efc22740 1 -- 172.21.15.96:0/2722742146 shutdown_connections 2024-09-10T22:55:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.125+0000 7f55efc22740 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5098bd0 0x5629c508d3d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.125+0000 7f55efc22740 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5096190 0x5629c50994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.125+0000 7f55efc22740 1 --2- 172.21.15.96:0/2722742146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5093860 0x5629c5098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.125+0000 7f55efc22740 1 -- 172.21.15.96:0/2722742146 >> 172.21.15.96:0/2722742146 conn(0x5629c508e9b0 msgr2=0x5629c5092d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.126+0000 7f55efc22740 1 -- 172.21.15.96:0/2722742146 shutdown_connections 2024-09-10T22:55:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.126+0000 7f55efc22740 1 -- 172.21.15.96:0/2722742146 wait complete. 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 -- start start 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 Processor -- start 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5093860 0x5629c50a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5098bd0 0x5629c50a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81ad420 con 0x5629c5098bd0 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81acd50 con 0x5629c5096190 2024-09-10T22:55:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81a9fe0 con 0x5629c5093860 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55efc22740 10 client.0 _ll_register_callbacks cb 0x7f55e0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.127+0000 7f55dffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52420/0 (socket says 172.21.15.96:52420) 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 -- 172.21.15.96:0/1941006724 learned_addr learned my addr 172.21.15.96:0/1941006724 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55e659c640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5093860 0x5629c50a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 -- 172.21.15.96:0/1941006724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5093860 msgr2=0x5629c50a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5093860 0x5629c50a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 -- 172.21.15.96:0/1941006724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5098bd0 msgr2=0x5629c50a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5098bd0 0x5629c50a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 -- 172.21.15.96:0/1941006724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c50a3900 con 0x5629c5096190 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55dffff640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55d00058c0 tx=0x7f55d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2f4a8744d14b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.128+0000 7f55cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.129+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00ac0a0 con 0x5629c5096190 2024-09-10T22:55:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.129+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e0050070 con 0x5629c5096190 2024-09-10T22:55:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.129+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55d00a89c0 con 0x5629c5096190 2024-09-10T22:55:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.131+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55d00abec0 con 0x5629c5096190 2024-09-10T22:55:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.132+0000 7f55efc22740 10 client.26687 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.132+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5629c50a4d40 con 0x5629c5096190 2024-09-10T22:55:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.134+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e0050070 con 0x5629c5096190 2024-09-10T22:55:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.135+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f55d0004320 con 0x5629c5096190 2024-09-10T22:55:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 10 client.26687 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 10 client.26687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 20 client.26687 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c5115560 0x5629c5117980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 4 client.26687 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629c50030f0 con 0x5629c5115560 2024-09-10T22:55:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.137+0000 7f55e659c640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c5115560 0x5629c5117980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.138+0000 7f55e659c640 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c5115560 0x5629c5117980 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d9031c8555ebaf server_cookie=f2cafa1e8cadb37f in_seq=0 out_seq=0 2024-09-10T22:55:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.138+0000 7f55cdffb640 10 client.26687 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55cdffb640 1 -- 172.21.15.96:0/1941006724 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629c50030f0 con 0x5629c5115560 2024-09-10T22:55:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55cdffb640 10 client.26687 handle_command_reply: tid=0 2024-09-10T22:55:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 client.26687 shutdown 2024-09-10T22:55:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 msgr2=0x5629c50a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f55d00058c0 tx=0x7f55d0004730 comp rx=0 tx=0).stop 2024-09-10T22:55:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 shutdown_connections 2024-09-10T22:55:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5629c5115560 0x5629c5117980 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c5096190 0x5629c50a2500 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5629c5098bd0 0x5629c50a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55cdffb640 0 client.26687 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 --2- 172.21.15.96:0/1941006724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5629c5093860 0x5629c50a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 >> 172.21.15.96:0/1941006724 conn(0x5629c508e9b0 msgr2=0x5629c5092d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 shutdown_connections 2024-09-10T22:55:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.152+0000 7f55efc22740 1 -- 172.21.15.96:0/1941006724 wait complete. 2024-09-10T22:55:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55efc22740 20 client.26687 trim_cache size 0 max 16384 2024-09-10T22:55:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:55:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:55:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:55:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:55:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 msgr2=0x7f55d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 0x7f55d40b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0004a40 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 msgr2=0x7f55e819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9c0 tx=0x7f55d000ee80 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 shutdown_connections 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55d40b52e0 0x7f55d40b77a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55e81048e0 0x7f55e819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e8103fa0 0x7f55e819b5c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 --2- 172.21.15.96:0/3840298211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e8101130 0x7f55e819af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 >> 172.21.15.96:0/3840298211 conn(0x7f55e80fc880 msgr2=0x7f55e8107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 shutdown_connections 2024-09-10T22:55:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:18.153+0000 7f55ece52640 1 -- 172.21.15.96:0/3840298211 wait complete. 2024-09-10T22:55:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:19 smithi096 ceph-mon[30869]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 590 op/s 2024-09-10T22:55:19.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:19 smithi121 ceph-mon[35119]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 590 op/s 2024-09-10T22:55:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:19 smithi117 ceph-mon[35081]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 590 op/s 2024-09-10T22:55:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 465.0 s, 2325.0 tps, lat 42.847 ms stddev 50.276 2024-09-10T22:55:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:21 smithi096 ceph-mon[30869]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 784 op/s 2024-09-10T22:55:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:21 smithi121 ceph-mon[35119]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 784 op/s 2024-09-10T22:55:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:21 smithi117 ceph-mon[35081]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 784 op/s 2024-09-10T22:55:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:23 smithi096 ceph-mon[30869]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 618 op/s 2024-09-10T22:55:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:23 smithi121 ceph-mon[35119]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 618 op/s 2024-09-10T22:55:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:23 smithi117 ceph-mon[35081]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 618 op/s 2024-09-10T22:55:24.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:24 smithi096 ceph-mon[30869]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-09-10T22:55:24.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:24 smithi121 ceph-mon[35119]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-09-10T22:55:24.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:24 smithi117 ceph-mon[35081]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 718 op/s 2024-09-10T22:55:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 470.0 s, 1751.4 tps, lat 57.040 ms stddev 59.495 2024-09-10T22:55:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:26 smithi096 ceph-mon[30869]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.7 MiB/s wr, 751 op/s 2024-09-10T22:55:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:26 smithi121 ceph-mon[35119]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.7 MiB/s wr, 751 op/s 2024-09-10T22:55:27.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:26 smithi117 ceph-mon[35081]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.7 MiB/s wr, 751 op/s 2024-09-10T22:55:27.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:55:27.801 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:28.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:28 smithi096 ceph-mon[30869]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 -- 172.21.15.96:0/3434865278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f8101c90 msgr2=0x7f57f8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/3434865278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f8101c90 0x7f57f8102070 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009900 tx=0x7f57e802a410 comp rx=0 tx=0).stop 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 -- 172.21.15.96:0/3434865278 shutdown_connections 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/3434865278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f81076a0 0x7f57f8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/3434865278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8108020 0x7f57f810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/3434865278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f8101c90 0x7f57f8102070 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 -- 172.21.15.96:0/3434865278 >> 172.21.15.96:0/3434865278 conn(0x7f57f80fd700 msgr2=0x7f57f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 -- 172.21.15.96:0/3434865278 shutdown_connections 2024-09-10T22:55:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.590+0000 7f57fd2f1640 1 -- 172.21.15.96:0/3434865278 wait complete. 2024-09-10T22:55:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.591+0000 7f57fd2f1640 1 Processor -- start 2024-09-10T22:55:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.591+0000 7f57fd2f1640 1 -- start start 2024-09-10T22:55:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 0x7f57f819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f8108020 0x7f57f81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8111d30 con 0x7f57f81076a0 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8111bb0 con 0x7f57f8108020 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57fd2f1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8111eb0 con 0x7f57f8101c90 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57098/0 (socket says 172.21.15.96:57098) 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 -- 172.21.15.96:0/4273572405 learned_addr learned my addr 172.21.15.96:0/4273572405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:28.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 0x7f57f819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f77fe640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f8108020 0x7f57f81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 -- 172.21.15.96:0/4273572405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 msgr2=0x7f57f819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 0x7f57f819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 -- 172.21.15.96:0/4273572405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f8108020 msgr2=0x7f57f81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f8108020 0x7f57f81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 -- 172.21.15.96:0/4273572405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f81a15a0 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 0x7f57f819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.592+0000 7f57f67fc640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9f0 tx=0x7f57ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.593+0000 7f57dffff640 1 -- 172.21.15.96:0/4273572405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00ccc0 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.593+0000 7f57dffff640 1 -- 172.21.15.96:0/4273572405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ec0108e0 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.593+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f81a1890 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.593+0000 7f57dffff640 1 -- 172.21.15.96:0/4273572405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec018880 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.593+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f81a7f50 con 0x7f57f81076a0 2024-09-10T22:55:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.596+0000 7f57ff6c6740 1 Processor -- start 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.597+0000 7f57ff6c6740 1 -- start start 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.597+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4dea4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.597+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.597+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f4de1ae50 con 0x559f4deb0d30 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f4de0a280 con 0x559f4dea4480 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f4de0e090 con 0x559f4dea4da0 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57dffff640 1 -- 172.21.15.96:0/4273572405 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f57ec010430 con 0x7f57f81076a0 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57104/0 (socket says 172.21.15.96:57104) 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f67fc640 1 -- 172.21.15.96:0/2128939779 learned_addr learned my addr 172.21.15.96:0/2128939779 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f77fe640 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4dea4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 -- 172.21.15.96:0/2128939779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 msgr2=0x559f4dea4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4dea4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 -- 172.21.15.96:0/2128939779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 msgr2=0x559f4deb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f6ffd640 1 -- 172.21.15.96:0/2128939779 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f4db68650 con 0x559f4dea4da0 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.598+0000 7f57f67fc640 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb07f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57e802aa00 tx=0x7f57e80043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e167bc733d94bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57dffff640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 0x7f57e00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57dd7fa640 1 -- 172.21.15.96:0/2128939779 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8035040 con 0x559f4dea4da0 2024-09-10T22:55:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57dd7fa640 1 -- 172.21.15.96:0/2128939779 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e8004770 con 0x559f4dea4da0 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57dd7fa640 1 -- 172.21.15.96:0/2128939779 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8003850 con 0x559f4dea4da0 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57f77fe640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 0x7f57e00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.599+0000 7f57dffff640 1 -- 172.21.15.96:0/4273572405 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f57ec0e5fb0 con 0x7f57f81076a0 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2128939779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 msgr2=0x559f4deb07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57f77fe640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 0x7f57e00b7820 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005fd0 tx=0x7f57f0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb07f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f57e802aa00 tx=0x7f57e80043f0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2128939779 shutdown_connections 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb07f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2128939779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4dea4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2128939779 >> 172.21.15.96:0/2128939779 conn(0x559f4dea22b0 msgr2=0x559f4dea3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2128939779 shutdown_connections 2024-09-10T22:55:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2128939779 wait complete. 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 -- start start 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.600+0000 7f57ff6c6740 1 Processor -- start 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4480 0x559f4deb0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36398/0 (socket says 172.21.15.96:36398) 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 -- 172.21.15.96:0/2911731383 learned_addr learned my addr 172.21.15.96:0/2911731383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4480 0x559f4deb0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f4dc6b700 con 0x559f4deb0d30 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f4dda5110 con 0x559f4dea4da0 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f4deb3950 con 0x559f4dea4480 2024-09-10T22:55:28.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57ff6c6740 10 client.0 _ll_register_callbacks cb 0x559f4ddbe990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 -- 172.21.15.96:0/2911731383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4480 msgr2=0x559f4deb0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4480 0x559f4deb0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 -- 172.21.15.96:0/2911731383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 msgr2=0x559f4deae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.601+0000 7f57f67fc640 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4ddd2c40 con 0x559f4dea4da0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57f67fc640 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0a8050 tx=0x7f57ec0e3ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b37786ea0abb5c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57daff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ac540 con 0x559f4dea4da0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f57ec0ab9d0 con 0x559f4dea4da0 2024-09-10T22:55:28.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e8004770 con 0x559f4dea4da0 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.602+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57ec0abee0 con 0x559f4dea4da0 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 10 client.26699 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f4debf260 con 0x559f4dea4da0 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e8004770 con 0x559f4dea4da0 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f57ec0a80c0 con 0x559f4dea4da0 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 10 client.26699 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 20 client.26699 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4dec0720 0x559f4dec2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 4 client.26699 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.603+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f4de1cda0 con 0x559f4dec0720 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.604+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4dec0720 0x559f4dec2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.604+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4dec0720 0x559f4dec2b50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245af6ee8ea81508 server_cookie=354eda2cc270d1fd in_seq=0 out_seq=0 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.604+0000 7f57daff5640 10 client.26699 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57daff5640 1 -- 172.21.15.96:0/2911731383 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f4de1cda0 con 0x559f4dec0720 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57daff5640 10 client.26699 handle_command_reply: tid=0 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 client.26699 shutdown 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 msgr2=0x559f4deada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0a8050 tx=0x7f57ec0e3ab0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 shutdown_connections 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4dec0720 0x559f4dec2b50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4da0 0x559f4deada70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57daff5640 0 client.26699 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4deb0d30 0x559f4deae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/2911731383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4480 0x559f4deb0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 >> 172.21.15.96:0/2911731383 conn(0x559f4dea22b0 msgr2=0x559f4dea7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 shutdown_connections 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 1 -- 172.21.15.96:0/2911731383 wait complete. 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.605+0000 7f57ff6c6740 20 client.26699 trim_cache size 0 max 16384 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.627+0000 7f57ff6c6740 1 Processor -- start 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 -- start start 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4debcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4deb0d30 0x559f4deb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57130/0 (socket says 172.21.15.96:57130) 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 -- 172.21.15.96:0/3325343009 learned_addr learned my addr 172.21.15.96:0/3325343009 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b0560 con 0x559f4dea4da0 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b20a0 con 0x559f4dea4480 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b0c30 con 0x559f4deb0d30 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f77fe640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4deb0d30 0x559f4deb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4debcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 -- 172.21.15.96:0/3325343009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4deb0d30 msgr2=0x559f4deb2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4deb0d30 0x559f4deb2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 -- 172.21.15.96:0/3325343009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 msgr2=0x559f4debcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.628+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4debcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57f67fc640 1 -- 172.21.15.96:0/3325343009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f4dea4bd0 con 0x559f4dea4da0 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4debcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ac470 tx=0x7f57ec010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc1e42347cade62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57dcff9640 1 -- 172.21.15.96:0/3325343009 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0d3040 con 0x559f4dea4da0 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57dcff9640 1 -- 172.21.15.96:0/3325343009 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ec0ac280 con 0x559f4dea4da0 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.629+0000 7f57dcff9640 1 -- 172.21.15.96:0/3325343009 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ac0c0 con 0x559f4dea4da0 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 msgr2=0x559f4dea12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ac470 tx=0x7f57ec010d00 comp rx=0 tx=0).stop 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 shutdown_connections 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4deb0d30 0x559f4deb2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4da0 0x559f4dea12c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3325343009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4dea4480 0x559f4debcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 >> 172.21.15.96:0/3325343009 conn(0x559f4dea22b0 msgr2=0x559f4dea0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 shutdown_connections 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3325343009 wait complete. 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 -- start start 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.630+0000 7f57ff6c6740 1 Processor -- start 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4deb0d30 0x559f4deb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b19d0 con 0x559f4dea4480 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81b1300 con 0x559f4deb0d30 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57f81ae540 con 0x559f4dea4da0 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 10 client.0 _ll_register_callbacks cb 0x7f57ec009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57136/0 (socket says 172.21.15.96:57136) 2024-09-10T22:55:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 -- 172.21.15.96:0/3033938566 learned_addr learned my addr 172.21.15.96:0/3033938566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f77fe640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4deb0d30 0x559f4deb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 -- 172.21.15.96:0/3033938566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 msgr2=0x559f4deb9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 -- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4deb0d30 msgr2=0x559f4deb3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4deb0d30 0x559f4deb3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 -- 172.21.15.96:0/3033938566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f4deb4410 con 0x559f4dea4480 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57f6ffd640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005e00 tx=0x7f57e8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a70fb3c9546164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57daff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8035050 con 0x559f4dea4480 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ec0ac280 con 0x559f4dea4480 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.631+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f57e80048e0 con 0x559f4dea4480 2024-09-10T22:55:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57e8005610 con 0x559f4dea4480 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 10 client.17121 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f4df2e190 con 0x559f4dea4480 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ec0ac280 con 0x559f4dea4480 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f57e8055020 con 0x559f4dea4480 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 10 client.17121 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 20 client.17121 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4df2e4d0 0x559f4df308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 4 client.17121 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f4de18850 con 0x559f4df2e4d0 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.632+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4df2e4d0 0x559f4df308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57f67fc640 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4df2e4d0 0x559f4df308c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e9fc5465c5fe5c server_cookie=49977b56c9e83c96 in_seq=0 out_seq=0 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57daff5640 10 client.17121 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57daff5640 1 -- 172.21.15.96:0/3033938566 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x559f4de18850 con 0x559f4df2e4d0 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:55:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:55:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:55:28.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57daff5640 10 client.17121 handle_command_reply: tid=0 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 client.17121 shutdown 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 msgr2=0x559f4deb8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005e00 tx=0x7f57e8009680 comp rx=0 tx=0).stop 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 shutdown_connections 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f4df2e4d0 0x559f4df308c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f4deb0d30 0x559f4deb3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f4dea4da0 0x559f4deb9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57daff5640 0 client.17121 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 --2- 172.21.15.96:0/3033938566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f4dea4480 0x559f4deb8e60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 >> 172.21.15.96:0/3033938566 conn(0x559f4dea22b0 msgr2=0x559f4dea13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 shutdown_connections 2024-09-10T22:55:28.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 1 -- 172.21.15.96:0/3033938566 wait complete. 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.633+0000 7f57ff6c6740 20 client.17121 trim_cache size 0 max 16384 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 msgr2=0x7f57e00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 0x7f57e00b7820 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005fd0 tx=0x7f57f0009500 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 msgr2=0x7f57f819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9f0 tx=0x7f57ec00edf0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 shutdown_connections 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57f8108020 0x7f57f81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f57e00b5360 0x7f57e00b7820 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57f81076a0 0x7f57f819ba30 secure :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00e9f0 tx=0x7f57ec00edf0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 --2- 172.21.15.96:0/4273572405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f8101c90 0x7f57f819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 >> 172.21.15.96:0/4273572405 conn(0x7f57f80fd700 msgr2=0x7f57f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 shutdown_connections 2024-09-10T22:55:28.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:28.635+0000 7f57fd2f1640 1 -- 172.21.15.96:0/4273572405 wait complete. 2024-09-10T22:55:29.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:28 smithi121 ceph-mon[35119]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-10T22:55:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:28 smithi117 ceph-mon[35081]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-10T22:55:29.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:55:29.735 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:55:30.098 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:30.806 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:30 smithi096 ceph-mon[30869]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 757 op/s 2024-09-10T22:55:30.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 -- 172.21.15.96:0/1246939848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c109140 msgr2=0x7f158c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 --2- 172.21.15.96:0/1246939848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c109140 0x7f158c10f940 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1584009ca0 tx=0x7f158402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 -- 172.21.15.96:0/1246939848 shutdown_connections 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 --2- 172.21.15.96:0/1246939848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c1087c0 0x7f158c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 --2- 172.21.15.96:0/1246939848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c109140 0x7f158c10f940 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 --2- 172.21.15.96:0/1246939848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c102db0 0x7f158c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 -- 172.21.15.96:0/1246939848 >> 172.21.15.96:0/1246939848 conn(0x7f158c0fe860 msgr2=0x7f158c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 -- 172.21.15.96:0/1246939848 shutdown_connections 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.904+0000 7f1592c3f640 1 -- 172.21.15.96:0/1246939848 wait complete. 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 Processor -- start 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 -- start start 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c1087c0 0x7f158c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c109140 0x7f158c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112e00 con 0x7f158c102db0 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112c80 con 0x7f158c109140 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f1592c3f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112f80 con 0x7f158c1087c0 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57156/0 (socket says 172.21.15.96:57156) 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 -- 172.21.15.96:0/2350753964 learned_addr learned my addr 172.21.15.96:0/2350753964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15911dc640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c109140 0x7f158c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f158bfff640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c1087c0 0x7f158c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 -- 172.21.15.96:0/2350753964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c1087c0 msgr2=0x7f158c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c1087c0 0x7f158c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 -- 172.21.15.96:0/2350753964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c109140 msgr2=0x7f158c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c109140 0x7f158c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.905+0000 7f15909db640 1 -- 172.21.15.96:0/2350753964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c19f2e0 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f15909db640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f158000e9f0 tx=0x7f158000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158000cd80 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15800054d0 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580010690 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c19f5d0 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.906+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a5c90 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.909+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c103190 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.910+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1580004e90 con 0x7f158c102db0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.911+0000 7f1589ffb640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 0x7f157c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.911+0000 7f158bfff640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 0x7f157c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:30.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.911+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f15800e5f60 con 0x7f158c102db0 2024-09-10T22:55:30.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.911+0000 7f158bfff640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 0x7f157c0b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f158c108280 tx=0x7f157800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:30.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:30.915+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f15800acea0 con 0x7f158c102db0 2024-09-10T22:55:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 475.0 s, 1879.4 tps, lat 53.302 ms stddev 57.657 2024-09-10T22:55:31.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:30 smithi121 ceph-mon[35119]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 757 op/s 2024-09-10T22:55:31.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:30 smithi117 ceph-mon[35081]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 757 op/s 2024-09-10T22:55:31.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.241+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158c19a360 con 0x7f158c102db0 2024-09-10T22:55:31.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.243+0000 7f1589ffb640 1 -- 172.21.15.96:0/2350753964 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f15800ac7a0 con 0x7f158c102db0 2024-09-10T22:55:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:55:31.245 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:55:31.245 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:55:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 msgr2=0x7f157c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 0x7f157c0b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f158c108280 tx=0x7f157800b410 comp rx=0 tx=0).stop 2024-09-10T22:55:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 msgr2=0x7f158c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f158000e9f0 tx=0x7f158000edf0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 shutdown_connections 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f158c109140 0x7f158c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f157c0b52e0 0x7f157c0b77a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f158c1087c0 0x7f158c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 --2- 172.21.15.96:0/2350753964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f158c102db0 0x7f158c199230 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.246+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 >> 172.21.15.96:0/2350753964 conn(0x7f158c0fe860 msgr2=0x7f158c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.247+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 shutdown_connections 2024-09-10T22:55:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:31.247+0000 7f1592c3f640 1 -- 172.21.15.96:0/2350753964 wait complete. 2024-09-10T22:55:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.275+0000 7f138bfff640 1 -- 172.21.15.96:0/106944875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390035020 con 0x7f139406bc30 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 -- 172.21.15.96:0/106944875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f139406bc30 msgr2=0x7f13941055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 --2- 172.21.15.96:0/106944875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f139406bc30 0x7f13941055a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1390009900 tx=0x7f139002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 -- 172.21.15.96:0/106944875 shutdown_connections 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 --2- 172.21.15.96:0/106944875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f1394107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 --2- 172.21.15.96:0/106944875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1394108690 0x7f139410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 --2- 172.21.15.96:0/106944875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f139406bc30 0x7f13941055a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 -- 172.21.15.96:0/106944875 >> 172.21.15.96:0/106944875 conn(0x7f13940fd130 msgr2=0x7f13940ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 -- 172.21.15.96:0/106944875 shutdown_connections 2024-09-10T22:55:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.276+0000 7f139b9da640 1 -- 172.21.15.96:0/106944875 wait complete. 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 Processor -- start 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 -- start start 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f139406bc30 0x7f139419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f139419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 0x7f13941a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1394114c70 con 0x7f1394108690 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1394114af0 con 0x7f1394105ae0 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f139b9da640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1394114df0 con 0x7f139406bc30 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f1399f77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 0x7f13941a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f1399776640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f139406bc30 0x7f139419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f1399f77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 0x7f13941a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57170/0 (socket says 172.21.15.96:57170) 2024-09-10T22:55:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f1399f77640 1 -- 172.21.15.96:0/328582285 learned_addr learned my addr 172.21.15.96:0/328582285 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.277+0000 7f1398f75640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f139419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 -- 172.21.15.96:0/328582285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f139406bc30 msgr2=0x7f139419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f139406bc30 0x7f139419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 -- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 msgr2=0x7f13941a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 0x7f13941a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 -- 172.21.15.96:0/328582285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13941a0f30 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f1398f75640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f139419fa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1384009760 tx=0x7f1384009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f138a7fc640 1 -- 172.21.15.96:0/328582285 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138401c070 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13941a1220 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139406cd50 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f138a7fc640 1 -- 172.21.15.96:0/328582285 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138400cc60 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.278+0000 7f138a7fc640 1 -- 172.21.15.96:0/328582285 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840165a0 con 0x7f1394105ae0 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.281+0000 7f139e885740 1 Processor -- start 2024-09-10T22:55:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f138a7fc640 1 -- 172.21.15.96:0/328582285 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1384016740 con 0x7f1394105ae0 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 -- start start 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 0x56366b5adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ae190 0x56366b5b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56366b51d6f0 con 0x56366b5ae190 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56366b50cb20 con 0x56366b5b0bd0 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f139e885740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56366b510930 con 0x56366b5ab860 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1399f77640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 0x56366b5adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1399f77640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 0x56366b5adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54282/0 (socket says 172.21.15.96:54282) 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36474/0 (socket says 172.21.15.96:36474) 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1399f77640 1 -- 172.21.15.96:0/2671936814 learned_addr learned my addr 172.21.15.96:0/2671936814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1399776640 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ae190 0x56366b5b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 -- 172.21.15.96:0/2671936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 msgr2=0x56366b5adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 0x56366b5adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 -- 172.21.15.96:0/2671936814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ae190 msgr2=0x56366b5b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ae190 0x56366b5b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.282+0000 7f1398f75640 1 -- 172.21.15.96:0/2671936814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366b4d2530 con 0x56366b5b0bd0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f1398f75640 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13840e3470 tx=0x7f13840ebe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a23a8264d92d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f138a7fc640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 0x7f13800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f137f7fe640 1 -- 172.21.15.96:0/2671936814 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840f4070 con 0x56366b5b0bd0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f137f7fe640 1 -- 172.21.15.96:0/2671936814 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13840abd20 con 0x56366b5b0bd0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f137f7fe640 1 -- 172.21.15.96:0/2671936814 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840b0ea0 con 0x56366b5b0bd0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f1399f77640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 0x7f13800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.283+0000 7f138a7fc640 1 -- 172.21.15.96:0/328582285 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13840e5900 con 0x7f1394105ae0 2024-09-10T22:55:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- 172.21.15.96:0/2671936814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 msgr2=0x56366b5b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f1399f77640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 0x7f13800b7820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f138c0099a0 tx=0x7f138c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13840e3470 tx=0x7f13840ebe60 comp rx=0 tx=0).stop 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- 172.21.15.96:0/2671936814 shutdown_connections 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b3090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ae190 0x56366b5b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 --2- 172.21.15.96:0/2671936814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ab860 0x56366b5adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- 172.21.15.96:0/2671936814 >> 172.21.15.96:0/2671936814 conn(0x56366b5a69b0 msgr2=0x56366b5a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- 172.21.15.96:0/2671936814 shutdown_connections 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- 172.21.15.96:0/2671936814 wait complete. 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 -- start start 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.284+0000 7f139e885740 1 Processor -- start 2024-09-10T22:55:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56366b444ab0 con 0x56366b5ab860 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56366b4b2730 con 0x56366b5b0bd0 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56366b5b3bb0 con 0x56366b5ae190 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f139e885740 10 client.0 _ll_register_callbacks cb 0x56366b5066b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1398f75640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57180/0 (socket says 172.21.15.96:57180) 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 -- 172.21.15.96:0/4113681078 learned_addr learned my addr 172.21.15.96:0/4113681078 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399f77640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 -- 172.21.15.96:0/4113681078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 msgr2=0x56366b5b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 -- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 msgr2=0x56366b5b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1399776640 1 -- 172.21.15.96:0/4113681078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366b26e790 con 0x56366b5ab860 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.285+0000 7f1398f75640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.286+0000 7f1399776640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1390009900 tx=0x7f1390004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c3cf35c5a607e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.286+0000 7f137cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.286+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390035070 con 0x56366b5ab860 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.286+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13900318d0 con 0x56366b5ab860 2024-09-10T22:55:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.286+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13840abd20 con 0x56366b5ab860 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1390031ad0 con 0x56366b5ab860 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 10 client.17139 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56366b2e7620 con 0x56366b5ab860 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13840abd20 con 0x56366b5ab860 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1390045430 con 0x56366b5ab860 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 10 client.17139 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 20 client.17139 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b5c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 4 client.17139 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.287+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366b51f640 con 0x56366b5c0850 2024-09-10T22:55:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.288+0000 7f1398f75640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b5c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.288+0000 7f137cff9640 10 client.17139 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.288+0000 7f1398f75640 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b5c2cc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3009e5c2736fa64c server_cookie=3c25ba7831545154 in_seq=0 out_seq=0 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f137cff9640 1 -- 172.21.15.96:0/4113681078 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56366b51f640 con 0x56366b5c0850 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f137cff9640 10 client.17139 handle_command_reply: tid=0 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 client.17139 shutdown 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 msgr2=0x56366b5b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1390009900 tx=0x7f1390004250 comp rx=0 tx=0).stop 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 shutdown_connections 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b5c2cc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f137cff9640 0 client.17139 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 --2- 172.21.15.96:0/4113681078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5b4ed0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 >> 172.21.15.96:0/4113681078 conn(0x56366b5a69b0 msgr2=0x56366b5a5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 shutdown_connections 2024-09-10T22:55:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 1 -- 172.21.15.96:0/4113681078 wait complete. 2024-09-10T22:55:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.289+0000 7f139e885740 20 client.17139 trim_cache size 0 max 16384 2024-09-10T22:55:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.307+0000 7f139e885740 1 Processor -- start 2024-09-10T22:55:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.307+0000 7f139e885740 1 -- start start 2024-09-10T22:55:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5a5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f137800e3a0 con 0x56366b5ab860 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f137800fee0 con 0x56366b5b0bd0 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f139e885740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f137800ea70 con 0x56366b5ae190 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f1398f75640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f1399f77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f1399f77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36494/0 (socket says 172.21.15.96:36494) 2024-09-10T22:55:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.308+0000 7f1399f77640 1 -- 172.21.15.96:0/654894551 learned_addr learned my addr 172.21.15.96:0/654894551 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 -- 172.21.15.96:0/654894551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 msgr2=0x56366b5a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 -- 172.21.15.96:0/654894551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 msgr2=0x56366b5a53d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:55:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 -- 172.21.15.96:0/654894551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366b5b08f0 con 0x56366b5ae190 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f1398f75640 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5a5910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f13840e3470 tx=0x7f13840abb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9863f348b26515f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f137effd640 1 -- 172.21.15.96:0/654894551 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840f4070 con 0x56366b5ae190 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f137effd640 1 -- 172.21.15.96:0/654894551 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13840abd20 con 0x56366b5ae190 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.309+0000 7f137effd640 1 -- 172.21.15.96:0/654894551 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138406d660 con 0x56366b5ae190 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 -- 172.21.15.96:0/654894551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 msgr2=0x56366b5a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5a5910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f13840e3470 tx=0x7f13840abb60 comp rx=0 tx=0).stop 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 -- 172.21.15.96:0/654894551 shutdown_connections 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5a5910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 --2- 172.21.15.96:0/654894551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 -- 172.21.15.96:0/654894551 >> 172.21.15.96:0/654894551 conn(0x56366b5a69b0 msgr2=0x56366b5a7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 -- 172.21.15.96:0/654894551 shutdown_connections 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.310+0000 7f139e885740 1 -- 172.21.15.96:0/654894551 wait complete. 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 -- start start 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 Processor -- start 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f137800f810 con 0x56366b5ab860 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f137800f140 con 0x56366b5b0bd0 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f137800c380 con 0x56366b5ae190 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f1398f75640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f139e885740 10 client.0 _ll_register_callbacks cb 0x7f138c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f1398f75640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54306/0 (socket says 172.21.15.96:54306) 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f1399776640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.311+0000 7f1398f75640 1 -- 172.21.15.96:0/2781554896 learned_addr learned my addr 172.21.15.96:0/2781554896 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1399f77640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 -- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 msgr2=0x56366b5bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 -- 172.21.15.96:0/2781554896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 msgr2=0x56366b5bca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5bca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 -- 172.21.15.96:0/2781554896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366b5adfc0 con 0x56366b5ae190 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f1398f75640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1384017d80 tx=0x7f13840a88e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6e3f23b62e0d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f137cff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13840f4050 con 0x56366b5ae190 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f13840cf110 con 0x56366b5ae190 2024-09-10T22:55:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.312+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13840abd20 con 0x56366b5ae190 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.313+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f138406d830 con 0x56366b5ae190 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.313+0000 7f139e885740 10 client.26785 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.313+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56366b62d780 con 0x56366b5ae190 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13840abd20 con 0x56366b5ae190 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f13840e30b0 con 0x56366b5ae190 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 10 client.26785 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 10 client.26785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 20 client.26785 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b62fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 4 client.26785 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56366b51b0f0 con 0x56366b5c0850 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f1399776640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b62fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.314+0000 7f137cff9640 10 client.26785 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.315+0000 7f1399776640 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b62fa20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6309895402095c1d server_cookie=df91cdfb3c90b304 in_seq=0 out_seq=0 2024-09-10T22:55:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f137cff9640 1 -- 172.21.15.96:0/2781554896 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56366b51b0f0 con 0x56366b5c0850 2024-09-10T22:55:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f137cff9640 10 client.26785 handle_command_reply: tid=0 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 client.26785 shutdown 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 msgr2=0x56366b5bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1384017d80 tx=0x7f13840a88e0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 shutdown_connections 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56366b5c0850 0x56366b62fa20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f137cff9640 0 client.26785 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56366b5b0bd0 0x56366b5bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56366b5ae190 0x56366b5bb700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 --2- 172.21.15.96:0/2781554896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56366b5ab860 0x56366b5bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.337+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 >> 172.21.15.96:0/2781554896 conn(0x56366b5a69b0 msgr2=0x56366b5a53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 shutdown_connections 2024-09-10T22:55:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139e885740 1 -- 172.21.15.96:0/2781554896 wait complete. 2024-09-10T22:55:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139e885740 20 client.26785 trim_cache size 0 max 16384 2024-09-10T22:55:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:55:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:55:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:55:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 msgr2=0x7f13800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 0x7f13800b7820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f138c0099a0 tx=0x7f138c009500 comp rx=0 tx=0).stop 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 msgr2=0x7f139419fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f139419fa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1384009760 tx=0x7f1384009c20 comp rx=0 tx=0).stop 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 shutdown_connections 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1394105ae0 0x7f139419fa40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f13800b5360 0x7f13800b7820 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1394108690 0x7f13941a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 --2- 172.21.15.96:0/328582285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f139406bc30 0x7f139419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.338+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 >> 172.21.15.96:0/328582285 conn(0x7f13940fd130 msgr2=0x7f1394109360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.339+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 shutdown_connections 2024-09-10T22:55:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:31.339+0000 7f139b9da640 1 -- 172.21.15.96:0/328582285 wait complete. 2024-09-10T22:55:32.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:31 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2350753964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:32.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:31 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2350753964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:32.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:31 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2350753964' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:55:32.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:55:32.413 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:32.796 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:32 smithi096 ceph-mon[30869]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T22:55:33.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:32 smithi121 ceph-mon[35119]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T22:55:33.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:32 smithi117 ceph-mon[35081]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.9 MiB/s wr, 627 op/s 2024-09-10T22:55:33.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 -- 172.21.15.96:0/132696780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 msgr2=0x7fdbdc10cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/132696780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc10cb10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009900 tx=0x7fdbcc02a410 comp rx=0 tx=0).stop 2024-09-10T22:55:33.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 -- 172.21.15.96:0/132696780 shutdown_connections 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/132696780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/132696780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc10cb10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/132696780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.214+0000 7fdbe32a8640 1 -- 172.21.15.96:0/132696780 >> 172.21.15.96:0/132696780 conn(0x7fdbdc0fd700 msgr2=0x7fdbdc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.215+0000 7fdbe32a8640 1 -- 172.21.15.96:0/132696780 shutdown_connections 2024-09-10T22:55:33.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.215+0000 7fdbe32a8640 1 -- 172.21.15.96:0/132696780 wait complete. 2024-09-10T22:55:33.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.215+0000 7fdbe32a8640 1 Processor -- start 2024-09-10T22:55:33.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.215+0000 7fdbe32a8640 1 -- start start 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.215+0000 7fdbe32a8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe32a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe32a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe32a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112d60 con 0x7fdbdc104530 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe32a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112be0 con 0x7fdbdc10d0e0 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe32a8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc112ee0 con 0x7fdbdc103b30 2024-09-10T22:55:33.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe0843640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe0843640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57206/0 (socket says 172.21.15.96:57206) 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe0843640 1 -- 172.21.15.96:0/2360482356 learned_addr learned my addr 172.21.15.96:0/2360482356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1845640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 -- 172.21.15.96:0/2360482356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 msgr2=0x7fdbdc19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 -- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 msgr2=0x7fdbdc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.216+0000 7fdbe1044640 1 -- 172.21.15.96:0/2360482356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19f3a0 con 0x7fdbdc103b30 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.217+0000 7fdbe1845640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.217+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc1992f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009fd0 tx=0x7fdbd800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.217+0000 7fdbd27fc640 1 -- 172.21.15.96:0/2360482356 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8004b20 con 0x7fdbdc103b30 2024-09-10T22:55:33.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.217+0000 7fdbd27fc640 1 -- 172.21.15.96:0/2360482356 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd8004cc0 con 0x7fdbdc103b30 2024-09-10T22:55:33.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.217+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19f690 con 0x7fdbdc103b30 2024-09-10T22:55:33.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.218+0000 7fdbd27fc640 1 -- 172.21.15.96:0/2360482356 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8010720 con 0x7fdbdc103b30 2024-09-10T22:55:33.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.218+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1a5d50 con 0x7fdbdc103b30 2024-09-10T22:55:33.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.220+0000 7fdbd27fc640 1 -- 172.21.15.96:0/2360482356 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdbd8005020 con 0x7fdbdc103b30 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.221+0000 7fdbd27fc640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 0x7fdbc80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.221+0000 7fdbe0843640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 0x7fdbc80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.221+0000 7fdbd27fc640 1 -- 172.21.15.96:0/2360482356 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdbd80e6290 con 0x7fdbdc103b30 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.221+0000 7fdbe0843640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 0x7fdbc80b78f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc031440 tx=0x7fdbcc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.222+0000 7fdbe567d740 1 Processor -- start 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.222+0000 7fdbe567d740 1 -- start start 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe239840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe23c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564afe1a3ea0 con 0x564afe22d4d0 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564afe1932d0 con 0x564afe239d80 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe567d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564afe1970e0 con 0x564afe22ddf0 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57216/0 (socket says 172.21.15.96:57216) 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 -- 172.21.15.96:0/109063249 learned_addr learned my addr 172.21.15.96:0/109063249 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1845640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe23c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe0843640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe239840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 -- 172.21.15.96:0/109063249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 msgr2=0x564afe239840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe239840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 -- 172.21.15.96:0/109063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 msgr2=0x564afe23c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe23c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1044640 1 -- 172.21.15.96:0/109063249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afdef1650 con 0x564afe22d4d0 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.223+0000 7fdbe1845640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe23c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.224+0000 7fdbe1044640 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80a90a0 tx=0x7fdbd80ac4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee3198c63ba1283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.225+0000 7fdbc77fe640 1 -- 172.21.15.96:0/109063249 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80ad1d0 con 0x564afe22d4d0 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.225+0000 7fdbc77fe640 1 -- 172.21.15.96:0/109063249 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80acad0 con 0x564afe22d4d0 2024-09-10T22:55:33.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.225+0000 7fdbc77fe640 1 -- 172.21.15.96:0/109063249 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80b11f0 con 0x564afe22d4d0 2024-09-10T22:55:33.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- 172.21.15.96:0/109063249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 msgr2=0x564afe22d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80a90a0 tx=0x7fdbd80ac4e0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- 172.21.15.96:0/109063249 shutdown_connections 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe23c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe239840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 --2- 172.21.15.96:0/109063249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe22d8b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- 172.21.15.96:0/109063249 >> 172.21.15.96:0/109063249 conn(0x564afe22b300 msgr2=0x564afe22c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- 172.21.15.96:0/109063249 shutdown_connections 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- 172.21.15.96:0/109063249 wait complete. 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 -- start start 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.226+0000 7fdbe567d740 1 Processor -- start 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe239630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe236a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe237040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564afdff4700 con 0x564afe22d4d0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564afe12e160 con 0x564afe239d80 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564afe23ca50 con 0x564afe22ddf0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe567d740 10 client.0 _ll_register_callbacks cb 0x564afe1479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe239630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe239630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57226/0 (socket says 172.21.15.96:57226) 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe236a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe1044640 1 -- 172.21.15.96:0/3960451441 learned_addr learned my addr 172.21.15.96:0/3960451441 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe1845640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe237040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 -- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 msgr2=0x564afe237040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe237040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 -- 172.21.15.96:0/3960451441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 msgr2=0x564afe239630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe239630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.227+0000 7fdbe0843640 1 -- 172.21.15.96:0/3960451441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afe15bc90 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe1845640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe237040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe0843640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe236a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc004770 tx=0x7fdbcc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e2b882f5f6231c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc04d070 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd80acad0 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdbcc0043d0 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc059050 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe567d740 10 client.26797 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564afe250c20 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80acad0 con 0x564afe22ddf0 2024-09-10T22:55:33.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdbcc0495e0 con 0x564afe22ddf0 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.228+0000 7fdbe567d740 10 client.26797 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe567d740 10 client.26797 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe567d740 20 client.26797 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe252080 0x564afe254490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe567d740 4 client.26797 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564afe1a5df0 con 0x564afe252080 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe1044640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe252080 0x564afe254490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbc4ff9640 10 client.26797 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.229+0000 7fdbe1044640 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe252080 0x564afe254490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de5236ced5d3395f server_cookie=9e5d7c399d9a04c1 in_seq=0 out_seq=0 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.230+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/3960451441 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564afe1a5df0 con 0x564afe252080 2024-09-10T22:55:33.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.230+0000 7fdbc4ff9640 10 client.26797 handle_command_reply: tid=0 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.230+0000 7fdbe567d740 1 client.26797 shutdown 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 msgr2=0x564afe236a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe236a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc004770 tx=0x7fdbcc009260 comp rx=0 tx=0).stop 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 shutdown_connections 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe252080 0x564afe254490 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe239d80 0x564afe237040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe236a80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 --2- 172.21.15.96:0/3960451441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe22d4d0 0x564afe239630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 >> 172.21.15.96:0/3960451441 conn(0x564afe22b300 msgr2=0x564afe230e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbc4ff9640 0 client.26797 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:55:33.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 shutdown_connections 2024-09-10T22:55:33.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.232+0000 7fdbe567d740 1 -- 172.21.15.96:0/3960451441 wait complete. 2024-09-10T22:55:33.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.233+0000 7fdbe567d740 20 client.26797 trim_cache size 0 max 16384 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 Processor -- start 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 -- start start 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22d4d0 0x564afe245d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe233cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1ac060 con 0x564afe239d80 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1adba0 con 0x564afe22d4d0 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.248+0000 7fdbe567d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1ac730 con 0x564afe22ddf0 2024-09-10T22:55:33.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe1845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe233cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54362/0 (socket says 172.21.15.96:54362) 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 -- 172.21.15.96:0/149778511 learned_addr learned my addr 172.21.15.96:0/149778511 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe1845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe233cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57230/0 (socket says 172.21.15.96:57230) 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe1044640 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22d4d0 0x564afe245d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 -- 172.21.15.96:0/149778511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22d4d0 msgr2=0x564afe245d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22d4d0 0x564afe245d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 -- 172.21.15.96:0/149778511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 msgr2=0x564afe233cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe233cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 -- 172.21.15.96:0/149778511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afe237910 con 0x564afe22ddf0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbe0843640 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc049a30 tx=0x7fdbcc0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ebe8fc1a6af4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/149778511 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc04d070 con 0x564afe22ddf0 2024-09-10T22:55:33.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/149778511 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbcc0035f0 con 0x564afe22ddf0 2024-09-10T22:55:33.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.249+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/149778511 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc035040 con 0x564afe22ddf0 2024-09-10T22:55:33.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- 172.21.15.96:0/149778511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 msgr2=0x564afe22ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc049a30 tx=0x7fdbcc0314a0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- 172.21.15.96:0/149778511 shutdown_connections 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe233cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22ddf0 0x564afe22ac10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 --2- 172.21.15.96:0/149778511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22d4d0 0x564afe245d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- 172.21.15.96:0/149778511 >> 172.21.15.96:0/149778511 conn(0x564afe22b300 msgr2=0x564afe231cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- 172.21.15.96:0/149778511 shutdown_connections 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- 172.21.15.96:0/149778511 wait complete. 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 -- start start 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.250+0000 7fdbe567d740 1 Processor -- start 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22ddf0 0x564afe2341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe2346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1ad4d0 con 0x564afe239d80 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1ace00 con 0x564afe22ddf0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1aa080 con 0x564afe22d4d0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe567d740 10 client.0 _ll_register_callbacks cb 0x7fdbcc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe1845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe2346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe1044640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54366/0 (socket says 172.21.15.96:54366) 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe1044640 1 -- 172.21.15.96:0/2709264146 learned_addr learned my addr 172.21.15.96:0/2709264146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.251+0000 7fdbe0843640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22ddf0 0x564afe2341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.252+0000 7fdbe1044640 1 -- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22ddf0 msgr2=0x564afe2341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.252+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22ddf0 0x564afe2341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.252+0000 7fdbe1044640 1 -- 172.21.15.96:0/2709264146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 msgr2=0x564afe2346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.252+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe2346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.252+0000 7fdbe1044640 1 -- 172.21.15.96:0/2709264146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afe2bdfe0 con 0x564afe22d4d0 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.253+0000 7fdbe1044640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80aca60 tx=0x7fdbd80a8f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc914171967db3b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:33.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.253+0000 7fdbc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.253+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80acad0 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.253+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdbd80acd30 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.253+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc0035f0 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdbd80ce240 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 10 client.26803 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564afe2bf2e0 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80acf30 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdbd80c0040 con 0x564afe22d4d0 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 10 client.26803 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:33.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 10 client.26803 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 20 client.26803 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe2bf570 0x564afe2c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 4 client.26803 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdbcc0035f0 con 0x564afe2bf570 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.254+0000 7fdbe0843640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe2bf570 0x564afe2c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbc4ff9640 10 client.26803 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe0843640 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe2bf570 0x564afe2c1960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e80ff1353d11714 server_cookie=24577ed1473043fd in_seq=0 out_seq=0 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbc4ff9640 1 -- 172.21.15.96:0/2709264146 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdbcc0035f0 con 0x564afe2bf570 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbc4ff9640 10 client.26803 handle_command_reply: tid=0 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 client.26803 shutdown 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 msgr2=0x564afe233c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80aca60 tx=0x7fdbd80a8f70 comp rx=0 tx=0).stop 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 shutdown_connections 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x564afe2bf570 0x564afe2c1960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbc4ff9640 0 client.26803 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564afe22ddf0 0x564afe2341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564afe239d80 0x564afe2346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 --2- 172.21.15.96:0/2709264146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564afe22d4d0 0x564afe233c70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.255+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 >> 172.21.15.96:0/2709264146 conn(0x564afe22b300 msgr2=0x564afe2380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 shutdown_connections 2024-09-10T22:55:33.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe567d740 1 -- 172.21.15.96:0/2709264146 wait complete. 2024-09-10T22:55:33.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe567d740 20 client.26803 trim_cache size 0 max 16384 2024-09-10T22:55:33.264 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:55:33.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 msgr2=0x7fdbc80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 0x7fdbc80b78f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc031440 tx=0x7fdbcc0058d0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 msgr2=0x7fdbdc1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc1992f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009fd0 tx=0x7fdbd800ef90 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 shutdown_connections 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbdc10d0e0 0x7fdbdc19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdbc80b5430 0x7fdbc80b78f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc104530 0x7fdbdc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 --2- 172.21.15.96:0/2360482356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbdc103b30 0x7fdbdc1992f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.256+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 >> 172.21.15.96:0/2360482356 conn(0x7fdbdc0fd700 msgr2=0x7fdbdc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.257+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 shutdown_connections 2024-09-10T22:55:33.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:33.257+0000 7fdbe32a8640 1 -- 172.21.15.96:0/2360482356 wait complete. 2024-09-10T22:55:34.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:55:34.616 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:34.877 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:34 smithi096 ceph-mon[30869]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-09-10T22:55:35.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:34 smithi121 ceph-mon[35119]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-09-10T22:55:35.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:34 smithi117 ceph-mon[35081]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 672 op/s 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/3984934741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 msgr2=0x7f5f78104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/3984934741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78104080 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009900 tx=0x7f5f6802a410 comp rx=0 tx=0).stop 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/3984934741 shutdown_connections 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/3984934741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f7810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/3984934741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/3984934741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78104080 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/3984934741 >> 172.21.15.96:0/3984934741 conn(0x7f5f780fd700 msgr2=0x7f5f780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/3984934741 shutdown_connections 2024-09-10T22:55:35.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.145+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/3984934741 wait complete. 2024-09-10T22:55:35.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 Processor -- start 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 -- start start 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78112d70 con 0x7f5f78103ca0 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78112bf0 con 0x7f5f781045c0 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7e8e6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78112ef0 con 0x7f5f7810d100 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7ce83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7ce83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54386/0 (socket says 172.21.15.96:54386) 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f781997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36572/0 (socket says 172.21.15.96:36572) 2024-09-10T22:55:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.146+0000 7f5f7ce83640 1 -- 172.21.15.96:0/1787184044 learned_addr learned my addr 172.21.15.96:0/1787184044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 -- 172.21.15.96:0/1787184044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 msgr2=0x7f5f781997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f781997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 -- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 msgr2=0x7f5f78199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 -- 172.21.15.96:0/1787184044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7819f330 con 0x7f5f7810d100 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7ce83640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000efd0 tx=0x7f5f7000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f757fa640 1 -- 172.21.15.96:0/1787184044 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70003b70 con 0x7f5f7810d100 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7819f620 con 0x7f5f7810d100 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f757fa640 1 -- 172.21.15.96:0/1787184044 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f70003d10 con 0x7f5f7810d100 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f757fa640 1 -- 172.21.15.96:0/1787184044 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700108a0 con 0x7f5f7810d100 2024-09-10T22:55:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.147+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a5ce0 con 0x7f5f7810d100 2024-09-10T22:55:35.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.151+0000 7f5f80cbb740 1 Processor -- start 2024-09-10T22:55:35.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f757fa640 1 -- 172.21.15.96:0/1787184044 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5f7001f020 con 0x7f5f7810d100 2024-09-10T22:55:35.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 -- start start 2024-09-10T22:55:35.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd71a10 0x55b6ffd81870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd81db0 0x55b6ffd841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffce7ea0 con 0x55b6ffd81db0 2024-09-10T22:55:35.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffcd72d0 con 0x55b6ffd71a10 2024-09-10T22:55:35.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.152+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffcdb0e0 con 0x55b6ffd710f0 2024-09-10T22:55:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54388/0 (socket says 172.21.15.96:54388) 2024-09-10T22:55:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 -- 172.21.15.96:0/2666790009 learned_addr learned my addr 172.21.15.96:0/2666790009 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd81db0 0x55b6ffd841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f77fff640 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd71a10 0x55b6ffd81870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 -- 172.21.15.96:0/2666790009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd71a10 msgr2=0x55b6ffd81870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f757fa640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 0x7f5f600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd71a10 0x55b6ffd81870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 -- 172.21.15.96:0/2666790009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd81db0 msgr2=0x55b6ffd841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd81db0 0x55b6ffd841a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f777fe640 1 -- 172.21.15.96:0/2666790009 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ffa35650 con 0x55b6ffd710f0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f7ce83640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 0x7f5f600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.153+0000 7f5f757fa640 1 -- 172.21.15.96:0/1787184044 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f700e5d80 con 0x7f5f7810d100 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.154+0000 7f5f777fe640 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0117c0 tx=0x7f5f6c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942025954488c133 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.154+0000 7f5f5e7fc640 1 -- 172.21.15.96:0/2666790009 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004050 con 0x55b6ffd710f0 2024-09-10T22:55:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.154+0000 7f5f7ce83640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 0x7f5f600b7820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5f700a97b0 tx=0x7f5f70009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.154+0000 7f5f5e7fc640 1 -- 172.21.15.96:0/2666790009 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c00fc70 con 0x55b6ffd710f0 2024-09-10T22:55:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.154+0000 7f5f5e7fc640 1 -- 172.21.15.96:0/2666790009 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003990 con 0x55b6ffd710f0 2024-09-10T22:55:35.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- 172.21.15.96:0/2666790009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 msgr2=0x55b6ffd714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0117c0 tx=0x7f5f6c005850 comp rx=0 tx=0).stop 2024-09-10T22:55:35.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- 172.21.15.96:0/2666790009 shutdown_connections 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd71a10 0x55b6ffd81870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd81db0 0x55b6ffd841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/2666790009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd714d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- 172.21.15.96:0/2666790009 >> 172.21.15.96:0/2666790009 conn(0x55b6ffd6ef20 msgr2=0x55b6ffd700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- 172.21.15.96:0/2666790009 shutdown_connections 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- 172.21.15.96:0/2666790009 wait complete. 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.155+0000 7f5f80cbb740 1 -- start start 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 Processor -- start 2024-09-10T22:55:35.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd710f0 0x55b6ffd81660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffb38700 con 0x55b6ffd71a10 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffc72160 con 0x55b6ffd710f0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffd84a60 con 0x55b6ffd81db0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f80cbb740 10 client.0 _ll_register_callbacks cb 0x55b6ffc8b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd710f0 0x55b6ffd81660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57260/0 (socket says 172.21.15.96:57260) 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.156+0000 7f5f77fff640 1 -- 172.21.15.96:0/1476831615 learned_addr learned my addr 172.21.15.96:0/1476831615 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 -- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 msgr2=0x55b6ffd7f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 -- 172.21.15.96:0/1476831615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd710f0 msgr2=0x55b6ffd81660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd710f0 0x55b6ffd81660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 -- 172.21.15.96:0/1476831615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ffc9fc90 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000c00 tx=0x7f5f68032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820f0e234c985011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f577fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68040070 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f680318b0 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.157+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c00fc70 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f68035040 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 10 client.17169 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6ffd90380 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c00fc70 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f680046c0 con 0x55b6ffd71a10 2024-09-10T22:55:35.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 10 client.17169 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 10 client.17169 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 20 client.17169 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffd91840 0x55b6ffd93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 4 client.17169 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ffce9df0 con 0x55b6ffd91840 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.158+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffd91840 0x55b6ffd93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.159+0000 7f5f577fe640 10 client.17169 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:55:35.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.159+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffd91840 0x55b6ffd93c30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5726a06ef5448d14 server_cookie=5ea61eadddb94ffb in_seq=0 out_seq=0 2024-09-10T22:55:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f577fe640 1 -- 172.21.15.96:0/1476831615 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ffce9df0 con 0x55b6ffd91840 2024-09-10T22:55:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f577fe640 10 client.17169 handle_command_reply: tid=0 2024-09-10T22:55:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 client.17169 shutdown 2024-09-10T22:55:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 msgr2=0x55b6ffd7eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000c00 tx=0x7f5f68032d50 comp rx=0 tx=0).stop 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 shutdown_connections 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffd91840 0x55b6ffd93c30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd7eb00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1476831615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd710f0 0x55b6ffd81660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 >> 172.21.15.96:0/1476831615 conn(0x55b6ffd6ef20 msgr2=0x55b6ffd74a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f577fe640 0 client.17169 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 shutdown_connections 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.160+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1476831615 wait complete. 2024-09-10T22:55:35.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.161+0000 7f5f80cbb740 20 client.17169 trim_cache size 0 max 16384 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.188+0000 7f5f80cbb740 1 Processor -- start 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.188+0000 7f5f80cbb740 1 -- start start 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd8dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ac040 con 0x55b6ffd71a10 2024-09-10T22:55:35.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781adb80 con 0x55b6ffd81db0 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ac710 con 0x55b6ffd710f0 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57264/0 (socket says 172.21.15.96:57264) 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 -- 172.21.15.96:0/4178587503 learned_addr learned my addr 172.21.15.96:0/4178587503 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 -- 172.21.15.96:0/4178587503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 msgr2=0x55b6ffd8dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd8dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 -- 172.21.15.96:0/4178587503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 msgr2=0x55b6ffd7d380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 -- 172.21.15.96:0/4178587503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ffd6f960 con 0x55b6ffd71a10 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.189+0000 7f5f77fff640 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5f68031440 tx=0x7f5f6802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1531192d4013e70b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.190+0000 7f5f5dffb640 1 -- 172.21.15.96:0/4178587503 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68040070 con 0x55b6ffd71a10 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.190+0000 7f5f5dffb640 1 -- 172.21.15.96:0/4178587503 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f68031b60 con 0x55b6ffd71a10 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.190+0000 7f5f5dffb640 1 -- 172.21.15.96:0/4178587503 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680325b0 con 0x55b6ffd71a10 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- 172.21.15.96:0/4178587503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 msgr2=0x55b6ffd6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5f68031440 tx=0x7f5f6802a410 comp rx=0 tx=0).stop 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- 172.21.15.96:0/4178587503 shutdown_connections 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd7d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd71a10 0x55b6ffd6e760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/4178587503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd710f0 0x55b6ffd8dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- 172.21.15.96:0/4178587503 >> 172.21.15.96:0/4178587503 conn(0x55b6ffd6ef20 msgr2=0x55b6ffd83b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- 172.21.15.96:0/4178587503 shutdown_connections 2024-09-10T22:55:35.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- 172.21.15.96:0/4178587503 wait complete. 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.191+0000 7f5f80cbb740 1 -- start start 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 Processor -- start 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 0x55b6ffd89d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ad4b0 con 0x55b6ffd710f0 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781acde0 con 0x55b6ffd81db0 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781aa060 con 0x55b6ffd71a10 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f80cbb740 10 client.0 _ll_register_callbacks cb 0x7f5f700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 0x55b6ffd89d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 0x55b6ffd89d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57266/0 (socket says 172.21.15.96:57266) 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f777fe640 1 -- 172.21.15.96:0/1029958491 learned_addr learned my addr 172.21.15.96:0/1029958491 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54420/0 (socket says 172.21.15.96:54420) 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.192+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 -- 172.21.15.96:0/1029958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 msgr2=0x55b6ffd86cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd86cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 -- 172.21.15.96:0/1029958491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 msgr2=0x55b6ffd89d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 0x55b6ffd89d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 -- 172.21.15.96:0/1029958491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ffd87230 con 0x55b6ffd71a10 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f77fff640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f6802a930 tx=0x7f5f68005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c184b7f1da81174c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:35.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f577fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68040030 con 0x55b6ffd71a10 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f68031b60 con 0x55b6ffd71a10 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f68004030 con 0x55b6ffd71a10 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f68009780 con 0x55b6ffd71a10 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f80cbb740 10 client.26818 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.193+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6ffdfeee0 con 0x55b6ffd71a10 2024-09-10T22:55:35.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f68031b60 con 0x55b6ffd71a10 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f680035f0 con 0x55b6ffd71a10 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 10 client.26818 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 10 client.26818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 20 client.26818 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffdff600 0x55b6ffe01a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 4 client.26818 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6ffce58a0 con 0x55b6ffdff600 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffdff600 0x55b6ffe01a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f577fe640 10 client.26818 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:55:35.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.194+0000 7f5f777fe640 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffdff600 0x55b6ffe01a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=230743e0d7d6279f server_cookie=cec2ab7d620a2f5a in_seq=0 out_seq=0 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f577fe640 1 -- 172.21.15.96:0/1029958491 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6ffce58a0 con 0x55b6ffdff600 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f577fe640 10 client.26818 handle_command_reply: tid=0 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 client.26818 shutdown 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 msgr2=0x55b6ffd8a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f6802a930 tx=0x7f5f68005b50 comp rx=0 tx=0).stop 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 shutdown_connections 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55b6ffdff600 0x55b6ffe01a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ffd81db0 0x55b6ffd86cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f577fe640 0 client.26818 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b6ffd71a10 0x55b6ffd8a260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 --2- 172.21.15.96:0/1029958491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6ffd710f0 0x55b6ffd89d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 >> 172.21.15.96:0/1029958491 conn(0x55b6ffd6ef20 msgr2=0x55b6ffd7d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 shutdown_connections 2024-09-10T22:55:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 1 -- 172.21.15.96:0/1029958491 wait complete. 2024-09-10T22:55:35.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.195+0000 7f5f80cbb740 20 client.26818 trim_cache size 0 max 16384 2024-09-10T22:55:35.202 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:55:35.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 msgr2=0x7f5f600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 0x7f5f600b7820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5f700a97b0 tx=0x7f5f70009840 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 msgr2=0x7f5f7819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000efd0 tx=0x7f5f7000c6a0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 shutdown_connections 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f781045c0 0x7f5f781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f600b5360 0x7f5f600b7820 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f7810d100 0x7f5f7819eb60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 --2- 172.21.15.96:0/1787184044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f78103ca0 0x7f5f78199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 >> 172.21.15.96:0/1787184044 conn(0x7f5f780fd700 msgr2=0x7f5f780fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 shutdown_connections 2024-09-10T22:55:35.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:35.196+0000 7f5f7e8e6640 1 -- 172.21.15.96:0/1787184044 wait complete. 2024-09-10T22:55:35.885 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:35 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 480.0 s, 1577.8 tps, lat 63.192 ms stddev 64.897 2024-09-10T22:55:35.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:55:36.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:35 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:35 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:55:36.368 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:36 smithi096 ceph-mon[30869]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-09-10T22:55:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:36.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3358350303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348103ce0 msgr2=0x7fc3481040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3358350303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348103ce0 0x7fc3481040c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc338009900 tx=0x7fc33802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3358350303 shutdown_connections 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3358350303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc348104600 0x7fc34810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3358350303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc34810d140 0x7fc34810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3358350303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348103ce0 0x7fc3481040c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3358350303 >> 172.21.15.96:0/3358350303 conn(0x7fc3480fd740 msgr2=0x7fc3480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3358350303 shutdown_connections 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.011+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3358350303 wait complete. 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 Processor -- start 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 -- start start 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc348103ce0 0x7fc34810bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348104600 0x7fc34810c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc348112db0 con 0x7fc348104600 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc348112c30 con 0x7fc34810d140 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.012+0000 7fc34e1a7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc348112f30 con 0x7fc348103ce0 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc346ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348104600 0x7fc34810c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:41822/0 (socket says 172.21.15.96:41822) 2024-09-10T22:55:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 -- 172.21.15.96:0/3644837857 learned_addr learned my addr 172.21.15.96:0/3644837857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:37.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:36 smithi121 ceph-mon[35119]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-09-10T22:55:37.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:37.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:37.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:55:37.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 -- 172.21.15.96:0/3644837857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc348103ce0 msgr2=0x7fc34810bea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:55:37.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc348103ce0 0x7fc34810bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 -- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348104600 msgr2=0x7fc34810c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348104600 0x7fc34810c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 -- 172.21.15.96:0/3644837857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3481a5970 con 0x7fc34810d140 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc347fff640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc34000eb10 tx=0x7fc34000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.013+0000 7fc344ff9640 1 -- 172.21.15.96:0/3644837857 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340004b20 con 0x7fc34810d140 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.014+0000 7fc344ff9640 1 -- 172.21.15.96:0/3644837857 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340004cc0 con 0x7fc34810d140 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.014+0000 7fc344ff9640 1 -- 172.21.15.96:0/3644837857 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340010720 con 0x7fc34810d140 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.014+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3481a5c60 con 0x7fc34810d140 2024-09-10T22:55:37.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.014+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc348198240 con 0x7fc34810d140 2024-09-10T22:55:37.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.018+0000 7fc344ff9640 1 -- 172.21.15.96:0/3644837857 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc3400108c0 con 0x7fc34810d140 2024-09-10T22:55:37.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.019+0000 7fc344ff9640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 0x7fc3300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.019+0000 7fc344ff9640 1 -- 172.21.15.96:0/3644837857 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc3400e60f0 con 0x7fc34810d140 2024-09-10T22:55:37.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.019+0000 7fc35057c740 1 Processor -- start 2024-09-10T22:55:37.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.019+0000 7fc3477fe640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 0x7fc3300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 -- start start 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 0x55bf36804840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf3676eea0 con 0x55bf367f84d0 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf3675e2d0 con 0x55bf367f8df0 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc35057c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bf367620e0 con 0x55bf36804d80 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc347fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc347fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36264/0 (socket says 172.21.15.96:36264) 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc347fff640 1 -- 172.21.15.96:0/1794229288 learned_addr learned my addr 172.21.15.96:0/1794229288 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc346ffd640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 0x55bf36804840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc3477fe640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc3477fe640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 0x7fc3300b7820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc33802a910 tx=0x7fc338000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc347fff640 1 -- 172.21.15.96:0/1794229288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 msgr2=0x55bf36804840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.020+0000 7fc347fff640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 0x55bf36804840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc347fff640 1 -- 172.21.15.96:0/1794229288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 msgr2=0x55bf367f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc347fff640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc347fff640 1 -- 172.21.15.96:0/1794229288 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf364bc650 con 0x55bf36804d80 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc346ffd640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 0x55bf36804840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc347fff640 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3400ac840 tx=0x7fc3400a9030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed536e434b8e2f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc32dffb640 1 -- 172.21.15.96:0/1794229288 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400b1070 con 0x55bf36804d80 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc32dffb640 1 -- 172.21.15.96:0/1794229288 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3400ac3c0 con 0x55bf36804d80 2024-09-10T22:55:37.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.021+0000 7fc32dffb640 1 -- 172.21.15.96:0/1794229288 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400ade40 con 0x55bf36804d80 2024-09-10T22:55:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- 172.21.15.96:0/1794229288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 msgr2=0x55bf36807170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3400ac840 tx=0x7fc3400a9030 comp rx=0 tx=0).stop 2024-09-10T22:55:37.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- 172.21.15.96:0/1794229288 shutdown_connections 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf367f8df0 0x55bf36804840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf36804d80 0x55bf36807170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 --2- 172.21.15.96:0/1794229288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- 172.21.15.96:0/1794229288 >> 172.21.15.96:0/1794229288 conn(0x55bf367f6300 msgr2=0x55bf367f74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- 172.21.15.96:0/1794229288 shutdown_connections 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- 172.21.15.96:0/1794229288 wait complete. 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.022+0000 7fc35057c740 1 -- start start 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 Processor -- start 2024-09-10T22:55:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36801b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf368021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf365bf700 con 0x55bf367f84d0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf366f9160 con 0x55bf36804d80 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bf36807a00 con 0x55bf367f8df0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc35057c740 10 client.0 _ll_register_callbacks cb 0x55bf367129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc3477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc3477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48960/0 (socket says 172.21.15.96:48960) 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc3477fe640 1 -- 172.21.15.96:0/3591732021 learned_addr learned my addr 172.21.15.96:0/3591732021 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.023+0000 7fc346ffd640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36801b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc347fff640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf368021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 -- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 msgr2=0x55bf368021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf368021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 -- 172.21.15.96:0/3591732021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 msgr2=0x55bf368046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 -- 172.21.15.96:0/3591732021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf36726c90 con 0x55bf367f8df0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc3477fe640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc347fff640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf368021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc346ffd640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36801b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc34810e8a0 tx=0x7fc33c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbcf047d39275935 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c004030 con 0x55bf367f8df0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc33c0041d0 con 0x55bf367f8df0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3400ac3c0 con 0x55bf367f8df0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc33c0043d0 con 0x55bf367f8df0 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.024+0000 7fc35057c740 10 client.26827 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.025+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf368134a0 con 0x55bf367f8df0 2024-09-10T22:55:37.031 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:36 smithi117 ceph-mon[35081]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 647 op/s 2024-09-10T22:55:37.031 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:55:37.031 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:55:37.031 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:55:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.025+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3400ac3c0 con 0x55bf367f8df0 2024-09-10T22:55:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc33c003a40 con 0x55bf367f8df0 2024-09-10T22:55:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 10 client.26827 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 10 client.26827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 20 client.26827 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf368149b0 0x55bf36816de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 4 client.26827 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.027+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf36770df0 con 0x55bf368149b0 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.028+0000 7fc3477fe640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf368149b0 0x55bf36816de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.029+0000 7fc326ffd640 10 client.26827 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.029+0000 7fc3477fe640 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf368149b0 0x55bf36816de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3269e820c4a2320 server_cookie=97fd4acc8c80ac0f in_seq=0 out_seq=0 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.029+0000 7fc326ffd640 1 -- 172.21.15.96:0/3591732021 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf36770df0 con 0x55bf368149b0 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.029+0000 7fc326ffd640 10 client.26827 handle_command_reply: tid=0 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 client.26827 shutdown 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 msgr2=0x55bf36801b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36801b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc34810e8a0 tx=0x7fc33c012b60 comp rx=0 tx=0).stop 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 shutdown_connections 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf368149b0 0x55bf36816de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf368021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36801b40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 --2- 172.21.15.96:0/3591732021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf368046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 >> 172.21.15.96:0/3591732021 conn(0x55bf367f6300 msgr2=0x55bf367fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc326ffd640 0 client.26827 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 shutdown_connections 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 1 -- 172.21.15.96:0/3591732021 wait complete. 2024-09-10T22:55:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.030+0000 7fc35057c740 20 client.26827 trim_cache size 0 max 16384 2024-09-10T22:55:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.047+0000 7fc35057c740 1 Processor -- start 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 -- start start 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36810c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400e190 con 0x55bf367f84d0 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400fcd0 con 0x55bf36804d80 2024-09-10T22:55:37.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc35057c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400e860 con 0x55bf367f8df0 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc347fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc347fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:41854/0 (socket says 172.21.15.96:41854) 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.048+0000 7fc347fff640 1 -- 172.21.15.96:0/1978004827 learned_addr learned my addr 172.21.15.96:0/1978004827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.049+0000 7fc347fff640 1 -- 172.21.15.96:0/1978004827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 msgr2=0x55bf36810c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.049+0000 7fc347fff640 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36810c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.049+0000 7fc347fff640 1 -- 172.21.15.96:0/1978004827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 msgr2=0x55bf367f78b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.049+0000 7fc347fff640 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.049+0000 7fc347fff640 1 -- 172.21.15.96:0/1978004827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf367f6d80 con 0x55bf36804d80 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.050+0000 7fc347fff640 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3400049d0 tx=0x7fc340004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac548aef13c62f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.050+0000 7fc32d7fa640 1 -- 172.21.15.96:0/1978004827 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400c7070 con 0x55bf36804d80 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.050+0000 7fc32d7fa640 1 -- 172.21.15.96:0/1978004827 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3400ac3c0 con 0x55bf36804d80 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.050+0000 7fc32d7fa640 1 -- 172.21.15.96:0/1978004827 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400b1040 con 0x55bf36804d80 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 -- 172.21.15.96:0/1978004827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 msgr2=0x55bf36806170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3400049d0 tx=0x7fc340004a40 comp rx=0 tx=0).stop 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 -- 172.21.15.96:0/1978004827 shutdown_connections 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf36806170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf36810c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 --2- 172.21.15.96:0/1978004827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf367f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 -- 172.21.15.96:0/1978004827 >> 172.21.15.96:0/1978004827 conn(0x55bf367f6300 msgr2=0x55bf367f71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.051+0000 7fc35057c740 1 -- 172.21.15.96:0/1978004827 shutdown_connections 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.052+0000 7fc35057c740 1 -- 172.21.15.96:0/1978004827 wait complete. 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.052+0000 7fc35057c740 1 -- start start 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.052+0000 7fc35057c740 1 Processor -- start 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.052+0000 7fc35057c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf3687ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.052+0000 7fc35057c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf3687b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400f600 con 0x55bf367f84d0 2024-09-10T22:55:37.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400ef30 con 0x55bf36804d80 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc33400c220 con 0x55bf367f8df0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 10 client.0 _ll_register_callbacks cb 0x7fc33c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36282/0 (socket says 172.21.15.96:36282) 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 -- 172.21.15.96:0/161134356 learned_addr learned my addr 172.21.15.96:0/161134356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc347fff640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf3687b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc3477fe640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf3687ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 -- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 msgr2=0x55bf3687b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf3687b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 -- 172.21.15.96:0/161134356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 msgr2=0x55bf3687ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf3687ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 -- 172.21.15.96:0/161134356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf368111d0 con 0x55bf367f8df0 2024-09-10T22:55:37.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc3477fe640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf3687ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc346ffd640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc33c001eb0 tx=0x7fc33c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c7d96e5808d1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c0038c0 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc33c003a60 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.053+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3400ac3c0 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc33c00f790 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 10 client.26830 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf3680c490 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc33c00fde0 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc33c030a90 con 0x55bf367f8df0 2024-09-10T22:55:37.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 10 client.26830 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 10 client.26830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 20 client.26830 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf3680cc40 0x55bf3680d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 4 client.26830 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc3400ac3c0 con 0x55bf3680cc40 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.054+0000 7fc3477fe640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf3680cc40 0x55bf3680d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc326ffd640 10 client.26830 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc3477fe640 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf3680cc40 0x55bf3680d050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28d5cb5034d7624f server_cookie=71803bc86daa9e20 in_seq=0 out_seq=0 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc326ffd640 1 -- 172.21.15.96:0/161134356 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc3400ac3c0 con 0x55bf3680cc40 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc326ffd640 10 client.26830 handle_command_reply: tid=0 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 client.26830 shutdown 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 msgr2=0x55bf3687f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc33c001eb0 tx=0x7fc33c004200 comp rx=0 tx=0).stop 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 shutdown_connections 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55bf3680cc40 0x55bf3680d050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf36804d80 0x55bf3687b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf367f8df0 0x55bf3687f010 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 --2- 172.21.15.96:0/161134356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf367f84d0 0x55bf3687ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 >> 172.21.15.96:0/161134356 conn(0x55bf367f6300 msgr2=0x55bf36803170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc326ffd640 0 client.26830 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 shutdown_connections 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 1 -- 172.21.15.96:0/161134356 wait complete. 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.055+0000 7fc35057c740 20 client.26830 trim_cache size 0 max 16384 2024-09-10T22:55:37.064 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 msgr2=0x7fc3300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 0x7fc3300b7820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc33802a910 tx=0x7fc338000bc0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 msgr2=0x7fc3481a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc34000eb10 tx=0x7fc34000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 shutdown_connections 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc34810d140 0x7fc3481a51a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc3300b5360 0x7fc3300b7820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc348104600 0x7fc34810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 --2- 172.21.15.96:0/3644837857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc348103ce0 0x7fc34810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 >> 172.21.15.96:0/3644837857 conn(0x7fc3480fd740 msgr2=0x7fc34810f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:37.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 shutdown_connections 2024-09-10T22:55:37.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:37.056+0000 7fc34e1a7640 1 -- 172.21.15.96:0/3644837857 wait complete. 2024-09-10T22:55:37.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:55:38.316 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:38 smithi096 ceph-mon[30869]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 522 op/s 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 -- 172.21.15.96:0/1379898242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee410d100 msgr2=0x7fbee410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 --2- 172.21.15.96:0/1379898242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee410d100 0x7fbee410f560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009ca0 tx=0x7fbedc02a8d0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 -- 172.21.15.96:0/1379898242 shutdown_connections 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 --2- 172.21.15.96:0/1379898242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee410d100 0x7fbee410f560 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 --2- 172.21.15.96:0/1379898242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee41045c0 0x7fbee410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 --2- 172.21.15.96:0/1379898242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee4103ca0 0x7fbee4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 -- 172.21.15.96:0/1379898242 >> 172.21.15.96:0/1379898242 conn(0x7fbee40fd700 msgr2=0x7fbee40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 -- 172.21.15.96:0/1379898242 shutdown_connections 2024-09-10T22:55:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.915+0000 7fbeeab34640 1 -- 172.21.15.96:0/1379898242 wait complete. 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 Processor -- start 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 -- start start 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 0x7fbee4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 0x7fbee4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 0x7fbee419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4112e00 con 0x7fbee4103ca0 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4112c80 con 0x7fbee410d100 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbeeab34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4112f80 con 0x7fbee41045c0 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 0x7fbee4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.916+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 0x7fbee4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 0x7fbee4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48992/0 (socket says 172.21.15.96:48992) 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee88d0640 1 -- 172.21.15.96:0/3121437659 learned_addr learned my addr 172.21.15.96:0/3121437659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee90d1640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 0x7fbee419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 -- 172.21.15.96:0/3121437659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 msgr2=0x7fbee419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 0x7fbee419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 -- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 msgr2=0x7fbee4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 0x7fbee4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 -- 172.21.15.96:0/3121437659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee419f300 con 0x7fbee41045c0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee90d1640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 0x7fbee419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee3fff640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 0x7fbee4199790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c940 tx=0x7fbed000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee1ffb640 1 -- 172.21.15.96:0/3121437659 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0007bb0 con 0x7fbee41045c0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee1ffb640 1 -- 172.21.15.96:0/3121437659 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed0007d50 con 0x7fbee41045c0 2024-09-10T22:55:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee419f5f0 con 0x7fbee41045c0 2024-09-10T22:55:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.917+0000 7fbee1ffb640 1 -- 172.21.15.96:0/3121437659 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0018660 con 0x7fbee41045c0 2024-09-10T22:55:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.918+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41a5e80 con 0x7fbee41045c0 2024-09-10T22:55:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.921+0000 7fbee1ffb640 1 -- 172.21.15.96:0/3121437659 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbed000f450 con 0x7fbee41045c0 2024-09-10T22:55:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.921+0000 7fbeecf09740 1 Processor -- start 2024-09-10T22:55:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.922+0000 7fbeecf09740 1 -- start start 2024-09-10T22:55:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.922+0000 7fbee1ffb640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 0x7fbed40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.922+0000 7fbee1ffb640 1 -- 172.21.15.96:0/3121437659 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbed00e5a60 con 0x7fbee41045c0 2024-09-10T22:55:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.922+0000 7fbee90d1640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 0x7fbed40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e90b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e90e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d6e875ea0 con 0x563d6e8ff4d0 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d6e8652d0 con 0x563d6e90bd80 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbeecf09740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563d6e8690e0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49002/0 (socket says 172.21.15.96:49002) 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 -- 172.21.15.96:0/2062219708 learned_addr learned my addr 172.21.15.96:0/2062219708 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee3fff640 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e90b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee90d1640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 0x7fbed40b7820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbedc02f3b0 tx=0x7fbedc02f320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 -- 172.21.15.96:0/2062219708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 msgr2=0x563d6e90b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e90b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 -- 172.21.15.96:0/2062219708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 msgr2=0x563d6e90e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e90e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 -- 172.21.15.96:0/2062219708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6e5c3650 con 0x563d6e8ff4d0 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee3fff640 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e90b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.923+0000 7fbee88d0640 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009b20 tx=0x7fbed800e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fd889b6033cb02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.924+0000 7fbeceffd640 1 -- 172.21.15.96:0/2062219708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8017070 con 0x563d6e8ff4d0 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.924+0000 7fbeceffd640 1 -- 172.21.15.96:0/2062219708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed800ec10 con 0x563d6e8ff4d0 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.924+0000 7fbeceffd640 1 -- 172.21.15.96:0/2062219708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8012630 con 0x563d6e8ff4d0 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.924+0000 7fbeecf09740 1 -- 172.21.15.96:0/2062219708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 msgr2=0x563d6e8ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.924+0000 7fbeecf09740 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009b20 tx=0x7fbed800e1d0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 -- 172.21.15.96:0/2062219708 shutdown_connections 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e90e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e90b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 --2- 172.21.15.96:0/2062219708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e8ff8b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 -- 172.21.15.96:0/2062219708 >> 172.21.15.96:0/2062219708 conn(0x563d6e8fd300 msgr2=0x563d6e8fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 -- 172.21.15.96:0/2062219708 shutdown_connections 2024-09-10T22:55:38.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 -- 172.21.15.96:0/2062219708 wait complete. 2024-09-10T22:55:38.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 -- start start 2024-09-10T22:55:38.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.925+0000 7fbeecf09740 1 Processor -- start 2024-09-10T22:55:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e90b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 0x563d6e908ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 0x563d6e909000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d6e6c6700 con 0x563d6e8ff4d0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d6e800160 con 0x563d6e8ffdf0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563d6e90ea00 con 0x563d6e90bd80 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbeecf09740 10 client.0 _ll_register_callbacks cb 0x563d6e8199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e90b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e90b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49016/0 (socket says 172.21.15.96:49016) 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee88d0640 1 -- 172.21.15.96:0/262552932 learned_addr learned my addr 172.21.15.96:0/262552932 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 0x563d6e908ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee90d1640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 0x563d6e909000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 -- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 msgr2=0x563d6e909000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 0x563d6e909000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 -- 172.21.15.96:0/262552932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 msgr2=0x563d6e90b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e90b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee3fff640 1 -- 172.21.15.96:0/262552932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6e82dc90 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.926+0000 7fbee90d1640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 0x563d6e909000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbee3fff640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 0x563d6e908ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbed00e3650 tx=0x7fbed00b5780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e07b6bf00c3bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed00abdd0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed800ec10 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbed000f980 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.927+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbed00abdd0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.928+0000 7fbeecf09740 10 client.26777 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.928+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d6e917ca0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.928+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed800ec10 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbed00cf070 con 0x563d6e8ffdf0 2024-09-10T22:55:38.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbeecf09740 10 client.26777 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbeecf09740 10 client.26777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbeecf09740 20 client.26777 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e91b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.929+0000 7fbeecf09740 4 client.26777 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.930+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6e877df0 con 0x563d6e918ff0 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.930+0000 7fbee88d0640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e91b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.930+0000 7fbee88d0640 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e91b4b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20f61d8b78435049 server_cookie=ecafe768ee476445 in_seq=0 out_seq=0 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.930+0000 7fbec3fff640 10 client.26777 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbec3fff640 1 -- 172.21.15.96:0/262552932 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d6e877df0 con 0x563d6e918ff0 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbec3fff640 10 client.26777 handle_command_reply: tid=0 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 client.26777 shutdown 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 msgr2=0x563d6e908ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 0x563d6e908ac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbed00e3650 tx=0x7fbed00b5780 comp rx=0 tx=0).stop 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 shutdown_connections 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e91b4b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbec3fff640 0 client.26777 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ffdf0 0x563d6e908ac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e90bd80 0x563d6e909000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 --2- 172.21.15.96:0/262552932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e90b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.931+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 >> 172.21.15.96:0/262552932 conn(0x563d6e8fd300 msgr2=0x563d6e902e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.932+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 shutdown_connections 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.932+0000 7fbeecf09740 1 -- 172.21.15.96:0/262552932 wait complete. 2024-09-10T22:55:38.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.932+0000 7fbeecf09740 20 client.26777 trim_cache size 0 max 16384 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.950+0000 7fbeecf09740 1 Processor -- start 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.950+0000 7fbeecf09740 1 -- start start 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.950+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ff4d0 0x563d6e907fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.950+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.950+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e90bd80 0x563d6e908fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbeecf09740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41ac380 con 0x563d6e90bd80 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbeecf09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41adec0 con 0x563d6e8ff4d0 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbeecf09740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41aca50 con 0x563d6e8ffdf0 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36322/0 (socket says 172.21.15.96:36322) 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 -- 172.21.15.96:0/426567866 learned_addr learned my addr 172.21.15.96:0/426567866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee90d1640 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e90bd80 0x563d6e908fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 -- 172.21.15.96:0/426567866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ff4d0 msgr2=0x563d6e907fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ff4d0 0x563d6e907fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 -- 172.21.15.96:0/426567866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e90bd80 msgr2=0x563d6e908fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e90bd80 0x563d6e908fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.951+0000 7fbee3fff640 1 -- 172.21.15.96:0/426567866 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6e906fc0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.952+0000 7fbee3fff640 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbed00ac250 tx=0x7fbed0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf7666ef8f1dc3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.952+0000 7fbece7fc640 1 -- 172.21.15.96:0/426567866 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed00abdd0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.952+0000 7fbece7fc640 1 -- 172.21.15.96:0/426567866 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed00ad3e0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.952+0000 7fbece7fc640 1 -- 172.21.15.96:0/426567866 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0093790 con 0x563d6e8ffdf0 2024-09-10T22:55:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- 172.21.15.96:0/426567866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 msgr2=0x563d6e908a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbed00ac250 tx=0x7fbed0000bc0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- 172.21.15.96:0/426567866 shutdown_connections 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e90bd80 0x563d6e908fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e908a90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 --2- 172.21.15.96:0/426567866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e8ff4d0 0x563d6e907fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- 172.21.15.96:0/426567866 >> 172.21.15.96:0/426567866 conn(0x563d6e8fd300 msgr2=0x563d6e903dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- 172.21.15.96:0/426567866 shutdown_connections 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- 172.21.15.96:0/426567866 wait complete. 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 -- start start 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.953+0000 7fbeecf09740 1 Processor -- start 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e936830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e936d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e941ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41ad7f0 con 0x563d6e8ff4d0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41ad120 con 0x563d6e90bd80 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41aa300 con 0x563d6e8ffdf0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbeecf09740 10 client.0 _ll_register_callbacks cb 0x7fbed0007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e936830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbee88d0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e936830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49026/0 (socket says 172.21.15.96:49026) 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbee88d0640 1 -- 172.21.15.96:0/732336950 learned_addr learned my addr 172.21.15.96:0/732336950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbee3fff640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e936d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.954+0000 7fbee90d1640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e941ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 -- 172.21.15.96:0/732336950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 msgr2=0x563d6e941ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e941ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 -- 172.21.15.96:0/732336950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 msgr2=0x563d6e936830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e936830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 -- 172.21.15.96:0/732336950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6e942660 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbee3fff640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e936d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbed00a8890 tx=0x7fbed00a8910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a0c64a85f4dca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0093790 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed00ad3e0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.955+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbed0018d70 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.956+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbed00aba30 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.956+0000 7fbeecf09740 10 client.26848 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.956+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d6e942660 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.957+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed00ad3e0 con 0x563d6e8ffdf0 2024-09-10T22:55:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.957+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbed00cf070 con 0x563d6e8ffdf0 2024-09-10T22:55:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 10 client.26848 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 10 client.26848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 20 client.26848 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e9ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 4 client.26848 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d6e8738a0 con 0x563d6e918ff0 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbee88d0640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e9ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbec3fff640 10 client.26848 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.958+0000 7fbee88d0640 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e9ad240 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5defe7b3eaeb7cfc server_cookie=e36e7b52470e5720 in_seq=0 out_seq=0 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbec3fff640 1 -- 172.21.15.96:0/732336950 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d6e8738a0 con 0x563d6e918ff0 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbec3fff640 10 client.26848 handle_command_reply: tid=0 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 client.26848 shutdown 2024-09-10T22:55:38.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 msgr2=0x563d6e936d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e936d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbed00a8890 tx=0x7fbed00a8910 comp rx=0 tx=0).stop 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 shutdown_connections 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x563d6e918ff0 0x563d6e9ad240 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d6e90bd80 0x563d6e941ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d6e8ffdf0 0x563d6e936d70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 --2- 172.21.15.96:0/732336950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d6e8ff4d0 0x563d6e936830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 >> 172.21.15.96:0/732336950 conn(0x563d6e8fd300 msgr2=0x563d6e908a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbec3fff640 0 client.26848 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 shutdown_connections 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 1 -- 172.21.15.96:0/732336950 wait complete. 2024-09-10T22:55:38.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.959+0000 7fbeecf09740 20 client.26848 trim_cache size 0 max 16384 2024-09-10T22:55:38.966 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:55:38.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 msgr2=0x7fbed40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 0x7fbed40b7820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbedc02f3b0 tx=0x7fbedc02f320 comp rx=0 tx=0).stop 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 msgr2=0x7fbee4199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 0x7fbee4199790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c940 tx=0x7fbed000ce00 comp rx=0 tx=0).stop 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 shutdown_connections 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee410d100 0x7fbee419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbed40b5360 0x7fbed40b7820 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbee41045c0 0x7fbee4199790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 --2- 172.21.15.96:0/3121437659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbee4103ca0 0x7fbee4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:38.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 >> 172.21.15.96:0/3121437659 conn(0x7fbee40fd700 msgr2=0x7fbee410f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:38.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 shutdown_connections 2024-09-10T22:55:38.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:38.960+0000 7fbeeab34640 1 -- 172.21.15.96:0/3121437659 wait complete. 2024-09-10T22:55:39.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:38 smithi121 ceph-mon[35119]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 522 op/s 2024-09-10T22:55:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:38 smithi117 ceph-mon[35081]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 522 op/s 2024-09-10T22:55:39.848 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:55:40.201 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:40.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.778+0000 7f9912a86640 1 -- 172.21.15.96:0/211504075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 msgr2=0x7f990c107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.778+0000 7f9912a86640 1 --2- 172.21.15.96:0/211504075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c107340 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f990401a910 tx=0x7f990403b430 comp rx=0 tx=0).stop 2024-09-10T22:55:40.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 -- 172.21.15.96:0/211504075 shutdown_connections 2024-09-10T22:55:40.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 --2- 172.21.15.96:0/211504075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c1068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 --2- 172.21.15.96:0/211504075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c107340 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 --2- 172.21.15.96:0/211504075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 -- 172.21.15.96:0/211504075 >> 172.21.15.96:0/211504075 conn(0x7f990c0fd700 msgr2=0x7f990c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 -- 172.21.15.96:0/211504075 shutdown_connections 2024-09-10T22:55:40.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.779+0000 7f9912a86640 1 -- 172.21.15.96:0/211504075 wait complete. 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 Processor -- start 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 -- start start 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c111f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c112440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f990c10a6e0 con 0x7f990c101ab0 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f990c10a560 con 0x7f990c1023d0 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9912a86640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f990c10a860 con 0x7f990c106ef0 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9910822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9911023640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9910822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c111f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49034/0 (socket says 172.21.15.96:49034) 2024-09-10T22:55:40.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f9910822640 1 -- 172.21.15.96:0/3154914421 learned_addr learned my addr 172.21.15.96:0/3154914421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:40.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.780+0000 7f990bfff640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c112440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 -- 172.21.15.96:0/3154914421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 msgr2=0x7f990c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 -- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 msgr2=0x7f990c111f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c111f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 -- 172.21.15.96:0/3154914421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c19d110 con 0x7f990c1023d0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.781+0000 7f990bfff640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c112440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c710 tx=0x7f98f400cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.782+0000 7f9909ffb640 1 -- 172.21.15.96:0/3154914421 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004220 con 0x7f990c1023d0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.782+0000 7f9909ffb640 1 -- 172.21.15.96:0/3154914421 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f400ce20 con 0x7f990c1023d0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.782+0000 7f9909ffb640 1 -- 172.21.15.96:0/3154914421 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4003da0 con 0x7f990c1023d0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.782+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c19d400 con 0x7f990c1023d0 2024-09-10T22:55:40.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.782+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1a3b30 con 0x7f990c1023d0 2024-09-10T22:55:40.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.785+0000 7f9914e5b740 1 Processor -- start 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.786+0000 7f9909ffb640 1 -- 172.21.15.96:0/3154914421 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f98f40079b0 con 0x7f990c1023d0 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9909ffb640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 0x7f98f80b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 -- start start 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9909ffb640 1 -- 172.21.15.96:0/3154914421 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98f40e5580 con 0x7f990c1023d0 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3da78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561be3d1dea0 con 0x561be3db3d80 2024-09-10T22:55:40.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561be3d0d2d0 con 0x561be3da74d0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9914e5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561be3d110e0 con 0x561be3da7df0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9911023640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 0x7f98f80b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f990bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9911023640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f990bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36368/0 (socket says 172.21.15.96:36368) 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f990bfff640 1 -- 172.21.15.96:0/1501839408 learned_addr learned my addr 172.21.15.96:0/1501839408 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.787+0000 7f9910822640 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3da78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 -- 172.21.15.96:0/1501839408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 msgr2=0x561be3da78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3da78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 -- 172.21.15.96:0/1501839408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 msgr2=0x561be3db6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 -- 172.21.15.96:0/1501839408 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be3a6b650 con 0x561be3da7df0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f9911023640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 0x7f98f80b7970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99040057c0 tx=0x7f990401a690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f990bfff640 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98f40abd80 tx=0x7f98f40ab740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad1790cd180bd0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f9902ffd640 1 -- 172.21.15.96:0/1501839408 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40b0070 con 0x561be3da7df0 2024-09-10T22:55:40.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f9902ffd640 1 -- 172.21.15.96:0/1501839408 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f40abdf0 con 0x561be3da7df0 2024-09-10T22:55:40.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.788+0000 7f9902ffd640 1 -- 172.21.15.96:0/1501839408 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f401d040 con 0x561be3da7df0 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.789+0000 7f9914e5b740 1 -- 172.21.15.96:0/1501839408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 msgr2=0x561be3db3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.789+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98f40abd80 tx=0x7f98f40ab740 comp rx=0 tx=0).stop 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 -- 172.21.15.96:0/1501839408 shutdown_connections 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da7df0 0x561be3db3840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1501839408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3da78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 -- 172.21.15.96:0/1501839408 >> 172.21.15.96:0/1501839408 conn(0x561be3da5300 msgr2=0x561be3da64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 -- 172.21.15.96:0/1501839408 shutdown_connections 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 -- 172.21.15.96:0/1501839408 wait complete. 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.791+0000 7f9914e5b740 1 -- start start 2024-09-10T22:55:40.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 Processor -- start 2024-09-10T22:55:40.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da74d0 0x561be3db36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3db0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561be3b6e700 con 0x561be3da74d0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561be3ca8160 con 0x561be3da7df0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561be3db6a00 con 0x561be3db3d80 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9911023640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9914e5b740 10 client.0 _ll_register_callbacks cb 0x561be3cc19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9911023640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36376/0 (socket says 172.21.15.96:36376) 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9911023640 1 -- 172.21.15.96:0/2123885719 learned_addr learned my addr 172.21.15.96:0/2123885719 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.792+0000 7f9910822640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da74d0 0x561be3db36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f990bfff640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3db0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 -- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 msgr2=0x561be3db0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3db0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 -- 172.21.15.96:0/2123885719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da74d0 msgr2=0x561be3db36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da74d0 0x561be3db36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 -- 172.21.15.96:0/2123885719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be3cd5c90 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.793+0000 7f9911023640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9904044000 tx=0x7f9904005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c16ac9c98e34f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f98ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904060070 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9904040ba0 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98f40abdf0 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f990406c050 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f9914e5b740 10 client.26860 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.794+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561be3dc23c0 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f40abdf0 con 0x561be3db3d80 2024-09-10T22:55:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f990404b050 con 0x561be3db3d80 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 10 client.26860 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 20 client.26860 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3dc3820 0x561be3dc5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 4 client.26860 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be3d1fdf0 con 0x561be3dc3820 2024-09-10T22:55:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.795+0000 7f9910822640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3dc3820 0x561be3dc5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.797+0000 7f9910822640 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3dc3820 0x561be3dc5c10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e66ea5293a8a63e server_cookie=94499f4509cabc98 in_seq=0 out_seq=0 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.797+0000 7f98ebfff640 10 client.26860 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f98ebfff640 1 -- 172.21.15.96:0/2123885719 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561be3d1fdf0 con 0x561be3dc3820 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f98ebfff640 10 client.26860 handle_command_reply: tid=0 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 client.26860 shutdown 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 msgr2=0x561be3db1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9904044000 tx=0x7f9904005a20 comp rx=0 tx=0).stop 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 shutdown_connections 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3dc3820 0x561be3dc5c10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3db0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db1180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f9914e5b740 1 --2- 172.21.15.96:0/2123885719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da74d0 0x561be3db36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.798+0000 7f98ebfff640 0 client.26860 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.799+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 >> 172.21.15.96:0/2123885719 conn(0x561be3da5300 msgr2=0x561be3daae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.799+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 shutdown_connections 2024-09-10T22:55:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.799+0000 7f9914e5b740 1 -- 172.21.15.96:0/2123885719 wait complete. 2024-09-10T22:55:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.799+0000 7f9914e5b740 20 client.26860 trim_cache size 0 max 16384 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.820+0000 7f9914e5b740 1 Processor -- start 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.820+0000 7f9914e5b740 1 -- start start 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3dbfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 0x561be3db2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1ac100 con 0x561be3da7df0 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1adc40 con 0x561be3da74d0 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9914e5b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1ac7d0 con 0x561be3db3d80 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f990bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 0x561be3db2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f990bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 0x561be3db2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49076/0 (socket says 172.21.15.96:49076) 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f990bfff640 1 -- 172.21.15.96:0/1452439775 learned_addr learned my addr 172.21.15.96:0/1452439775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9910822640 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3dbfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 -- 172.21.15.96:0/1452439775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 msgr2=0x561be3dbfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3dbfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 -- 172.21.15.96:0/1452439775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 msgr2=0x561be3db2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 0x561be3db2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.821+0000 7f9911023640 1 -- 172.21.15.96:0/1452439775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be3da69e0 con 0x561be3db3d80 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.822+0000 7f9911023640 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9904004f70 tx=0x7f99040057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b315ddaef4c56679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.822+0000 7f99027fc640 1 -- 172.21.15.96:0/1452439775 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904060070 con 0x561be3db3d80 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.822+0000 7f99027fc640 1 -- 172.21.15.96:0/1452439775 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99040053b0 con 0x561be3db3d80 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.822+0000 7f99027fc640 1 -- 172.21.15.96:0/1452439775 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904046040 con 0x561be3db3d80 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.823+0000 7f9914e5b740 1 -- 172.21.15.96:0/1452439775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 msgr2=0x561be3db5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.823+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db5170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9904004f70 tx=0x7f99040057f0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 -- 172.21.15.96:0/1452439775 shutdown_connections 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3db3d80 0x561be3db5170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3da7df0 0x561be3db2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 --2- 172.21.15.96:0/1452439775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da74d0 0x561be3dbfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 -- 172.21.15.96:0/1452439775 >> 172.21.15.96:0/1452439775 conn(0x561be3da5300 msgr2=0x561be3db2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 -- 172.21.15.96:0/1452439775 shutdown_connections 2024-09-10T22:55:40.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 -- 172.21.15.96:0/1452439775 wait complete. 2024-09-10T22:55:40.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 -- start start 2024-09-10T22:55:40.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.824+0000 7f9914e5b740 1 Processor -- start 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3dbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9910822640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9910822640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36400/0 (socket says 172.21.15.96:36400) 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9910822640 1 -- 172.21.15.96:0/3439631743 learned_addr learned my addr 172.21.15.96:0/3439631743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1ad570 con 0x561be3db3d80 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1acea0 con 0x561be3da7df0 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1aa0e0 con 0x561be3da74d0 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.825+0000 7f9914e5b740 10 client.0 _ll_register_callbacks cb 0x7f99040048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:40.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f990bfff640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3dbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 -- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 msgr2=0x561be3dbc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3dbc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 -- 172.21.15.96:0/3439631743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 msgr2=0x561be3db6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be3db7500 con 0x561be3da74d0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9910822640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x561be3dc6450 tx=0x7f98fc003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3bc7646c503ff9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f98ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc004240 con 0x561be3da74d0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98fc0043e0 con 0x561be3da74d0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.826+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561be3d1b8a0 con 0x561be3da74d0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.827+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f98fc004600 con 0x561be3da74d0 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.827+0000 7f9914e5b740 10 client.26866 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:40.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.827+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561be3e312c0 con 0x561be3da74d0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.827+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc02a070 con 0x561be3da74d0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.827+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f98fc00f040 con 0x561be3da74d0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 10 client.26866 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 20 client.26866 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3e31a40 0x561be3e33e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 4 client.26866 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561be3d1b8a0 con 0x561be3e31a40 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f990bfff640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3e31a40 0x561be3e33e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f98ebfff640 10 client.26866 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.828+0000 7f990bfff640 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3e31a40 0x561be3e33e60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b289196f43a82d0e server_cookie=264c3b6a947156de in_seq=0 out_seq=0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f98ebfff640 1 -- 172.21.15.96:0/3439631743 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561be3d1b8a0 con 0x561be3e31a40 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f98ebfff640 10 client.26866 handle_command_reply: tid=0 2024-09-10T22:55:40.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 client.26866 shutdown 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 msgr2=0x561be3dbbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x561be3dc6450 tx=0x7f98fc003f60 comp rx=0 tx=0).stop 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 shutdown_connections 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561be3e31a40 0x561be3e33e60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f98ebfff640 0 client.26866 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be3da7df0 0x561be3dbc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561be3db3d80 0x561be3db6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 --2- 172.21.15.96:0/3439631743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561be3da74d0 0x561be3dbbfa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 >> 172.21.15.96:0/3439631743 conn(0x561be3da5300 msgr2=0x561be3db2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 shutdown_connections 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 1 -- 172.21.15.96:0/3439631743 wait complete. 2024-09-10T22:55:40.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.829+0000 7f9914e5b740 20 client.26866 trim_cache size 0 max 16384 2024-09-10T22:55:40.835 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:55:40.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.830+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 msgr2=0x7f98f80b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.830+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 0x7f98f80b7970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99040057c0 tx=0x7f990401a690 comp rx=0 tx=0).stop 2024-09-10T22:55:40.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 msgr2=0x7f990c112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:40.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c112440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c710 tx=0x7f98f400cbd0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 shutdown_connections 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f990c1023d0 0x7f990c112440 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98f80b54b0 0x7f98f80b7970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f990c106ef0 0x7f990c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 --2- 172.21.15.96:0/3154914421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f990c101ab0 0x7f990c111f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 >> 172.21.15.96:0/3154914421 conn(0x7f990c0fd700 msgr2=0x7f990c104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 shutdown_connections 2024-09-10T22:55:40.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:40.831+0000 7f9912a86640 1 -- 172.21.15.96:0/3154914421 wait complete. 2024-09-10T22:55:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 485.0 s, 1710.4 tps, lat 58.941 ms stddev 69.615 2024-09-10T22:55:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:40 smithi096 ceph-mon[30869]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-09-10T22:55:41.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:40 smithi121 ceph-mon[35119]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-09-10T22:55:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:40 smithi117 ceph-mon[35081]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-09-10T22:55:42.605 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:55:42.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:55:42.972 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:43.003 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:42 smithi096 ceph-mon[30869]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-09-10T22:55:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:42 smithi121 ceph-mon[35119]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-09-10T22:55:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:42 smithi117 ceph-mon[35081]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 -- 172.21.15.96:0/1052936130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca5810a150 msgr2=0x7fca5810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/1052936130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca5810a150 0x7fca5810a530 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fca48009900 tx=0x7fca4802a480 comp rx=0 tx=0).stop 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 -- 172.21.15.96:0/1052936130 shutdown_connections 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/1052936130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca58115a10 0x7fca58117e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/1052936130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/1052936130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca5810a150 0x7fca5810a530 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 -- 172.21.15.96:0/1052936130 >> 172.21.15.96:0/1052936130 conn(0x7fca5806dfc0 msgr2=0x7fca5806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 -- 172.21.15.96:0/1052936130 shutdown_connections 2024-09-10T22:55:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.755+0000 7fca5ce8a640 1 -- 172.21.15.96:0/1052936130 wait complete. 2024-09-10T22:55:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.756+0000 7fca5ce8a640 1 Processor -- start 2024-09-10T22:55:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.756+0000 7fca5ce8a640 1 -- start start 2024-09-10T22:55:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.757+0000 7fca5ce8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.757+0000 7fca5ce8a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5ce8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca58115a10 0x7fca581a7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42008/0 (socket says 172.21.15.96:42008) 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5659c640 1 -- 172.21.15.96:0/4255055847 learned_addr learned my addr 172.21.15.96:0/4255055847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811b550 con 0x7fca58115a10 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811b3d0 con 0x7fca5810a150 2024-09-10T22:55:43.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811b6d0 con 0x7fca5810aa70 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca56d9d640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca58115a10 0x7fca581a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 -- 172.21.15.96:0/4255055847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 msgr2=0x7fca581a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 -- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca58115a10 msgr2=0x7fca581a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca58115a10 0x7fca581a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca581a7b10 con 0x7fca5810aa70 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca5659c640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca55d9b640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581a1fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca40009b50 tx=0x7fca4000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.758+0000 7fca4f7fe640 1 -- 172.21.15.96:0/4255055847 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40009e50 con 0x7fca5810aa70 2024-09-10T22:55:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.759+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca581a7e00 con 0x7fca5810aa70 2024-09-10T22:55:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.759+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5810b260 con 0x7fca5810aa70 2024-09-10T22:55:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.759+0000 7fca4f7fe640 1 -- 172.21.15.96:0/4255055847 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400054d0 con 0x7fca5810aa70 2024-09-10T22:55:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.759+0000 7fca4f7fe640 1 -- 172.21.15.96:0/4255055847 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40019670 con 0x7fca5810aa70 2024-09-10T22:55:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.763+0000 7fca4f7fe640 1 -- 172.21.15.96:0/4255055847 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fca40004e80 con 0x7fca5810aa70 2024-09-10T22:55:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.764+0000 7fca4f7fe640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 0x7fca440b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.764+0000 7fca5659c640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 0x7fca440b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.764+0000 7fca4f7fe640 1 -- 172.21.15.96:0/4255055847 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca400e9600 con 0x7fca5810aa70 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5659c640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 0x7fca440b79c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca48031440 tx=0x7fca480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 Processor -- start 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 -- start start 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f2f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f304790 0x55560f306b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55560f26ee60 con 0x55560f304790 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f2f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f2f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36426/0 (socket says 172.21.15.96:36426) 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42020/0 (socket says 172.21.15.96:42020) 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5659c640 1 -- 172.21.15.96:0/3407604534 learned_addr learned my addr 172.21.15.96:0/3407604534 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55560f25e290 con 0x55560f2f8800 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55560f2620a0 con 0x55560f2f7ee0 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.765+0000 7fca56d9d640 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f304790 0x55560f306b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 -- 172.21.15.96:0/3407604534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 msgr2=0x55560f2f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f2f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 -- 172.21.15.96:0/3407604534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f304790 msgr2=0x55560f306b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f304790 0x55560f306b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 -- 172.21.15.96:0/3407604534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560efbc930 con 0x55560f2f8800 2024-09-10T22:55:43.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca55d9b640 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca400e71f0 tx=0x7fca40010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a234a839d577fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca4cff9640 1 -- 172.21.15.96:0/3407604534 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400afe60 con 0x55560f2f8800 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca4cff9640 1 -- 172.21.15.96:0/3407604534 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400b4940 con 0x55560f2f8800 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.766+0000 7fca4cff9640 1 -- 172.21.15.96:0/3407604534 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400970d0 con 0x55560f2f8800 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 msgr2=0x55560f304250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca400e71f0 tx=0x7fca40010860 comp rx=0 tx=0).stop 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 shutdown_connections 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f304250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f304790 0x55560f306b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3407604534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f2f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 >> 172.21.15.96:0/3407604534 conn(0x55560f2f5d10 msgr2=0x55560f2f6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 shutdown_connections 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.767+0000 7fca5f25f740 1 -- 172.21.15.96:0/3407604534 wait complete. 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 -- start start 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 Processor -- start 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f301c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f304790 0x55560f302190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55560f0bf890 con 0x55560f2f8800 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55560f1f8d40 con 0x55560f304790 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55560f307440 con 0x55560f2f7ee0 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f301c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f301c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49116/0 (socket says 172.21.15.96:49116) 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36430/0 (socket says 172.21.15.96:36430) 2024-09-10T22:55:43.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca55d9b640 1 -- 172.21.15.96:0/3204574379 learned_addr learned my addr 172.21.15.96:0/3204574379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca5f25f740 10 client.0 _ll_register_callbacks cb 0x55560f25cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.768+0000 7fca56d9d640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f304790 0x55560f302190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 -- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f304790 msgr2=0x55560f302190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f304790 0x55560f302190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 -- 172.21.15.96:0/3204574379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 msgr2=0x55560f301c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f301c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 -- 172.21.15.96:0/3204574379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560f3107f0 con 0x55560f2f7ee0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5659c640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca48005780 tx=0x7fca48009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=358e2f59bd557840 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48035040 con 0x55560f2f7ee0 2024-09-10T22:55:43.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca480038b0 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca400b4940 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca48003ab0 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5f25f740 10 client.26878 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55560f313160 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.769+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400b4940 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca4803ccc0 con 0x55560f2f7ee0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 10 client.26878 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 20 client.26878 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f314570 0x55560f3169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 4 client.26878 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55560f270db0 con 0x55560f314570 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca55d9b640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f314570 0x55560f3169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca3dffb640 10 client.26878 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:43.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.770+0000 7fca55d9b640 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f314570 0x55560f3169a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bbb30668df326 server_cookie=332ae4bd3d73cf10 in_seq=0 out_seq=0 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca3dffb640 1 -- 172.21.15.96:0/3204574379 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55560f270db0 con 0x55560f314570 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca3dffb640 10 client.26878 handle_command_reply: tid=0 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 client.26878 shutdown 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 msgr2=0x55560f301710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca48005780 tx=0x7fca48009260 comp rx=0 tx=0).stop 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 shutdown_connections 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f314570 0x55560f3169a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca3dffb640 0 client.26878 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f304790 0x55560f302190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f301c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 --2- 172.21.15.96:0/3204574379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f7ee0 0x55560f301710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 >> 172.21.15.96:0/3204574379 conn(0x55560f2f5d10 msgr2=0x55560f2f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 shutdown_connections 2024-09-10T22:55:43.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.771+0000 7fca5f25f740 1 -- 172.21.15.96:0/3204574379 wait complete. 2024-09-10T22:55:43.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.772+0000 7fca5f25f740 20 client.26878 trim_cache size 0 max 16384 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.796+0000 7fca5f25f740 1 Processor -- start 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.796+0000 7fca5f25f740 1 -- start start 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.796+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f2ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.796+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.796+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f31c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca5f25f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b47c0 con 0x55560f2f8800 2024-09-10T22:55:43.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca5659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f2ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca5f25f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b6300 con 0x55560f2f7ee0 2024-09-10T22:55:43.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca56d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f31c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca5f25f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b4e90 con 0x55560f2f5d10 2024-09-10T22:55:43.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42042/0 (socket says 172.21.15.96:42042) 2024-09-10T22:55:43.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 -- 172.21.15.96:0/1735518307 learned_addr learned my addr 172.21.15.96:0/1735518307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 -- 172.21.15.96:0/1735518307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 msgr2=0x55560f2ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f2ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 -- 172.21.15.96:0/1735518307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 msgr2=0x55560f31c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f31c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 -- 172.21.15.96:0/1735518307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560f2f6790 con 0x55560f2f7ee0 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca56d9d640 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f31c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.797+0000 7fca55d9b640 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca400afdf0 tx=0x7fca400e9800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b73e02b9b366311 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.798+0000 7fca3ffff640 1 -- 172.21.15.96:0/1735518307 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400af970 con 0x55560f2f7ee0 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.798+0000 7fca3ffff640 1 -- 172.21.15.96:0/1735518307 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400b4940 con 0x55560f2f7ee0 2024-09-10T22:55:43.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.798+0000 7fca3ffff640 1 -- 172.21.15.96:0/1735518307 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400970d0 con 0x55560f2f7ee0 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.798+0000 7fca5f25f740 1 -- 172.21.15.96:0/1735518307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 msgr2=0x55560f31c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca400afdf0 tx=0x7fca400e9800 comp rx=0 tx=0).stop 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 -- 172.21.15.96:0/1735518307 shutdown_connections 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f7ee0 0x55560f31c270 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f8800 0x55560f31c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 --2- 172.21.15.96:0/1735518307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f2ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.799+0000 7fca5f25f740 1 -- 172.21.15.96:0/1735518307 >> 172.21.15.96:0/1735518307 conn(0x55560f2fd0b0 msgr2=0x55560f2feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.801+0000 7fca5f25f740 1 -- 172.21.15.96:0/1735518307 shutdown_connections 2024-09-10T22:55:43.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.801+0000 7fca5f25f740 1 -- 172.21.15.96:0/1735518307 wait complete. 2024-09-10T22:55:43.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.801+0000 7fca5f25f740 1 -- start start 2024-09-10T22:55:43.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.801+0000 7fca5f25f740 1 Processor -- start 2024-09-10T22:55:43.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f30ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 0x55560f30a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b5c30 con 0x55560f2f7ee0 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b5560 con 0x55560f2f8800 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b2800 con 0x55560f2f5d10 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 0x55560f30a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42056/0 (socket says 172.21.15.96:42056) 2024-09-10T22:55:43.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca55d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 0x55560f30a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49144/0 (socket says 172.21.15.96:49144) 2024-09-10T22:55:43.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 -- 172.21.15.96:0/578312257 learned_addr learned my addr 172.21.15.96:0/578312257 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:43.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5659c640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f30ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca5f25f740 10 client.0 _ll_register_callbacks cb 0x7fca40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:43.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 -- 172.21.15.96:0/578312257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 msgr2=0x55560f30ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f30ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 -- 172.21.15.96:0/578312257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 msgr2=0x55560f30a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 0x55560f30a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.802+0000 7fca56d9d640 1 -- 172.21.15.96:0/578312257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560f30b680 con 0x55560f2f8800 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca56d9d640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca50010dd0 tx=0x7fca5000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca695ae56bfb16f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:43.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50022060 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca50003b70 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca400b4940 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca5000fe60 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 10 client.26813 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55560f382510 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400b4940 con 0x55560f2f8800 2024-09-10T22:55:43.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca50027070 con 0x55560f2f8800 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 10 client.26813 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.803+0000 7fca5f25f740 20 client.26813 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f304790 0x55560f384d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca5f25f740 4 client.26813 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55560f26c860 con 0x55560f304790 2024-09-10T22:55:43.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca5659c640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f304790 0x55560f384d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:43.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca5659c640 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f304790 0x55560f384d00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74475a1a1ff0212 server_cookie=be2fb7496499b05 in_seq=0 out_seq=0 2024-09-10T22:55:43.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.804+0000 7fca3dffb640 10 client.26813 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:43.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.805+0000 7fca3dffb640 1 -- 172.21.15.96:0/578312257 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55560f26c860 con 0x55560f304790 2024-09-10T22:55:43.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.805+0000 7fca3dffb640 10 client.26813 handle_command_reply: tid=0 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 client.26813 shutdown 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 msgr2=0x55560f30ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca50010dd0 tx=0x7fca5000f790 comp rx=0 tx=0).stop 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 shutdown_connections 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55560f304790 0x55560f384d00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55560f2f8800 0x55560f30ae70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55560f2f7ee0 0x55560f30a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca3dffb640 0 client.26813 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 --2- 172.21.15.96:0/578312257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55560f2f5d10 0x55560f30ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 >> 172.21.15.96:0/578312257 conn(0x55560f2fd0b0 msgr2=0x55560f2f4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 shutdown_connections 2024-09-10T22:55:43.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 1 -- 172.21.15.96:0/578312257 wait complete. 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.806+0000 7fca5f25f740 20 client.26813 trim_cache size 0 max 16384 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "bd68d588-6aa2-48ab-be58-244974ea644c", 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:55:43.816 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:55:43.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 msgr2=0x7fca440b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 0x7fca440b79c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca48031440 tx=0x7fca480058d0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 msgr2=0x7fca581a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581a1fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca40009b50 tx=0x7fca4000b690 comp rx=0 tx=0).stop 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 shutdown_connections 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca440b5500 0x7fca440b79c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca58115a10 0x7fca581a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca5810aa70 0x7fca581a1fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 --2- 172.21.15.96:0/4255055847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca5810a150 0x7fca581a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 >> 172.21.15.96:0/4255055847 conn(0x7fca5806dfc0 msgr2=0x7fca580729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 shutdown_connections 2024-09-10T22:55:43.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:43.807+0000 7fca5ce8a640 1 -- 172.21.15.96:0/4255055847 wait complete. 2024-09-10T22:55:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.431+0000 7fb8927c6640 1 -- 172.21.15.96:0/3044573170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 msgr2=0x7fb88c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.431+0000 7fb8927c6640 1 --2- 172.21.15.96:0/3044573170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c103f60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb874009930 tx=0x7fb87402a420 comp rx=0 tx=0).stop 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 -- 172.21.15.96:0/3044573170 shutdown_connections 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- 172.21.15.96:0/3044573170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c1044a0 0x7fb88c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- 172.21.15.96:0/3044573170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c103f60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- 172.21.15.96:0/3044573170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c102990 0x7fb88c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 -- 172.21.15.96:0/3044573170 >> 172.21.15.96:0/3044573170 conn(0x7fb88c0fe250 msgr2=0x7fb88c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 -- 172.21.15.96:0/3044573170 shutdown_connections 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 -- 172.21.15.96:0/3044573170 wait complete. 2024-09-10T22:55:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 Processor -- start 2024-09-10T22:55:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 -- start start 2024-09-10T22:55:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 0x7fb88c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.432+0000 7fb8927c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c1044a0 0x7fb88c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb8927c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10e630 con 0x7fb88c103ae0 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb8927c6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10e4b0 con 0x7fb88c1044a0 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb8927c6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10e7b0 con 0x7fb88c102990 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49162/0 (socket says 172.21.15.96:49162) 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 -- 172.21.15.96:0/904883723 learned_addr learned my addr 172.21.15.96:0/904883723 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb88bfff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 0x7fb88c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb890d63640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c1044a0 0x7fb88c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 -- 172.21.15.96:0/904883723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 msgr2=0x7fb88c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 0x7fb88c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 -- 172.21.15.96:0/904883723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c1044a0 msgr2=0x7fb88c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c1044a0 0x7fb88c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 -- 172.21.15.96:0/904883723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c19a8f0 con 0x7fb88c103ae0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb88bfff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 0x7fb88c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb883fff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb874003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb889ffb640 1 -- 172.21.15.96:0/904883723 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87403f070 con 0x7fb88c103ae0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c19ab80 con 0x7fb88c103ae0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb889ffb640 1 -- 172.21.15.96:0/904883723 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb874031c50 con 0x7fb88c103ae0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c1a12b0 con 0x7fb88c103ae0 2024-09-10T22:55:44.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.433+0000 7fb889ffb640 1 -- 172.21.15.96:0/904883723 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874049060 con 0x7fb88c103ae0 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb889ffb640 1 -- 172.21.15.96:0/904883723 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb8740317a0 con 0x7fb88c103ae0 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 Processor -- start 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 -- start start 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x556281517580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562814846f0 con 0x556281512860 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556281473b20 con 0x556281517bd0 2024-09-10T22:55:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb89566e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556281477930 con 0x556281515190 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49172/0 (socket says 172.21.15.96:49172) 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 -- 172.21.15.96:0/1855612891 learned_addr learned my addr 172.21.15.96:0/1855612891 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb883fff640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb88bfff640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x556281517580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 -- 172.21.15.96:0/1855612891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 msgr2=0x556281517580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x556281517580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 -- 172.21.15.96:0/1855612891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 msgr2=0x55628151a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb890d63640 1 -- 172.21.15.96:0/1855612891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556281439530 con 0x556281512860 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.436+0000 7fb88bfff640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x556281517580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb890d63640 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb884009f90 tx=0x7fb8840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4be7cfe1909d831 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb889ffb640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 0x7fb8780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb8827fc640 1 -- 172.21.15.96:0/1855612891 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884004d20 con 0x556281512860 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb88bfff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 0x7fb8780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb8827fc640 1 -- 172.21.15.96:0/1855612891 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb884004050 con 0x556281512860 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb8827fc640 1 -- 172.21.15.96:0/1855612891 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884005240 con 0x556281512860 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb889ffb640 1 -- 172.21.15.96:0/904883723 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb874106490 con 0x7fb88c103ae0 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb89566e740 1 -- 172.21.15.96:0/1855612891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 msgr2=0x556281514c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb88bfff640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 0x7fb8780b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009fd0 tx=0x7fb87c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.437+0000 7fb89566e740 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb884009f90 tx=0x7fb8840118d0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- 172.21.15.96:0/1855612891 shutdown_connections 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x556281517580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- 172.21.15.96:0/1855612891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x556281514c50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- 172.21.15.96:0/1855612891 >> 172.21.15.96:0/1855612891 conn(0x55628150d9b0 msgr2=0x55628150e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- 172.21.15.96:0/1855612891 shutdown_connections 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- 172.21.15.96:0/1855612891 wait complete. 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- start start 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 Processor -- start 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x55628151bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x55628151c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562813abab0 con 0x556281512860 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556281419730 con 0x556281517bd0 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55628151abb0 con 0x556281515190 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb89566e740 10 client.0 _ll_register_callbacks cb 0x55628146d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.438+0000 7fb88bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x55628151bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x55628151c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb88bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x55628151bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49186/0 (socket says 172.21.15.96:49186) 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb88bfff640 1 -- 172.21.15.96:0/2376182297 learned_addr learned my addr 172.21.15.96:0/2376182297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb890d63640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 -- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 msgr2=0x55628151d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 -- 172.21.15.96:0/2376182297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 msgr2=0x55628151bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x55628151bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 -- 172.21.15.96:0/2376182297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562811d5790 con 0x556281515190 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb883fff640 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x55628151c540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb874104080 tx=0x7fb87403b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6bd2f4bc1e6aaab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb86b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740c93f0 con 0x556281515190 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb8741060d0 con 0x556281515190 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb884004050 con 0x556281515190 2024-09-10T22:55:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8740cc510 con 0x556281515190 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb89566e740 10 client.26896 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.439+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55628124e620 con 0x556281515190 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb884004050 con 0x556281515190 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb87403bbd0 con 0x556281515190 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 10 client.26896 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 10 client.26896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 20 client.26896 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281527770 0x556281529b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 4 client.26896 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556281486640 con 0x556281527770 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb88bfff640 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281527770 0x556281529b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb86b7fe640 10 client.26896 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.440+0000 7fb88bfff640 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281527770 0x556281529b60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3587dfd800d2f6a5 server_cookie=bdf80d4fd886992d in_seq=0 out_seq=0 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb86b7fe640 1 -- 172.21.15.96:0/2376182297 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556281486640 con 0x556281527770 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb86b7fe640 10 client.26896 handle_command_reply: tid=0 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 client.26896 shutdown 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 msgr2=0x55628151c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x55628151c540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb874104080 tx=0x7fb87403b690 comp rx=0 tx=0).stop 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 shutdown_connections 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281527770 0x556281529b60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x55628151d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281515190 0x55628151c540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 --2- 172.21.15.96:0/2376182297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281512860 0x55628151bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 >> 172.21.15.96:0/2376182297 conn(0x55628150d9b0 msgr2=0x55628150cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb86b7fe640 0 client.26896 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 shutdown_connections 2024-09-10T22:55:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 1 -- 172.21.15.96:0/2376182297 wait complete. 2024-09-10T22:55:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.443+0000 7fb89566e740 20 client.26896 trim_cache size 0 max 16384 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.458+0000 7fb89566e740 1 Processor -- start 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.458+0000 7fb89566e740 1 -- start start 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281512860 0x55628151c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 0x556281529780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1a9e30 con 0x556281515190 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1ab970 con 0x556281512860 2024-09-10T22:55:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb89566e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1aa500 con 0x556281517bd0 2024-09-10T22:55:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49196/0 (socket says 172.21.15.96:49196) 2024-09-10T22:55:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb890d63640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 0x556281529780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 -- 172.21.15.96:0/2354496449 learned_addr learned my addr 172.21.15.96:0/2354496449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb88bfff640 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281512860 0x55628151c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 -- 172.21.15.96:0/2354496449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 msgr2=0x556281529780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 0x556281529780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 -- 172.21.15.96:0/2354496449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281512860 msgr2=0x55628151c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281512860 0x55628151c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 -- 172.21.15.96:0/2354496449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556281514fc0 con 0x556281515190 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb890d63640 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 0x556281529780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb883fff640 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb87403bfb0 tx=0x7fb874005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337e8acba39af971 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb881ffb640 1 -- 172.21.15.96:0/2354496449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740cc6f0 con 0x556281515190 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb881ffb640 1 -- 172.21.15.96:0/2354496449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb874031e00 con 0x556281515190 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.459+0000 7fb881ffb640 1 -- 172.21.15.96:0/2354496449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740ccb70 con 0x556281515190 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 -- 172.21.15.96:0/2354496449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 msgr2=0x55628150e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb87403bfb0 tx=0x7fb874005d50 comp rx=0 tx=0).stop 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 -- 172.21.15.96:0/2354496449 shutdown_connections 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281517bd0 0x556281529780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x55628150e670 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 --2- 172.21.15.96:0/2354496449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281512860 0x55628151c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 -- 172.21.15.96:0/2354496449 >> 172.21.15.96:0/2354496449 conn(0x55628150d9b0 msgr2=0x55628151d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 -- 172.21.15.96:0/2354496449 shutdown_connections 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.460+0000 7fb89566e740 1 -- 172.21.15.96:0/2354496449 wait complete. 2024-09-10T22:55:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 -- start start 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 Processor -- start 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281512860 0x5562815231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x556281523c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1ab2a0 con 0x556281515190 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1aabd0 con 0x556281517bd0 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c1a7e30 con 0x556281512860 2024-09-10T22:55:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb89566e740 10 client.0 _ll_register_callbacks cb 0x7fb874009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49212/0 (socket says 172.21.15.96:49212) 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 -- 172.21.15.96:0/950252882 learned_addr learned my addr 172.21.15.96:0/950252882 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb890d63640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x556281523c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb88bfff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281512860 0x5562815231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 -- 172.21.15.96:0/950252882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281512860 msgr2=0x5562815231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281512860 0x5562815231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 -- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 msgr2=0x556281523c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.461+0000 7fb883fff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x556281523c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb883fff640 1 -- 172.21.15.96:0/950252882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562815928b0 con 0x556281515190 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb883fff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b8230 tx=0x7fb87403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c952910b721a2ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:55:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740c8c40 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb8740b2bc0 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562814820f0 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8740ccb70 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 10 client.17253 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562815928b0 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5562814820f0 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb8740cc120 con 0x556281515190 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 10 client.17253 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 20 client.17253 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281594100 0x556281596520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 4 client.17253 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.462+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55628147f520 con 0x556281594100 2024-09-10T22:55:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.463+0000 7fb88bfff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281594100 0x556281596520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.463+0000 7fb86b7fe640 10 client.17253 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.463+0000 7fb88bfff640 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281594100 0x556281596520 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa006ae07ad398f8 server_cookie=6ce5f5e1274f3ce in_seq=0 out_seq=0 2024-09-10T22:55:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.591+0000 7fb86b7fe640 1 -- 172.21.15.96:0/950252882 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55628147f520 con 0x556281594100 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.591+0000 7fb86b7fe640 10 client.17253 handle_command_reply: tid=0 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.591+0000 7fb89566e740 1 client.17253 shutdown 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 msgr2=0x556281523710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b8230 tx=0x7fb87403b720 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 shutdown_connections 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556281594100 0x556281596520 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb86b7fe640 0 client.17253 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556281517bd0 0x556281523c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556281515190 0x556281523710 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 --2- 172.21.15.96:0/950252882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556281512860 0x5562815231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 >> 172.21.15.96:0/950252882 conn(0x55628150d9b0 msgr2=0x556281529780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 shutdown_connections 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 1 -- 172.21.15.96:0/950252882 wait complete. 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.592+0000 7fb89566e740 20 client.17253 trim_cache size 0 max 16384 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:55:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 msgr2=0x7fb8780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 0x7fb8780b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009fd0 tx=0x7fb87c005f90 comp rx=0 tx=0).stop 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 msgr2=0x7fb88c199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb874003c10 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 shutdown_connections 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb88c1044a0 0x7fb88c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb8780b52e0 0x7fb8780b77a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c103ae0 0x7fb88c199400 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 --2- 172.21.15.96:0/904883723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88c102990 0x7fb88c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 >> 172.21.15.96:0/904883723 conn(0x7fb88c0fe250 msgr2=0x7fb88c106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 shutdown_connections 2024-09-10T22:55:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:44.593+0000 7fb8927c6640 1 -- 172.21.15.96:0/904883723 wait complete. 2024-09-10T22:55:44.641 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:55:45.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-10T22:55:45.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:55:45.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:55:45.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:55:45.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:44 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-10T22:55:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:55:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:55:45.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:55:45.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:44 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:55:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:44 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.028 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:55:45.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 -- 172.21.15.96:0/2709626498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 msgr2=0x7fd1fc10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 --2- 172.21.15.96:0/2709626498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc10f930 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009ca0 tx=0x7fd1f402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 -- 172.21.15.96:0/2709626498 shutdown_connections 2024-09-10T22:55:45.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 --2- 172.21.15.96:0/2709626498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc10f930 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 --2- 172.21.15.96:0/2709626498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 --2- 172.21.15.96:0/2709626498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.555+0000 7fd20114e640 1 -- 172.21.15.96:0/2709626498 >> 172.21.15.96:0/2709626498 conn(0x7fd1fc0fe850 msgr2=0x7fd1fc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.556+0000 7fd20114e640 1 -- 172.21.15.96:0/2709626498 shutdown_connections 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.556+0000 7fd20114e640 1 -- 172.21.15.96:0/2709626498 wait complete. 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.557+0000 7fd20114e640 1 Processor -- start 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.557+0000 7fd20114e640 1 -- start start 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc10ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc1b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc112df0 con 0x7fd1fc109130 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc112c70 con 0x7fd1fc102da0 2024-09-10T22:55:45.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd20114e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc112f70 con 0x7fd1fc1087b0 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36538/0 (socket says 172.21.15.96:36538) 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 -- 172.21.15.96:0/1632629399 learned_addr learned my addr 172.21.15.96:0/1632629399 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc10ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc1b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 -- 172.21.15.96:0/1632629399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 msgr2=0x7fd1fc10ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc10ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 -- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 msgr2=0x7fd1fc1b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc1b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fa59c640 1 -- 172.21.15.96:0/1632629399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc1b4210 con 0x7fd1fc1087b0 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.558+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc10ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:45.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ea60 tx=0x7fd1f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd1e3fff640 1 -- 172.21.15.96:0/1632629399 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f000cde0 con 0x7fd1fc1087b0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd1e3fff640 1 -- 172.21.15.96:0/1632629399 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f00035f0 con 0x7fd1fc1087b0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd1e3fff640 1 -- 172.21.15.96:0/1632629399 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0010690 con 0x7fd1fc1087b0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc1b4500 con 0x7fd1fc1087b0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.559+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc1b49f0 con 0x7fd1fc1087b0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 Processor -- start 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 -- start start 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d09860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c086c7fe50 con 0x55c086d15d30 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c086c6f280 con 0x55c086d09da0 2024-09-10T22:55:45.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.562+0000 7fd203523740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c086c73090 con 0x55c086d09480 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42128/0 (socket says 172.21.15.96:42128) 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1924906561 learned_addr learned my addr 172.21.15.96:0/1924906561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d09860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1e3fff640 1 -- 172.21.15.96:0/1632629399 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd1f0004000 con 0x7fd1fc1087b0 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1924906561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 msgr2=0x55c086d09860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d09860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1924906561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 msgr2=0x55c086d18120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1924906561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0869cd650 con 0x55c086d09da0 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0115f0 tx=0x7fd1ec011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3992192ce71e0c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.563+0000 7fd1e17fa640 1 -- 172.21.15.96:0/1924906561 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec01e070 con 0x55c086d09da0 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.564+0000 7fd1e17fa640 1 -- 172.21.15.96:0/1924906561 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ec00faf0 con 0x55c086d09da0 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.564+0000 7fd1e17fa640 1 -- 172.21.15.96:0/1924906561 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec019b10 con 0x55c086d09da0 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.565+0000 7fd203523740 1 -- 172.21.15.96:0/1924906561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 msgr2=0x55c086d157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.565+0000 7fd203523740 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0115f0 tx=0x7fd1ec011ab0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.565+0000 7fd1e3fff640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 0x7fd1e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 -- 172.21.15.96:0/1924906561 shutdown_connections 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d157f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 --2- 172.21.15.96:0/1924906561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d09860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 -- 172.21.15.96:0/1924906561 >> 172.21.15.96:0/1924906561 conn(0x55c086d072b0 msgr2=0x55c086d08450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd1e3fff640 1 -- 172.21.15.96:0/1632629399 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1f00e5e10 con 0x7fd1fc1087b0 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 0x7fd1e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 -- 172.21.15.96:0/1924906561 shutdown_connections 2024-09-10T22:55:45.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.566+0000 7fd203523740 1 -- 172.21.15.96:0/1924906561 wait complete. 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 -- start start 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 Processor -- start 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 0x7fd1e40b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006010 tx=0x7fd1f40098e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09480 0x55c086d156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c086ad0700 con 0x55c086d15d30 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c086c0a110 con 0x55c086d09480 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd203523740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c086d189b0 con 0x55c086d09da0 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36560/0 (socket says 172.21.15.96:36560) 2024-09-10T22:55:45.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09480 0x55c086d156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.567+0000 7fd1fa59c640 1 -- 172.21.15.96:0/4145719604 learned_addr learned my addr 172.21.15.96:0/4145719604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd203523740 10 client.0 _ll_register_callbacks cb 0x55c086c23990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 -- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09480 msgr2=0x55c086d156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09480 0x55c086d156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 -- 172.21.15.96:0/4145719604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 msgr2=0x55c086d131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 -- 172.21.15.96:0/4145719604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c086c37c40 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00ac370 tx=0x7fd1f00abde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe683bb260b17031 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1deff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00ac670 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec00faf0 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1f00037a0 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1f00b1150 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd203523740 10 client.26914 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c086d243a0 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.568+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ec00faf0 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd1f00c5020 con 0x55c086d09da0 2024-09-10T22:55:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 10 client.26914 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 20 client.26914 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d257b0 0x55c086d27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 4 client.26914 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c086c81da0 con 0x55c086d257b0 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d257b0 0x55c086d27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d257b0 0x55c086d27ba0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f43e2afa6a0ad22 server_cookie=eb460de0db5eae8 in_seq=0 out_seq=0 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.569+0000 7fd1deff5640 10 client.26914 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd1deff5640 1 -- 172.21.15.96:0/4145719604 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c086c81da0 con 0x55c086d257b0 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd1deff5640 10 client.26914 handle_command_reply: tid=0 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 client.26914 shutdown 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 msgr2=0x55c086d12b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00ac370 tx=0x7fd1f00abde0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 shutdown_connections 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d257b0 0x55c086d27ba0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d12b20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 --2- 172.21.15.96:0/4145719604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09480 0x55c086d156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.571+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 >> 172.21.15.96:0/4145719604 conn(0x55c086d072b0 msgr2=0x55c086d0ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.572+0000 7fd1deff5640 0 client.26914 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:45.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.572+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 shutdown_connections 2024-09-10T22:55:45.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.572+0000 7fd203523740 1 -- 172.21.15.96:0/4145719604 wait complete. 2024-09-10T22:55:45.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.572+0000 7fd203523740 20 client.26914 trim_cache size 0 max 16384 2024-09-10T22:55:45.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.602+0000 7fd203523740 1 Processor -- start 2024-09-10T22:55:45.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.602+0000 7fd203523740 1 -- start start 2024-09-10T22:55:45.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d15d30 0x55c086d0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49250/0 (socket says 172.21.15.96:49250) 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1bcfb0 con 0x55c086d09480 2024-09-10T22:55:45.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1beaf0 con 0x55c086d15d30 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd203523740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1bd680 con 0x55c086d09da0 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd1fa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd1fad9d640 1 -- 172.21.15.96:0/3911530454 learned_addr learned my addr 172.21.15.96:0/3911530454 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.603+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d15d30 0x55c086d0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 -- 172.21.15.96:0/3911530454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 msgr2=0x55c086d062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 -- 172.21.15.96:0/3911530454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d15d30 msgr2=0x55c086d0fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d15d30 0x55c086d0fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 -- 172.21.15.96:0/3911530454 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c086d12f50 con 0x55c086d09480 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.604+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0115f0 tx=0x7fd1ec019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ea1b8b55c6d09b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.607+0000 7fd1e0ff9640 1 -- 172.21.15.96:0/3911530454 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec053070 con 0x55c086d09480 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.607+0000 7fd1e0ff9640 1 -- 172.21.15.96:0/3911530454 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ec029070 con 0x55c086d09480 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.607+0000 7fd1e0ff9640 1 -- 172.21.15.96:0/3911530454 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec01e040 con 0x55c086d09480 2024-09-10T22:55:45.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.608+0000 7fd203523740 1 -- 172.21.15.96:0/3911530454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 msgr2=0x55c086d0d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.608+0000 7fd203523740 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0115f0 tx=0x7fd1ec019670 comp rx=0 tx=0).stop 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.613+0000 7fd203523740 1 -- 172.21.15.96:0/3911530454 shutdown_connections 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.613+0000 7fd203523740 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d15d30 0x55c086d0fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.613+0000 7fd203523740 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09da0 0x55c086d062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.613+0000 7fd203523740 1 --2- 172.21.15.96:0/3911530454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d09480 0x55c086d0d3c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.613+0000 7fd203523740 1 -- 172.21.15.96:0/3911530454 >> 172.21.15.96:0/3911530454 conn(0x55c086d072b0 msgr2=0x55c086d05bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.620+0000 7fd203523740 1 -- 172.21.15.96:0/3911530454 shutdown_connections 2024-09-10T22:55:45.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.622+0000 7fd203523740 1 -- 172.21.15.96:0/3911530454 wait complete. 2024-09-10T22:55:45.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 -- start start 2024-09-10T22:55:45.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 Processor -- start 2024-09-10T22:55:45.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1be420 con 0x55c086d15d30 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1bdd50 con 0x55c086d09da0 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1baf90 con 0x55c086d09480 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.623+0000 7fd203523740 10 client.0 _ll_register_callbacks cb 0x7fd1f0004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.627+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.627+0000 7fd1fad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:36588/0 (socket says 172.21.15.96:36588) 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.627+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1510837715 learned_addr learned my addr 172.21.15.96:0/1510837715 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.627+0000 7fd1fb59e640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.627+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.631+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 msgr2=0x55c086d1e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.631+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d1e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.631+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1510837715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 msgr2=0x55c086d18c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:45.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.631+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.631+0000 7fd1fad9d640 1 -- 172.21.15.96:0/1510837715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c086d1e9c0 con 0x55c086d09480 2024-09-10T22:55:45.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.632+0000 7fd1fad9d640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009380 tx=0x7fd1ec005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67675537c47bbada server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:45.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.633+0000 7fd1deff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:45.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.635+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec05c060 con 0x55c086d09480 2024-09-10T22:55:45.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.635+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec029070 con 0x55c086d09480 2024-09-10T22:55:45.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.635+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1ec00fd20 con 0x55c086d09480 2024-09-10T22:55:45.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.641+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1ec053070 con 0x55c086d09480 2024-09-10T22:55:45.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.643+0000 7fd203523740 10 client.26920 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:45.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.643+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c086d92c30 con 0x55c086d09480 2024-09-10T22:55:45.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.648+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1ec029070 con 0x55c086d09480 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd1ec0196e0 con 0x55c086d09480 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 10 client.26920 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 10 client.26920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 20 client.26920 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d930b0 0x55c086d954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 4 client.26920 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:55:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.649+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c086c7d850 con 0x55c086d930b0 2024-09-10T22:55:45.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.651+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d930b0 0x55c086d954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.652+0000 7fd1deff5640 10 client.26920 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.652+0000 7fd1fa59c640 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d930b0 0x55c086d954a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb9854bdcd92a55 server_cookie=a2257cfa0dccc1c7 in_seq=0 out_seq=0 2024-09-10T22:55:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.653+0000 7fd1deff5640 1 -- 172.21.15.96:0/1510837715 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55c086c7d850 con 0x55c086d930b0 2024-09-10T22:55:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.653+0000 7fd1deff5640 10 client.26920 handle_command_reply: tid=0 2024-09-10T22:55:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.653+0000 7fd203523740 1 client.26920 shutdown 2024-09-10T22:55:45.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.654+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 msgr2=0x55c086d1dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.654+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009380 tx=0x7fd1ec005f10 comp rx=0 tx=0).stop 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 shutdown_connections 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c086d930b0 0x55c086d954a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c086d09da0 0x55c086d1e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c086d15d30 0x55c086d18c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 --2- 172.21.15.96:0/1510837715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c086d09480 0x55c086d1dc50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 >> 172.21.15.96:0/1510837715 conn(0x55c086d072b0 msgr2=0x55c086d063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd1deff5640 0 client.26920 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:45.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.655+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 shutdown_connections 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "bd68d588-6aa2-48ab-be58-244974ea644c", 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:55:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:55:45.666 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.656+0000 7fd203523740 1 -- 172.21.15.96:0/1510837715 wait complete. 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.656+0000 7fd203523740 20 client.26920 trim_cache size 0 max 16384 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 msgr2=0x7fd1e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 0x7fd1e40b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4006010 tx=0x7fd1f40098e0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 msgr2=0x7fd1fc10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000ea60 tx=0x7fd1f000ef20 comp rx=0 tx=0).stop 2024-09-10T22:55:45.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 shutdown_connections 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1e40b52e0 0x7fd1e40b77a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1fc109130 0x7fd1fc1b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc1087b0 0x7fd1fc10f120 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 --2- 172.21.15.96:0/1632629399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1fc102da0 0x7fd1fc10ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 >> 172.21.15.96:0/1632629399 conn(0x7fd1fc0fe850 msgr2=0x7fd1fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.657+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 shutdown_connections 2024-09-10T22:55:45.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:55:45.658+0000 7fd20114e640 1 -- 172.21.15.96:0/1632629399 wait complete. 2024-09-10T22:55:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:45 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 490.0 s, 2359.2 tps, lat 42.333 ms stddev 49.111 2024-09-10T22:55:46.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:45 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:46.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:45 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:55:46.578 INFO:tasks.cephfs.filesystem:scrub status for tag:bd68d588-6aa2-48ab-be58-244974ea644c - {'path': '/', 'tag': 'bd68d588-6aa2-48ab-be58-244974ea644c', 'options': 'recursive,force'} 2024-09-10T22:55:47.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:46 smithi096 ceph-mon[30869]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 692 op/s 2024-09-10T22:55:47.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:46 smithi121 ceph-mon[35119]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 692 op/s 2024-09-10T22:55:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:46 smithi117 ceph-mon[35081]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.5 MiB/s wr, 692 op/s 2024-09-10T22:55:49.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:49 smithi121 ceph-mon[35119]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-09-10T22:55:49.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:49 smithi117 ceph-mon[35081]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-09-10T22:55:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:49 smithi096 ceph-mon[30869]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.1 MiB/s wr, 566 op/s 2024-09-10T22:55:50.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:50 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:55:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:50 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:55:50.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:50 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:55:50.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:50 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:55:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:50 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:55:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:50 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:55:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 495.0 s, 2623.0 tps, lat 37.792 ms stddev 41.283 2024-09-10T22:55:51.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:51 smithi121 ceph-mon[35119]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-09-10T22:55:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:51 smithi117 ceph-mon[35081]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-09-10T22:55:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:51 smithi096 ceph-mon[30869]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.3 MiB/s wr, 842 op/s 2024-09-10T22:55:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:53 smithi117 ceph-mon[35081]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.6 MiB/s wr, 751 op/s 2024-09-10T22:55:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:53 smithi096 ceph-mon[30869]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.6 MiB/s wr, 751 op/s 2024-09-10T22:55:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:53 smithi121 ceph-mon[35119]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.6 MiB/s wr, 751 op/s 2024-09-10T22:55:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:55 smithi117 ceph-mon[35081]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.3 MiB/s wr, 836 op/s 2024-09-10T22:55:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:55 smithi096 ceph-mon[30869]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.3 MiB/s wr, 836 op/s 2024-09-10T22:55:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:55 smithi121 ceph-mon[35119]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.3 MiB/s wr, 836 op/s 2024-09-10T22:55:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 500.0 s, 2693.8 tps, lat 37.436 ms stddev 42.107 2024-09-10T22:55:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:57 smithi117 ceph-mon[35081]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.4 MiB/s wr, 808 op/s 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 -- 172.21.15.96:0/2915267958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc103fa0 msgr2=0x7f30cc1043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 --2- 172.21.15.96:0/2915267958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc103fa0 0x7f30cc1043a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009930 tx=0x7f30bc02a420 comp rx=0 tx=0).stop 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 -- 172.21.15.96:0/2915267958 shutdown_connections 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 --2- 172.21.15.96:0/2915267958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc101130 0x7f30cc10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 --2- 172.21.15.96:0/2915267958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 --2- 172.21.15.96:0/2915267958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc103fa0 0x7f30cc1043a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 -- 172.21.15.96:0/2915267958 >> 172.21.15.96:0/2915267958 conn(0x7f30cc0fc880 msgr2=0x7f30cc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 -- 172.21.15.96:0/2915267958 shutdown_connections 2024-09-10T22:55:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.705+0000 7f30d19fb640 1 -- 172.21.15.96:0/2915267958 wait complete. 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 Processor -- start 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 -- start start 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc103fa0 0x7f30cc199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc10e5f0 con 0x7f30cc101130 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc10e470 con 0x7f30cc103fa0 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.706+0000 7f30d19fb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc10e770 con 0x7f30cc1048e0 2024-09-10T22:55:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37708/0 (socket says 172.21.15.96:37708) 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30ca7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc103fa0 0x7f30cc199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 -- 172.21.15.96:0/3055076014 learned_addr learned my addr 172.21.15.96:0/3055076014 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 -- 172.21.15.96:0/3055076014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 msgr2=0x7f30cc19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 -- 172.21.15.96:0/3055076014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc103fa0 msgr2=0x7f30cc199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc103fa0 0x7f30cc199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 -- 172.21.15.96:0/3055076014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc19a930 con 0x7f30cc101130 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30caffd640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009a50 tx=0x7f30bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30d09f9640 1 -- 172.21.15.96:0/3055076014 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc03f030 con 0x7f30cc101130 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.707+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc19abc0 con 0x7f30cc101130 2024-09-10T22:55:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.708+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a12f0 con 0x7f30cc101130 2024-09-10T22:55:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.708+0000 7f30d09f9640 1 -- 172.21.15.96:0/3055076014 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30bc031910 con 0x7f30cc101130 2024-09-10T22:55:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.708+0000 7f30d09f9640 1 -- 172.21.15.96:0/3055076014 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc035070 con 0x7f30cc101130 2024-09-10T22:55:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 Processor -- start 2024-09-10T22:55:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d09f9640 1 -- 172.21.15.96:0/3055076014 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f30bc03ba60 con 0x7f30cc101130 2024-09-10T22:55:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 -- start start 2024-09-10T22:55:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e97892580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 0x560e97895090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e977ff6f0 con 0x560e9788d860 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e977eeb20 con 0x560e97892bd0 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30d47c8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560e977f2930 con 0x560e97890190 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37722/0 (socket says 172.21.15.96:37722) 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30cb7fe640 1 -- 172.21.15.96:0/591459474 learned_addr learned my addr 172.21.15.96:0/591459474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 0x560e97895090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.712+0000 7f30caffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e97892580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 -- 172.21.15.96:0/591459474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 msgr2=0x560e97892580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e97892580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 -- 172.21.15.96:0/591459474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 msgr2=0x560e97895090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 0x560e97895090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 -- 172.21.15.96:0/591459474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e977b4530 con 0x560e9788d860 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 0x560e97895090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30c40117a0 tx=0x7f30c4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919cc567d057a8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30d09f9640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 0x7f30b40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30b1ffb640 1 -- 172.21.15.96:0/591459474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4011220 con 0x560e9788d860 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30caffd640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 0x7f30b40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30b1ffb640 1 -- 172.21.15.96:0/591459474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c400f9a0 con 0x560e9788d860 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30b1ffb640 1 -- 172.21.15.96:0/591459474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4004470 con 0x560e9788d860 2024-09-10T22:55:57.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.713+0000 7f30d09f9640 1 -- 172.21.15.96:0/3055076014 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f30bc1062d0 con 0x7f30cc101130 2024-09-10T22:55:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 -- 172.21.15.96:0/591459474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 msgr2=0x560e9788fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30caffd640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 0x7f30b40b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f30bc103e70 tx=0x7f30bc109df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30c40117a0 tx=0x7f30c4011c60 comp rx=0 tx=0).stop 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 -- 172.21.15.96:0/591459474 shutdown_connections 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e97892bd0 0x560e97895090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e97892580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 --2- 172.21.15.96:0/591459474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9788d860 0x560e9788fc50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 -- 172.21.15.96:0/591459474 >> 172.21.15.96:0/591459474 conn(0x560e978889b0 msgr2=0x560e97889740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 -- 172.21.15.96:0/591459474 shutdown_connections 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.714+0000 7f30d47c8740 1 -- 172.21.15.96:0/591459474 wait complete. 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 -- start start 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 Processor -- start 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e9789b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97892bd0 0x560e9789c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e97726ab0 con 0x560e97892bd0 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e97794730 con 0x560e9788d860 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560e97899f30 con 0x560e97890190 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97892bd0 0x560e9789c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30ca7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e9789b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44044/0 (socket says 172.21.15.96:44044) 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.715+0000 7f30d47c8740 10 client.0 _ll_register_callbacks cb 0x560e977e86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 -- 172.21.15.96:0/322678565 learned_addr learned my addr 172.21.15.96:0/322678565 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30ca7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e9789b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39234/0 (socket says 172.21.15.96:39234) 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 -- 172.21.15.96:0/322678565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 msgr2=0x560e9789b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e9789b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 -- 172.21.15.96:0/322678565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97892bd0 msgr2=0x560e9789c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97892bd0 0x560e9789c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 -- 172.21.15.96:0/322678565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e97550790 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005fd0 tx=0x7f30c4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493ae1a7cc7f42a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4011260 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c400f9a0 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f30c4004a40 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30c4004c40 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30d47c8740 10 client.26861 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.716+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560e975c9620 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c400f9a0 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f30c4021020 con 0x560e9788d860 2024-09-10T22:55:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 10 client.26861 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 10 client.26861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 20 client.26861 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e978a6940 0x560e978a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 4 client.26861 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e97801640 con 0x560e978a6940 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e978a6940 0x560e978a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e978a6940 0x560e978a8d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4375b042b9384684 server_cookie=75d86597cc8be710 in_seq=0 out_seq=0 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.717+0000 7f30aaffd640 10 client.26861 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30aaffd640 1 -- 172.21.15.96:0/322678565 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e97801640 con 0x560e978a6940 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30aaffd640 10 client.26861 handle_command_reply: tid=0 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 client.26861 shutdown 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 msgr2=0x560e9789b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005fd0 tx=0x7f30c4000f40 comp rx=0 tx=0).stop 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 shutdown_connections 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e978a6940 0x560e978a8d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97892bd0 0x560e9789c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97890190 0x560e9789b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 --2- 172.21.15.96:0/322678565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e9789b270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30aaffd640 0 client.26861 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 >> 172.21.15.96:0/322678565 conn(0x560e978889b0 msgr2=0x560e97887bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 shutdown_connections 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 1 -- 172.21.15.96:0/322678565 wait complete. 2024-09-10T22:55:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.718+0000 7f30d47c8740 20 client.26861 trim_cache size 0 max 16384 2024-09-10T22:55:57.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:57 smithi096 ceph-mon[30869]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.4 MiB/s wr, 808 op/s 2024-09-10T22:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.749+0000 7f30d47c8740 1 Processor -- start 2024-09-10T22:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.749+0000 7f30d47c8740 1 -- start start 2024-09-10T22:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97892fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a7bc0 con 0x560e97890190 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a9700 con 0x560e9788d860 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30d47c8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a8290 con 0x560e97892bd0 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39250/0 (socket says 172.21.15.96:39250) 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30cb7fe640 1 -- 172.21.15.96:0/1676368590 learned_addr learned my addr 172.21.15.96:0/1676368590 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.750+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97892fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.751+0000 7f30cb7fe640 1 -- 172.21.15.96:0/1676368590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 msgr2=0x560e97892fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.751+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97892fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.751+0000 7f30cb7fe640 1 -- 172.21.15.96:0/1676368590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 msgr2=0x560e978934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.751+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.751+0000 7f30cb7fe640 1 -- 172.21.15.96:0/1676368590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9789b5e0 con 0x560e97892bd0 2024-09-10T22:55:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.753+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005d70 tx=0x7f30c4011c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2c70b0b685f66d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.757+0000 7f30b17fa640 1 -- 172.21.15.96:0/1676368590 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4036070 con 0x560e97892bd0 2024-09-10T22:55:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.757+0000 7f30b17fa640 1 -- 172.21.15.96:0/1676368590 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c4037040 con 0x560e97892bd0 2024-09-10T22:55:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.757+0000 7f30b17fa640 1 -- 172.21.15.96:0/1676368590 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4011260 con 0x560e97892bd0 2024-09-10T22:55:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.759+0000 7f30d47c8740 1 -- 172.21.15.96:0/1676368590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 msgr2=0x560e97893a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.759+0000 7f30d47c8740 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f30c4005d70 tx=0x7f30c4011c60 comp rx=0 tx=0).stop 2024-09-10T22:55:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:57 smithi121 ceph-mon[35119]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.4 MiB/s wr, 808 op/s 2024-09-10T22:55:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.767+0000 7f30d47c8740 1 -- 172.21.15.96:0/1676368590 shutdown_connections 2024-09-10T22:55:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.767+0000 7f30d47c8740 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97893a30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.767+0000 7f30d47c8740 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.767+0000 7f30d47c8740 1 --2- 172.21.15.96:0/1676368590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97892fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.767+0000 7f30d47c8740 1 -- 172.21.15.96:0/1676368590 >> 172.21.15.96:0/1676368590 conn(0x560e978889b0 msgr2=0x560e9788cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.774+0000 7f30d47c8740 1 -- 172.21.15.96:0/1676368590 shutdown_connections 2024-09-10T22:55:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.778+0000 7f30d47c8740 1 -- 172.21.15.96:0/1676368590 wait complete. 2024-09-10T22:55:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.778+0000 7f30d47c8740 1 -- start start 2024-09-10T22:55:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.778+0000 7f30d47c8740 1 Processor -- start 2024-09-10T22:55:57.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97895d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a9030 con 0x560e97890190 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a8960 con 0x560e9788d860 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc1a5ba0 con 0x560e97892bd0 2024-09-10T22:55:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.781+0000 7f30d47c8740 10 client.0 _ll_register_callbacks cb 0x7f30c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:55:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97895d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39260/0 (socket says 172.21.15.96:39260) 2024-09-10T22:55:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 -- 172.21.15.96:0/3582650298 learned_addr learned my addr 172.21.15.96:0/3582650298 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:55:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 -- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 msgr2=0x560e97895d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97895d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 -- 172.21.15.96:0/3582650298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 msgr2=0x560e978963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.782+0000 7f30cb7fe640 1 -- 172.21.15.96:0/3582650298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9789c850 con 0x560e97892bd0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30cb7fe640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f30c40050d0 tx=0x7f30c4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=611156a7ee5c6eba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4036040 con 0x560e97892bd0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c4037040 con 0x560e97892bd0 2024-09-10T22:55:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f30c4005670 con 0x560e97892bd0 2024-09-10T22:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.783+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30c4004de0 con 0x560e97892bd0 2024-09-10T22:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.784+0000 7f30d47c8740 10 client.26938 fetch_fsmap learned FSMap version 32 2024-09-10T22:55:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.784+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560e978928f0 con 0x560e97892bd0 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.785+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c4037040 con 0x560e97892bd0 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f30c4021040 con 0x560e97892bd0 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 10 client.26938 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 10 client.26938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 20 client.26938 populate_metadata read hostname 'smithi096' 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e97897fd0 0x560e978983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 4 client.26938 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:55:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.786+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e977fd0f0 con 0x560e97897fd0 2024-09-10T22:55:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.789+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e97897fd0 0x560e978983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:55:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.790+0000 7f30ca7fc640 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e97897fd0 0x560e978983e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6521019b9f5eb96d server_cookie=b32ed4a9425efa7c in_seq=0 out_seq=0 2024-09-10T22:55:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.790+0000 7f30aaffd640 10 client.26938 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30aaffd640 1 -- 172.21.15.96:0/3582650298 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e977fd0f0 con 0x560e97897fd0 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30aaffd640 10 client.26938 handle_command_reply: tid=0 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 client.26938 shutdown 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 msgr2=0x560e97896f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f30c40050d0 tx=0x7f30c4000f40 comp rx=0 tx=0).stop 2024-09-10T22:55:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 shutdown_connections 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560e97897fd0 0x560e978983e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560e97892bd0 0x560e97896f80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e97890190 0x560e978963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 --2- 172.21.15.96:0/3582650298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e9788d860 0x560e97895d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 >> 172.21.15.96:0/3582650298 conn(0x560e978889b0 msgr2=0x560e97887bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.816+0000 7f30aaffd640 0 client.26938 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 shutdown_connections 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d47c8740 1 -- 172.21.15.96:0/3582650298 wait complete. 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d47c8740 20 client.26938 trim_cache size 0 max 16384 2024-09-10T22:55:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:55:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:55:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:55:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 msgr2=0x7f30b40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 0x7f30b40b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f30bc103e70 tx=0x7f30bc109df0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 msgr2=0x7f30cc198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009a50 tx=0x7f30bc0042f0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 shutdown_connections 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30cc103fa0 0x7f30cc199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f30b40b5360 0x7f30b40b7820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30cc1048e0 0x7f30cc19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 --2- 172.21.15.96:0/3055076014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30cc101130 0x7f30cc198df0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.817+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 >> 172.21.15.96:0/3055076014 conn(0x7f30cc0fc880 msgr2=0x7f30cc1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.818+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 shutdown_connections 2024-09-10T22:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:55:57.818+0000 7f30d19fb640 1 -- 172.21.15.96:0/3055076014 wait complete. 2024-09-10T22:55:59.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:55:59 smithi117 ceph-mon[35081]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-09-10T22:55:59.551 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:55:59 smithi096 ceph-mon[30869]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-09-10T22:55:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:55:59 smithi121 ceph-mon[35119]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-09-10T22:56:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 505.0 s, 1784.2 tps, lat 55.544 ms stddev 61.144 2024-09-10T22:56:01.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:01 smithi096 ceph-mon[30869]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.8 MiB/s wr, 884 op/s 2024-09-10T22:56:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:01 smithi121 ceph-mon[35119]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.8 MiB/s wr, 884 op/s 2024-09-10T22:56:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:01 smithi117 ceph-mon[35081]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.8 MiB/s wr, 884 op/s 2024-09-10T22:56:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:03 smithi096 ceph-mon[30869]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 699 op/s 2024-09-10T22:56:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:03 smithi121 ceph-mon[35119]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 699 op/s 2024-09-10T22:56:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:03 smithi117 ceph-mon[35081]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 699 op/s 2024-09-10T22:56:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:05 smithi096 ceph-mon[30869]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-09-10T22:56:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:05 smithi121 ceph-mon[35119]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-09-10T22:56:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:05 smithi117 ceph-mon[35081]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 729 op/s 2024-09-10T22:56:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 510.0 s, 1941.6 tps, lat 51.973 ms stddev 56.030 2024-09-10T22:56:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:07 smithi096 ceph-mon[30869]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 684 op/s 2024-09-10T22:56:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:07 smithi121 ceph-mon[35119]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 684 op/s 2024-09-10T22:56:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:07 smithi117 ceph-mon[35081]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 684 op/s 2024-09-10T22:56:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:09 smithi096 ceph-mon[30869]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-09-10T22:56:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:09 smithi121 ceph-mon[35119]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-09-10T22:56:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:09 smithi117 ceph-mon[35081]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 596 op/s 2024-09-10T22:56:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 -- 172.21.15.96:0/2954623414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 msgr2=0x7f271c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 --2- 172.21.15.96:0/2954623414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 0x7f271c102d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f270c009930 tx=0x7f270c02a420 comp rx=0 tx=0).stop 2024-09-10T22:56:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 -- 172.21.15.96:0/2954623414 shutdown_connections 2024-09-10T22:56:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 --2- 172.21.15.96:0/2954623414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c103ae0 0x7f271c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 --2- 172.21.15.96:0/2954623414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c1044a0 0x7f271c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 --2- 172.21.15.96:0/2954623414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 0x7f271c102d90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.928+0000 7f27210cd640 1 -- 172.21.15.96:0/2954623414 >> 172.21.15.96:0/2954623414 conn(0x7f271c0fe250 msgr2=0x7f271c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.929+0000 7f27210cd640 1 -- 172.21.15.96:0/2954623414 shutdown_connections 2024-09-10T22:56:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.929+0000 7f27210cd640 1 -- 172.21.15.96:0/2954623414 wait complete. 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 Processor -- start 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 -- start start 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 0x7f271c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c1044a0 0x7f271c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10e600 con 0x7f271c102990 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10e480 con 0x7f271c1044a0 2024-09-10T22:56:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f27210cd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10e780 con 0x7f271c103ae0 2024-09-10T22:56:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f271b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c1044a0 0x7f271c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:58998/0 (socket says 172.21.15.96:58998) 2024-09-10T22:56:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.930+0000 7f271a59c640 1 -- 172.21.15.96:0/883092075 learned_addr learned my addr 172.21.15.96:0/883092075 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 -- 172.21.15.96:0/883092075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c1044a0 msgr2=0x7f271c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c1044a0 0x7f271c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 -- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 msgr2=0x7f271c198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 0x7f271c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 -- 172.21.15.96:0/883092075 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c19a920 con 0x7f271c103ae0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f271a59c640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f271000d950 tx=0x7f271000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.931+0000 7f2703fff640 1 -- 172.21.15.96:0/883092075 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710017070 con 0x7f271c103ae0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.932+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c19ac10 con 0x7f271c103ae0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.932+0000 7f2703fff640 1 -- 172.21.15.96:0/883092075 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2710009c30 con 0x7f271c103ae0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.932+0000 7f2703fff640 1 -- 172.21.15.96:0/883092075 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271000f620 con 0x7f271c103ae0 2024-09-10T22:56:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.932+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a12e0 con 0x7f271c103ae0 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2703fff640 1 -- 172.21.15.96:0/883092075 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f2710010460 con 0x7f271c103ae0 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 Processor -- start 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 -- start start 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7772190 0x563ee7774580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee7777090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ee76e16f0 con 0x563ee776f860 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ee76d0b20 con 0x563ee7772190 2024-09-10T22:56:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f2723ea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ee76d4930 con 0x563ee7774bd0 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f271b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f271b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45448/0 (socket says 172.21.15.96:45448) 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.935+0000 7f271b59e640 1 -- 172.21.15.96:0/1219186407 learned_addr learned my addr 172.21.15.96:0/1219186407 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271a59c640 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee7777090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271ad9d640 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7772190 0x563ee7774580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 -- 172.21.15.96:0/1219186407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 msgr2=0x563ee7777090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee7777090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 -- 172.21.15.96:0/1219186407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7772190 msgr2=0x563ee7774580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7772190 0x563ee7774580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 -- 172.21.15.96:0/1219186407 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee7696530 con 0x563ee776f860 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f2703fff640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 0x7f27040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f2703fff640 1 -- 172.21.15.96:0/883092075 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f27100e5880 con 0x7f271c103ae0 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271b59e640 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2714009eb0 tx=0x7f2714010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d30d4286cc26d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f271ad9d640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 0x7f27040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f27017fa640 1 -- 172.21.15.96:0/1219186407 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714016070 con 0x563ee776f860 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f27017fa640 1 -- 172.21.15.96:0/1219186407 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2714004d20 con 0x563ee776f860 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.936+0000 7f27017fa640 1 -- 172.21.15.96:0/1219186407 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714012690 con 0x563ee776f860 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- 172.21.15.96:0/1219186407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 msgr2=0x563ee7771c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f271ad9d640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 0x7f27040b77a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f270c031500 tx=0x7f270c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2714009eb0 tx=0x7f2714010740 comp rx=0 tx=0).stop 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- 172.21.15.96:0/1219186407 shutdown_connections 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7772190 0x563ee7774580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee7777090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 --2- 172.21.15.96:0/1219186407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee7771c50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- 172.21.15.96:0/1219186407 >> 172.21.15.96:0/1219186407 conn(0x563ee776a9b0 msgr2=0x563ee776b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- 172.21.15.96:0/1219186407 shutdown_connections 2024-09-10T22:56:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- 172.21.15.96:0/1219186407 wait complete. 2024-09-10T22:56:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 -- start start 2024-09-10T22:56:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.937+0000 7f2723ea2740 1 Processor -- start 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee776f860 0x563ee7778f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7772190 0x563ee77795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ee7608ab0 con 0x563ee7772190 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563ee7676730 con 0x563ee7774bd0 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ee7777bb0 con 0x563ee776f860 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f2723ea2740 10 client.0 _ll_register_callbacks cb 0x563ee76ca6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f271b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f271b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52934/0 (socket says 172.21.15.96:52934) 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f271b59e640 1 -- 172.21.15.96:0/164480107 learned_addr learned my addr 172.21.15.96:0/164480107 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f271ad9d640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee776f860 0x563ee7778f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.938+0000 7f271a59c640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7772190 0x563ee77795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 -- 172.21.15.96:0/164480107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee776f860 msgr2=0x563ee7778f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee776f860 0x563ee7778f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 -- 172.21.15.96:0/164480107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7772190 msgr2=0x563ee77795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7772190 0x563ee77795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 -- 172.21.15.96:0/164480107 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee7432790 con 0x563ee7774bd0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f271b59e640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2714006010 tx=0x7f2714004ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23363cbeeef5303 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f26feff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714028060 con 0x563ee7774bd0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2714004d20 con 0x563ee7774bd0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2714005280 con 0x563ee7774bd0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2714016070 con 0x563ee7774bd0 2024-09-10T22:56:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.939+0000 7f2723ea2740 10 client.26879 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ee74ab620 con 0x563ee7774bd0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2714004d20 con 0x563ee7774bd0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f271402d070 con 0x563ee7774bd0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 10 client.26879 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 10 client.26879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 20 client.26879 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee7783ed0 0x563ee7784320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 4 client.26879 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee76e3640 con 0x563ee7783ed0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.940+0000 7f271ad9d640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee7783ed0 0x563ee7784320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.941+0000 7f271ad9d640 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee7783ed0 0x563ee7784320 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce99f107feda28b9 server_cookie=3901977dcafc0232 in_seq=0 out_seq=0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.941+0000 7f26feff5640 10 client.26879 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.941+0000 7f26feff5640 1 -- 172.21.15.96:0/164480107 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ee76e3640 con 0x563ee7783ed0 2024-09-10T22:56:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.941+0000 7f26feff5640 10 client.26879 handle_command_reply: tid=0 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 client.26879 shutdown 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 msgr2=0x563ee777a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2714006010 tx=0x7f2714004ed0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 shutdown_connections 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee7783ed0 0x563ee7784320 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee7774bd0 0x563ee777a190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7772190 0x563ee77795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f26feff5640 0 client.26879 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 --2- 172.21.15.96:0/164480107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee776f860 0x563ee7778f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 >> 172.21.15.96:0/164480107 conn(0x563ee776a9b0 msgr2=0x563ee7769bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 shutdown_connections 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 1 -- 172.21.15.96:0/164480107 wait complete. 2024-09-10T22:56:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.942+0000 7f2723ea2740 20 client.26879 trim_cache size 0 max 16384 2024-09-10T22:56:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 515.0 s, 3065.1 tps, lat 32.597 ms stddev 37.291 2024-09-10T22:56:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.966+0000 7f2723ea2740 1 Processor -- start 2024-09-10T22:56:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.966+0000 7f2723ea2740 1 -- start start 2024-09-10T22:56:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.966+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 0x563ee7778160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.966+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.966+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 0x563ee7785010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59026/0 (socket says 172.21.15.96:59026) 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f271a59c640 1 -- 172.21.15.96:0/3625715439 learned_addr learned my addr 172.21.15.96:0/3625715439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a7cd0 con 0x563ee7774bd0 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a9810 con 0x563ee776f860 2024-09-10T22:56:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.967+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a83a0 con 0x563ee7772190 2024-09-10T22:56:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.970+0000 7f271b59e640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 0x563ee7785010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.971+0000 7f271ad9d640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 0x563ee7778160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271a59c640 1 -- 172.21.15.96:0/3625715439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 msgr2=0x563ee7778160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271a59c640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 0x563ee7778160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271a59c640 1 -- 172.21.15.96:0/3625715439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 msgr2=0x563ee7785010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271a59c640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 0x563ee7785010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271a59c640 1 -- 172.21.15.96:0/3625715439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee77748f0 con 0x563ee7772190 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271b59e640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 0x563ee7785010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.973+0000 7f271ad9d640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 0x563ee7778160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.975+0000 7f271a59c640 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f27100abde0 tx=0x7f27100eb530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e04b4e495edd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.980+0000 7f2700ff9640 1 -- 172.21.15.96:0/3625715439 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710020070 con 0x563ee7772190 2024-09-10T22:56:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.980+0000 7f2700ff9640 1 -- 172.21.15.96:0/3625715439 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27100abbf0 con 0x563ee7772190 2024-09-10T22:56:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.980+0000 7f2700ff9640 1 -- 172.21.15.96:0/3625715439 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710009de0 con 0x563ee7772190 2024-09-10T22:56:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.982+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 msgr2=0x563ee7784ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.982+0000 7f2723ea2740 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f27100abde0 tx=0x7f27100eb530 comp rx=0 tx=0).stop 2024-09-10T22:56:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.983+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 shutdown_connections 2024-09-10T22:56:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.983+0000 7f2723ea2740 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee7774bd0 0x563ee7785010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.983+0000 7f2723ea2740 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7772190 0x563ee7784ad0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f27100abde0 tx=0x7f27100eb530 comp rx=0 tx=0).stop 2024-09-10T22:56:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.983+0000 7f2723ea2740 1 --2- 172.21.15.96:0/3625715439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee776f860 0x563ee7778160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.983+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 >> 172.21.15.96:0/3625715439 conn(0x563ee776a9b0 msgr2=0x563ee776ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.992+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 shutdown_connections 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.993+0000 7f2723ea2740 1 -- 172.21.15.96:0/3625715439 wait complete. 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 -- start start 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 Processor -- start 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee777dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee777e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 0x563ee777f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a9140 con 0x563ee776f860 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a8a70 con 0x563ee777ed20 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1a5cb0 con 0x563ee7774bd0 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f2723ea2740 10 client.0 _ll_register_callbacks cb 0x7f270c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee777e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f271b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 0x563ee777f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f271a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee777e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59040/0 (socket says 172.21.15.96:59040) 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.994+0000 7f271a59c640 1 -- 172.21.15.96:0/2590478129 learned_addr learned my addr 172.21.15.96:0/2590478129 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 -- 172.21.15.96:0/2590478129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 msgr2=0x563ee777e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee777e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 -- 172.21.15.96:0/2590478129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 msgr2=0x563ee777dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee777dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 -- 172.21.15.96:0/2590478129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee777f9d0 con 0x563ee777ed20 2024-09-10T22:56:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:10.995+0000 7f271b59e640 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 0x563ee777f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2714012e10 tx=0x7f2714010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499ae6df8aec6d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.001+0000 7f26feff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.001+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714033030 con 0x563ee777ed20 2024-09-10T22:56:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.001+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2714020040 con 0x563ee777ed20 2024-09-10T22:56:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.001+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27100abbf0 con 0x563ee777ed20 2024-09-10T22:56:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.004+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f271403e050 con 0x563ee777ed20 2024-09-10T22:56:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.004+0000 7f2723ea2740 10 client.26885 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.004+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ee7780cb0 con 0x563ee777ed20 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.007+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27100abbf0 con 0x563ee777ed20 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.007+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f271402d070 con 0x563ee777ed20 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.007+0000 7f2723ea2740 10 client.26885 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.007+0000 7f2723ea2740 10 client.26885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.007+0000 7f2723ea2740 20 client.26885 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.008+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee778c070 0x563ee778c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.008+0000 7f2723ea2740 4 client.26885 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.008+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ee76df0f0 con 0x563ee778c070 2024-09-10T22:56:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.008+0000 7f271ad9d640 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee778c070 0x563ee778c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.009+0000 7f271ad9d640 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee778c070 0x563ee778c480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3118bd645a11e236 server_cookie=bafebd82689f5c39 in_seq=0 out_seq=0 2024-09-10T22:56:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.009+0000 7f26feff5640 10 client.26885 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f26feff5640 1 -- 172.21.15.96:0/2590478129 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ee76df0f0 con 0x563ee778c070 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f26feff5640 10 client.26885 handle_command_reply: tid=0 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 client.26885 shutdown 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 msgr2=0x563ee777f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 0x563ee777f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2714012e10 tx=0x7f2714010c80 comp rx=0 tx=0).stop 2024-09-10T22:56:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 shutdown_connections 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563ee778c070 0x563ee778c480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563ee777ed20 0x563ee777f1a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ee7774bd0 0x563ee777e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f26feff5640 0 client.26885 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 --2- 172.21.15.96:0/2590478129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ee776f860 0x563ee777dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.040+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 >> 172.21.15.96:0/2590478129 conn(0x563ee776a9b0 msgr2=0x563ee7774fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 shutdown_connections 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f2723ea2740 1 -- 172.21.15.96:0/2590478129 wait complete. 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f2723ea2740 20 client.26885 trim_cache size 0 max 16384 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:56:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:56:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:56:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:56:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 msgr2=0x7f27040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 0x7f27040b77a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f270c031500 tx=0x7f270c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 msgr2=0x7f271c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f271000d950 tx=0x7f271000de10 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 shutdown_connections 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f271c1044a0 0x7f271c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27040b52e0 0x7f27040b77a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f271c103ae0 0x7f271c199430 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 --2- 172.21.15.96:0/883092075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f271c102990 0x7f271c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.041+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 >> 172.21.15.96:0/883092075 conn(0x7f271c0fe250 msgr2=0x7f271c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.042+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 shutdown_connections 2024-09-10T22:56:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:11.042+0000 7f27210cd640 1 -- 172.21.15.96:0/883092075 wait complete. 2024-09-10T22:56:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:11 smithi096 ceph-mon[30869]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 705 op/s 2024-09-10T22:56:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:11 smithi121 ceph-mon[35119]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 705 op/s 2024-09-10T22:56:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:11 smithi117 ceph-mon[35081]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 705 op/s 2024-09-10T22:56:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:13 smithi096 ceph-mon[30869]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-10T22:56:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:13 smithi121 ceph-mon[35119]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-10T22:56:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:13 smithi117 ceph-mon[35081]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-10T22:56:15.577 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:15 smithi117 ceph-mon[35081]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 662 op/s 2024-09-10T22:56:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:15 smithi096 ceph-mon[30869]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 662 op/s 2024-09-10T22:56:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:15 smithi121 ceph-mon[35119]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 662 op/s 2024-09-10T22:56:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 520.0 s, 2144.0 tps, lat 46.587 ms stddev 54.917 2024-09-10T22:56:16.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:56:16.963 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:17.450 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:17 smithi096 ceph-mon[30869]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T22:56:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 -- 172.21.15.96:0/3022920008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 msgr2=0x7f541410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 --2- 172.21.15.96:0/3022920008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f541410cab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5404009900 tx=0x7f540402a400 comp rx=0 tx=0).stop 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 -- 172.21.15.96:0/3022920008 shutdown_connections 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 --2- 172.21.15.96:0/3022920008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 --2- 172.21.15.96:0/3022920008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f541410cab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 --2- 172.21.15.96:0/3022920008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 -- 172.21.15.96:0/3022920008 >> 172.21.15.96:0/3022920008 conn(0x7f54140fd700 msgr2=0x7f54140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 -- 172.21.15.96:0/3022920008 shutdown_connections 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.588+0000 7f541b39f640 1 -- 172.21.15.96:0/3022920008 wait complete. 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 Processor -- start 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 -- start start 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f5414199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5414112d70 con 0x7f5414103ca0 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5414112bf0 con 0x7f541410d100 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541b39f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5414112ef0 con 0x7f54141045c0 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f5414199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541913b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f5414199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51384/0 (socket says 172.21.15.96:51384) 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541913b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48154/0 (socket says 172.21.15.96:48154) 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541893a640 1 -- 172.21.15.96:0/650674409 learned_addr learned my addr 172.21.15.96:0/650674409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 -- 172.21.15.96:0/650674409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 msgr2=0x7f5414199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f5414199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 -- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 msgr2=0x7f5414199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.589+0000 7f541993c640 1 -- 172.21.15.96:0/650674409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541419f3b0 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.590+0000 7f541993c640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541419ebe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f540c009fd0 tx=0x7f540c00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.591+0000 7f540a7fc640 1 -- 172.21.15.96:0/650674409 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c004b20 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.591+0000 7f540a7fc640 1 -- 172.21.15.96:0/650674409 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c004cc0 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.591+0000 7f540a7fc640 1 -- 172.21.15.96:0/650674409 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0053d0 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.591+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541419f6a0 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.591+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5414101830 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 Processor -- start 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f540a7fc640 1 -- 172.21.15.96:0/650674409 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f540c019410 con 0x7f541410d100 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 -- start start 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207c8b0 0x558d3207cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d3208b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558d31feae50 con 0x558d32089160 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d31fda280 con 0x558d3207c8b0 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.595+0000 7f541d774740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558d31fde090 con 0x558d3207d1d0 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51400/0 (socket says 172.21.15.96:51400) 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 -- 172.21.15.96:0/549194841 learned_addr learned my addr 172.21.15.96:0/549194841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541893a640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207c8b0 0x558d3207cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541893a640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d3208b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 -- 172.21.15.96:0/549194841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207c8b0 msgr2=0x558d3207cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207c8b0 0x558d3207cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 -- 172.21.15.96:0/549194841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 msgr2=0x558d3208b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d3208b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 -- 172.21.15.96:0/549194841 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d31d38650 con 0x558d3207d1d0 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541893a640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d3208b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f540a7fc640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 0x7f54000b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f540a7fc640 1 -- 172.21.15.96:0/650674409 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f540c0e5350 con 0x7f541410d100 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541913b640 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f541000bd10 tx=0x7f54100107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8007bf3cf91b8bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.596+0000 7f541893a640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 0x7f54000b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f53ff7fe640 1 -- 172.21.15.96:0/549194841 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410004030 con 0x558d3207d1d0 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f53ff7fe640 1 -- 172.21.15.96:0/549194841 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54100035f0 con 0x558d3207d1d0 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f53ff7fe640 1 -- 172.21.15.96:0/549194841 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100117b0 con 0x558d3207d1d0 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f541d774740 1 -- 172.21.15.96:0/549194841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 msgr2=0x558d32088c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f541893a640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 0x7f54000b7970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f541410f810 tx=0x7f5404005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.597+0000 7f541d774740 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f541000bd10 tx=0x7f54100107a0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 -- 172.21.15.96:0/549194841 shutdown_connections 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d3208b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32088c20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 --2- 172.21.15.96:0/549194841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207c8b0 0x558d3207cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 -- 172.21.15.96:0/549194841 >> 172.21.15.96:0/549194841 conn(0x558d3207a6e0 msgr2=0x558d3207b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 -- 172.21.15.96:0/549194841 shutdown_connections 2024-09-10T22:56:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 -- 172.21.15.96:0/549194841 wait complete. 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 -- start start 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.598+0000 7f541d774740 1 Processor -- start 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32088a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d32086460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558d31e3b700 con 0x558d3207c8b0 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d31f75110 con 0x558d32089160 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558d3208bde0 con 0x558d3207d1d0 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541d774740 10 client.0 _ll_register_callbacks cb 0x558d31f8e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541993c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d32086460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51414/0 (socket says 172.21.15.96:51414) 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541893a640 1 -- 172.21.15.96:0/2155275650 learned_addr learned my addr 172.21.15.96:0/2155275650 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.599+0000 7f541913b640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32088a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 -- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 msgr2=0x558d32086460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d32086460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 -- 172.21.15.96:0/2155275650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 msgr2=0x558d32088a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32088a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 -- 172.21.15.96:0/2155275650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d31fa2c40 con 0x558d3207d1d0 2024-09-10T22:56:17.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541893a640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5404005940 tx=0x7f540402a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c0581e3eb32b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f53fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540404e030 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5404031830 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.600+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54100035f0 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f540405a050 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f541d774740 10 client.26968 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d320a8730 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54100035f0 con 0x558d3207d1d0 2024-09-10T22:56:17.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5404047050 con 0x558d3207d1d0 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f541d774740 10 client.26968 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.601+0000 7f541d774740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541d774740 20 client.26968 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d320a9bf0 0x558d320abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541d774740 4 client.26968 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d31fecda0 con 0x558d320a9bf0 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541913b640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d320a9bf0 0x558d320abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f541913b640 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d320a9bf0 0x558d320abfe0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1bb69bb876139c server_cookie=e1b5e7d7d43c0473 in_seq=0 out_seq=0 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.602+0000 7f53fcff9640 10 client.26968 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f53fcff9640 1 -- 172.21.15.96:0/2155275650 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d31fecda0 con 0x558d320a9bf0 2024-09-10T22:56:17.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f53fcff9640 10 client.26968 handle_command_reply: tid=0 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 client.26968 shutdown 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 msgr2=0x558d32085e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5404005940 tx=0x7f540402a400 comp rx=0 tx=0).stop 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 shutdown_connections 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d320a9bf0 0x558d320abfe0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d32086460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32085e80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 --2- 172.21.15.96:0/2155275650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32088a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 >> 172.21.15.96:0/2155275650 conn(0x558d3207a6e0 msgr2=0x558d32080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.603+0000 7f53fcff9640 0 client.26968 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.604+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 shutdown_connections 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.604+0000 7f541d774740 1 -- 172.21.15.96:0/2155275650 wait complete. 2024-09-10T22:56:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.604+0000 7f541d774740 20 client.26968 trim_cache size 0 max 16384 2024-09-10T22:56:17.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.626+0000 7f541d774740 1 Processor -- start 2024-09-10T22:56:17.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 -- start start 2024-09-10T22:56:17.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 0x558d320835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d320b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541913b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541913b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51420/0 (socket says 172.21.15.96:51420) 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541893a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 0x558d320835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541893a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 0x558d320835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35016/0 (socket says 172.21.15.96:35016) 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5414072f20 con 0x558d32089160 2024-09-10T22:56:17.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5414074a60 con 0x558d3207d1d0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.627+0000 7f541d774740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54140735f0 con 0x558d3207c8b0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 -- 172.21.15.96:0/915148057 learned_addr learned my addr 172.21.15.96:0/915148057 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541993c640 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d320b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 -- 172.21.15.96:0/915148057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 msgr2=0x558d320835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 0x558d320835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 -- 172.21.15.96:0/915148057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 msgr2=0x558d320b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d320b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 -- 172.21.15.96:0/915148057 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d3207b160 con 0x558d3207c8b0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541993c640 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d320b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f541913b640 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f541000bd10 tx=0x7f5410003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec11f2776c708c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.628+0000 7f53feffd640 1 -- 172.21.15.96:0/915148057 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100035f0 con 0x558d3207c8b0 2024-09-10T22:56:17.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f53feffd640 1 -- 172.21.15.96:0/915148057 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5410005040 con 0x558d3207c8b0 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f53feffd640 1 -- 172.21.15.96:0/915148057 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100116a0 con 0x558d3207c8b0 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 -- 172.21.15.96:0/915148057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 msgr2=0x558d32083090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f541000bd10 tx=0x7f5410003d20 comp rx=0 tx=0).stop 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 -- 172.21.15.96:0/915148057 shutdown_connections 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d3207d1d0 0x558d320835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d32089160 0x558d320b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 --2- 172.21.15.96:0/915148057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207c8b0 0x558d32083090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 -- 172.21.15.96:0/915148057 >> 172.21.15.96:0/915148057 conn(0x558d3207a6e0 msgr2=0x558d32084700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 -- 172.21.15.96:0/915148057 shutdown_connections 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.629+0000 7f541d774740 1 -- 172.21.15.96:0/915148057 wait complete. 2024-09-10T22:56:17.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.630+0000 7f541d774740 1 -- start start 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.630+0000 7f541d774740 1 Processor -- start 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32091510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d3208c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5414074390 con 0x558d3207c8b0 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5414073cc0 con 0x558d32089160 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5414070f20 con 0x558d3207d1d0 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541d774740 10 client.0 _ll_register_callbacks cb 0x7f5404009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51426/0 (socket says 172.21.15.96:51426) 2024-09-10T22:56:17.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 -- 172.21.15.96:0/2574462823 learned_addr learned my addr 172.21.15.96:0/2574462823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541993c640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d3208c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 -- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 msgr2=0x558d3208c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d3208c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 -- 172.21.15.96:0/2574462823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 msgr2=0x558d32091510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32091510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 -- 172.21.15.96:0/2574462823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d3208ca90 con 0x558d3207d1d0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.631+0000 7f541893a640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5404031440 tx=0x7f5404032c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e4a87ae2521836 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f53fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404057050 con 0x558d3207d1d0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f540404ae00 con 0x558d3207d1d0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5410005040 con 0x558d3207d1d0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f540404e040 con 0x558d3207d1d0 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f541d774740 10 client.26974 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:17.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.632+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558d321177a0 con 0x558d3207d1d0 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5410005040 con 0x558d3207d1d0 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5404066030 con 0x558d3207d1d0 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 10 client.26974 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 20 client.26974 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d32117f50 0x558d3211a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 4 client.26974 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.633+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d31fe8850 con 0x558d32117f50 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.634+0000 7f541913b640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d32117f50 0x558d3211a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f53fcff9640 10 client.26974 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f541913b640 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d32117f50 0x558d3211a3d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564e1c8beb314398 server_cookie=18078c534efda4f0 in_seq=0 out_seq=0 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f53fcff9640 1 -- 172.21.15.96:0/2574462823 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x558d31fe8850 con 0x558d32117f50 2024-09-10T22:56:17.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f53fcff9640 10 client.26974 handle_command_reply: tid=0 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f541d774740 1 client.26974 shutdown 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 msgr2=0x558d32091a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5404031440 tx=0x7f5404032c90 comp rx=0 tx=0).stop 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.635+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 shutdown_connections 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558d32117f50 0x558d3211a3d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f53fcff9640 0 client.26974 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d32089160 0x558d3208c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558d3207d1d0 0x558d32091a90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 --2- 172.21.15.96:0/2574462823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d3207c8b0 0x558d32091510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 >> 172.21.15.96:0/2574462823 conn(0x558d3207a6e0 msgr2=0x558d32087cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 shutdown_connections 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 1 -- 172.21.15.96:0/2574462823 wait complete. 2024-09-10T22:56:17.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.636+0000 7f541d774740 20 client.26974 trim_cache size 0 max 16384 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:56:17.643 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:56:17.649 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 msgr2=0x7f54000b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 0x7f54000b7970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f541410f810 tx=0x7f5404005db0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 msgr2=0x7f541419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541419ebe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f540c009fd0 tx=0x7f540c00b690 comp rx=0 tx=0).stop 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 shutdown_connections 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f541410d100 0x7f541419ebe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f54000b54b0 0x7f54000b7970 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54141045c0 0x7f5414199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 --2- 172.21.15.96:0/650674409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5414103ca0 0x7f5414199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 >> 172.21.15.96:0/650674409 conn(0x7f54140fd700 msgr2=0x7f54140fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 shutdown_connections 2024-09-10T22:56:17.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:17.637+0000 7f541b39f640 1 -- 172.21.15.96:0/650674409 wait complete. 2024-09-10T22:56:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:17 smithi121 ceph-mon[35119]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T22:56:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:17 smithi117 ceph-mon[35081]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T22:56:18.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:56:18.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:56:18.902 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:19.565 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:19 smithi121 ceph-mon[35119]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-09-10T22:56:19.656 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:19 smithi096 ceph-mon[30869]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 -- 172.21.15.96:0/1650787598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751810a3b0 msgr2=0x7f751810a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 --2- 172.21.15.96:0/1650787598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751810a3b0 0x7f751810a790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802b510 comp rx=0 tx=0).stop 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 -- 172.21.15.96:0/1650787598 shutdown_connections 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 --2- 172.21.15.96:0/1650787598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7518075070 0x7f75180754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 --2- 172.21.15.96:0/1650787598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f751807c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 --2- 172.21.15.96:0/1650787598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f751810a3b0 0x7f751810a790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 -- 172.21.15.96:0/1650787598 >> 172.21.15.96:0/1650787598 conn(0x7f751806dfc0 msgr2=0x7f751806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:19.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 -- 172.21.15.96:0/1650787598 shutdown_connections 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.761+0000 7f751ef39640 1 -- 172.21.15.96:0/1650787598 wait complete. 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 Processor -- start 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 -- start start 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 0x7f751807be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f751810a3b0 0x7f7518079600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f751807f690 con 0x7f75180759f0 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f751807f510 con 0x7f751810a3b0 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ef39640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f751807f810 con 0x7f7518075070 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f7517fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f751ccd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 0x7f751807be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f7517fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48186/0 (socket says 172.21.15.96:48186) 2024-09-10T22:56:19.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.762+0000 7f7517fff640 1 -- 172.21.15.96:0/609628694 learned_addr learned my addr 172.21.15.96:0/609628694 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f751d4d6640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f751810a3b0 0x7f7518079600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 -- 172.21.15.96:0/609628694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 msgr2=0x7f751807be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 0x7f751807be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 -- 172.21.15.96:0/609628694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f751810a3b0 msgr2=0x7f7518079600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f751810a3b0 0x7f7518079600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 -- 172.21.15.96:0/609628694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7518079e70 con 0x7f75180759f0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f751ccd5640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 0x7f751807be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7517fff640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f750000e9f0 tx=0x7f750000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750000cd80 con 0x7f75180759f0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75181befe0 con 0x7f75180759f0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75000108e0 con 0x7f75180759f0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500018980 con 0x7f75180759f0 2024-09-10T22:56:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.763+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181bf490 con 0x7f75180759f0 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.765+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75180754b0 con 0x7f75180759f0 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.767+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7500010430 con 0x7f75180759f0 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.769+0000 7f7515ffb640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 0x7f75040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.769+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f75000e3f70 con 0x7f75180759f0 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.769+0000 7f751ccd5640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 0x7f75040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.770+0000 7f751ccd5640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 0x7f75040b77a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7508032440 tx=0x7f750800d320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:19.772+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f75000acbd0 con 0x7f75180759f0 2024-09-10T22:56:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:19 smithi117 ceph-mon[35081]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-09-10T22:56:20.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.112+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7518083510 con 0x7f75180759f0 2024-09-10T22:56:20.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.113+0000 7f7515ffb640 1 -- 172.21.15.96:0/609628694 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f75000b1600 con 0x7f75180759f0 2024-09-10T22:56:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:56:20.115 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:56:20.115 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:56:20.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 msgr2=0x7f75040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 0x7f75040b77a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7508032440 tx=0x7f750800d320 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 msgr2=0x7f75180790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f750000e9f0 tx=0x7f750000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 shutdown_connections 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f751810a3b0 0x7f7518079600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f75040b52e0 0x7f75040b77a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75180759f0 0x7f75180790c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 --2- 172.21.15.96:0/609628694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7518075070 0x7f751807be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 >> 172.21.15.96:0/609628694 conn(0x7f751806dfc0 msgr2=0x7f751810d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 shutdown_connections 2024-09-10T22:56:20.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:20.115+0000 7f751ef39640 1 -- 172.21.15.96:0/609628694 wait complete. 2024-09-10T22:56:20.497 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:20 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/609628694' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:20.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:20 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/609628694' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:20.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:20 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/609628694' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:56:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 525.0 s, 2124.2 tps, lat 46.926 ms stddev 55.058 2024-09-10T22:56:21.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:56:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:21 smithi096 ceph-mon[30869]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 846 op/s 2024-09-10T22:56:21.624 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:21 smithi121 ceph-mon[35119]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 846 op/s 2024-09-10T22:56:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:21 smithi117 ceph-mon[35081]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 846 op/s 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 -- 172.21.15.96:0/1390283437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc101450 msgr2=0x7fc1cc101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 --2- 172.21.15.96:0/1390283437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc101450 0x7fc1cc101830 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00aa10 tx=0x7fc1bc02a610 comp rx=0 tx=0).stop 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 -- 172.21.15.96:0/1390283437 shutdown_connections 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 --2- 172.21.15.96:0/1390283437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc107800 0x7fc1cc10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 --2- 172.21.15.96:0/1390283437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 0x7fc1cc1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 --2- 172.21.15.96:0/1390283437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc101450 0x7fc1cc101830 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 -- 172.21.15.96:0/1390283437 >> 172.21.15.96:0/1390283437 conn(0x7fc1cc078140 msgr2=0x7fc1cc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 -- 172.21.15.96:0/1390283437 shutdown_connections 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.273+0000 7fc1d123f640 1 -- 172.21.15.96:0/1390283437 wait complete. 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.274+0000 7fc1d123f640 1 Processor -- start 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.274+0000 7fc1d123f640 1 -- start start 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc101450 0x7fc1cc19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 0x7fc1cc19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc110bd0 con 0x7fc1cc107800 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc110a50 con 0x7fc1cc101450 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1d123f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc110d50 con 0x7fc1cc106e60 2024-09-10T22:56:23.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48204/0 (socket says 172.21.15.96:48204) 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 -- 172.21.15.96:0/2196281677 learned_addr learned my addr 172.21.15.96:0/2196281677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 0x7fc1cc19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc101450 0x7fc1cc19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 -- 172.21.15.96:0/2196281677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 msgr2=0x7fc1cc19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 0x7fc1cc19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 -- 172.21.15.96:0/2196281677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc101450 msgr2=0x7fc1cc19b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc101450 0x7fc1cc19b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 -- 172.21.15.96:0/2196281677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc1a1600 con 0x7fc1cc107800 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.275+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40099e0 tx=0x7fc1c4009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.276+0000 7fc1c8d99640 1 -- 172.21.15.96:0/2196281677 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4004b20 con 0x7fc1cc107800 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.276+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1a18f0 con 0x7fc1cc107800 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.276+0000 7fc1c8d99640 1 -- 172.21.15.96:0/2196281677 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c4004cc0 con 0x7fc1cc107800 2024-09-10T22:56:23.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.276+0000 7fc1c8d99640 1 -- 172.21.15.96:0/2196281677 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4005760 con 0x7fc1cc107800 2024-09-10T22:56:23.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.276+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1a7fb0 con 0x7fc1cc107800 2024-09-10T22:56:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.278+0000 7fc1d3614740 1 Processor -- start 2024-09-10T22:56:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.278+0000 7fc1c8d99640 1 -- 172.21.15.96:0/2196281677 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc1c400e840 con 0x7fc1cc107800 2024-09-10T22:56:23.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.278+0000 7fc1d3614740 1 -- start start 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc6694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc677d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dfc5dfea0 con 0x559dfc6690f0 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559dfc5cf2d0 con 0x559dfc669a10 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1d3614740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559dfc5d30e0 con 0x559dfc6759a0 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35066/0 (socket says 172.21.15.96:35066) 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3900873492 learned_addr learned my addr 172.21.15.96:0/3900873492 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc677d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc6694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3900873492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 msgr2=0x559dfc677d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc677d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3900873492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 msgr2=0x559dfc6694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc6694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3900873492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfc32d650 con 0x559dfc669a10 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.279+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc677d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c8d99640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 0x7fc1b80b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc02ab80 tx=0x7fc1bc0040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d82f2b46a359b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 0x7fc1b80b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c8d99640 1 -- 172.21.15.96:0/2196281677 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc1c40e57a0 con 0x7fc1cc107800 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c17fa640 1 -- 172.21.15.96:0/3900873492 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc035030 con 0x559dfc669a10 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c17fa640 1 -- 172.21.15.96:0/3900873492 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc0044c0 con 0x559dfc669a10 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.280+0000 7fc1c17fa640 1 -- 172.21.15.96:0/3900873492 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004e50 con 0x559dfc669a10 2024-09-10T22:56:23.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- 172.21.15.96:0/3900873492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 msgr2=0x559dfc675460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 0x7fc1b80b7860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400bfd0 tx=0x7fc1b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc02ab80 tx=0x7fc1bc0040d0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- 172.21.15.96:0/3900873492 shutdown_connections 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc675460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc677d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3900873492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc6694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- 172.21.15.96:0/3900873492 >> 172.21.15.96:0/3900873492 conn(0x559dfc666f20 msgr2=0x559dfc6680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- 172.21.15.96:0/3900873492 shutdown_connections 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- 172.21.15.96:0/3900873492 wait complete. 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 -- start start 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.281+0000 7fc1d3614740 1 Processor -- start 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc672660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc672be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dfc430700 con 0x559dfc6690f0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559dfc56a160 con 0x559dfc669a10 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559dfc6785c0 con 0x559dfc6759a0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48222/0 (socket says 172.21.15.96:48222) 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3216733807 learned_addr learned my addr 172.21.15.96:0/3216733807 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc672660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc672be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1d3614740 10 client.0 _ll_register_callbacks cb 0x559dfc5839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3216733807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 msgr2=0x559dfc672be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc672be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 msgr2=0x559dfc672660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc672660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 -- 172.21.15.96:0/3216733807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfc597c90 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc02ac00 tx=0x7fc1bc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844500ea0b6c54dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.282+0000 7fc1b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc035040 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc1bc003c20 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559dfc5e1df0 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc00a630 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 10 client.17328 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dfc683eb0 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559dfc5e1df0 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc1bc054030 con 0x559dfc6690f0 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 10 client.17328 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 10 client.17328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:56:23.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 20 client.17328 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:23.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc685200 0x559dfc6875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 4 client.17328 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:23.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dfc5dd8a0 con 0x559dfc685200 2024-09-10T22:56:23.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.283+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc685200 0x559dfc6875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.285+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc685200 0x559dfc6875f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c90638c84af013f server_cookie=daf35d435315f117 in_seq=0 out_seq=0 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1b27fc640 10 client.17328 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1b27fc640 1 -- 172.21.15.96:0/3216733807 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dfc5dd8a0 con 0x559dfc685200 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1b27fc640 10 client.17328 handle_command_reply: tid=0 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 client.17328 shutdown 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 msgr2=0x559dfc675210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc02ac00 tx=0x7fc1bc003820 comp rx=0 tx=0).stop 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 shutdown_connections 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc685200 0x559dfc6875f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc672660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc672be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 --2- 172.21.15.96:0/3216733807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc675210 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 >> 172.21.15.96:0/3216733807 conn(0x559dfc666f20 msgr2=0x559dfc66ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1b27fc640 0 client.17328 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:56:23.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 shutdown_connections 2024-09-10T22:56:23.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.286+0000 7fc1d3614740 1 -- 172.21.15.96:0/3216733807 wait complete. 2024-09-10T22:56:23.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.287+0000 7fc1d3614740 20 client.17328 trim_cache size 0 max 16384 2024-09-10T22:56:23.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.304+0000 7fc1d3614740 1 Processor -- start 2024-09-10T22:56:23.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.304+0000 7fc1d3614740 1 -- start start 2024-09-10T22:56:23.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc66c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc66ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b0460 con 0x559dfc6690f0 2024-09-10T22:56:23.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b1fa0 con 0x559dfc669a10 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1d3614740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b0b30 con 0x559dfc6759a0 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48224/0 (socket says 172.21.15.96:48224) 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 -- 172.21.15.96:0/2431402868 learned_addr learned my addr 172.21.15.96:0/2431402868 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc66c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc66ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 -- 172.21.15.96:0/2431402868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 msgr2=0x559dfc66ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc66ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 -- 172.21.15.96:0/2431402868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 msgr2=0x559dfc66c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc66c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 -- 172.21.15.96:0/2431402868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfc669840 con 0x559dfc6690f0 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc66ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:23.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc001eb0 tx=0x7fc1bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353a260f2449673b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.305+0000 7fc1c0ff9640 1 -- 172.21.15.96:0/2431402868 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc035070 con 0x559dfc6690f0 2024-09-10T22:56:23.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.306+0000 7fc1c0ff9640 1 -- 172.21.15.96:0/2431402868 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc004f10 con 0x559dfc6690f0 2024-09-10T22:56:23.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.306+0000 7fc1c0ff9640 1 -- 172.21.15.96:0/2431402868 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0053b0 con 0x559dfc6690f0 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.306+0000 7fc1d3614740 1 -- 172.21.15.96:0/2431402868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 msgr2=0x559dfc666760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.306+0000 7fc1d3614740 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc001eb0 tx=0x7fc1bc000bc0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 -- 172.21.15.96:0/2431402868 shutdown_connections 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc669a10 0x559dfc66c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc66ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 --2- 172.21.15.96:0/2431402868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc6690f0 0x559dfc666760 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 -- 172.21.15.96:0/2431402868 >> 172.21.15.96:0/2431402868 conn(0x559dfc666f20 msgr2=0x559dfc66c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 -- 172.21.15.96:0/2431402868 shutdown_connections 2024-09-10T22:56:23.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.308+0000 7fc1d3614740 1 -- 172.21.15.96:0/2431402868 wait complete. 2024-09-10T22:56:23.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.309+0000 7fc1d3614740 1 -- start start 2024-09-10T22:56:23.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.309+0000 7fc1d3614740 1 Processor -- start 2024-09-10T22:56:23.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.309+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.309+0000 7fc1d3614740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc669a10 0x559dfc679620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.309+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1cad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35100/0 (socket says 172.21.15.96:35100) 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1cad9d640 1 -- 172.21.15.96:0/4139137103 learned_addr learned my addr 172.21.15.96:0/4139137103 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc679c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b18d0 con 0x559dfc669a10 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b1200 con 0x559dfc6690f0 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1cb59e640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc679c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ae440 con 0x559dfc6759a0 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1d3614740 10 client.0 _ll_register_callbacks cb 0x7fc1b400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:23.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.310+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc669a10 0x559dfc679620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 -- 172.21.15.96:0/4139137103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 msgr2=0x559dfc679c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc679c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 -- 172.21.15.96:0/4139137103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc669a10 msgr2=0x559dfc679620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc669a10 0x559dfc679620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfc6f0e20 con 0x559dfc6690f0 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1cad9d640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0059d0 tx=0x7fc1bc0042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7aa3a1532fa115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:23.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.311+0000 7fc1b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc035040 con 0x559dfc6690f0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc1bc003c90 con 0x559dfc6690f0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc004f10 con 0x559dfc6690f0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc003e90 con 0x559dfc6690f0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 10 client.26927 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dfc6f1f70 con 0x559dfc6690f0 2024-09-10T22:56:23.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc005360 con 0x559dfc6690f0 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc1bc072050 con 0x559dfc6690f0 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 10 client.26927 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 10 client.26927 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.314+0000 7fc1d3614740 20 client.26927 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.315+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc6f23f0 0x559dfc6f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.315+0000 7fc1d3614740 4 client.26927 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.315+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc1bc004f10 con 0x559dfc6f23f0 2024-09-10T22:56:23.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.315+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc6f23f0 0x559dfc6f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.316+0000 7fc1c3fff640 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc6f23f0 0x559dfc6f47e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b15a355b6d64020f server_cookie=9f3afe00b70a5e2a in_seq=0 out_seq=0 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.316+0000 7fc1b27fc640 10 client.26927 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.316+0000 7fc1b27fc640 1 -- 172.21.15.96:0/4139137103 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc1bc004f10 con 0x559dfc6f23f0 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.316+0000 7fc1b27fc640 10 client.26927 handle_command_reply: tid=0 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.316+0000 7fc1d3614740 1 client.26927 shutdown 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 msgr2=0x559dfc67e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0059d0 tx=0x7fc1bc0042c0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 shutdown_connections 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x559dfc6f23f0 0x559dfc6f47e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559dfc6759a0 0x559dfc679c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dfc669a10 0x559dfc679620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 --2- 172.21.15.96:0/4139137103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfc6690f0 0x559dfc67e150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 >> 172.21.15.96:0/4139137103 conn(0x559dfc666f20 msgr2=0x559dfc66c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1b27fc640 0 client.26927 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 shutdown_connections 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.317+0000 7fc1d3614740 1 -- 172.21.15.96:0/4139137103 wait complete. 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d3614740 20 client.26927 trim_cache size 0 max 16384 2024-09-10T22:56:23.323 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 msgr2=0x7fc1b80b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 0x7fc1b80b7860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400bfd0 tx=0x7fc1b400b390 comp rx=0 tx=0).stop 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 msgr2=0x7fc1cc1a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40099e0 tx=0x7fc1c4009ea0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 shutdown_connections 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc1b80b53a0 0x7fc1b80b7860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1cc107800 0x7fc1cc1a0e30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1cc106e60 0x7fc1cc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 --2- 172.21.15.96:0/2196281677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1cc101450 0x7fc1cc19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:23.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.318+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 >> 172.21.15.96:0/2196281677 conn(0x7fc1cc078140 msgr2=0x7fc1cc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:23.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.319+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 shutdown_connections 2024-09-10T22:56:23.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:23.319+0000 7fc1d123f640 1 -- 172.21.15.96:0/2196281677 wait complete. 2024-09-10T22:56:23.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:23 smithi096 ceph-mon[30869]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 828 op/s 2024-09-10T22:56:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:23 smithi121 ceph-mon[35119]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 828 op/s 2024-09-10T22:56:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:23 smithi117 ceph-mon[35081]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 828 op/s 2024-09-10T22:56:24.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.168+0000 7fca8339b640 1 -- 172.21.15.96:0/472176612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c1029b0 msgr2=0x7fca7c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.168+0000 7fca8339b640 1 --2- 172.21.15.96:0/472176612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c1029b0 0x7fca7c102e30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009900 tx=0x7fca6c02a410 comp rx=0 tx=0).stop 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 -- 172.21.15.96:0/472176612 shutdown_connections 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 --2- 172.21.15.96:0/472176612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c103370 0x7fca7c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 --2- 172.21.15.96:0/472176612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c1029b0 0x7fca7c102e30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 --2- 172.21.15.96:0/472176612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c101860 0x7fca7c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 -- 172.21.15.96:0/472176612 >> 172.21.15.96:0/472176612 conn(0x7fca7c0fd140 msgr2=0x7fca7c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 -- 172.21.15.96:0/472176612 shutdown_connections 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 -- 172.21.15.96:0/472176612 wait complete. 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 Processor -- start 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.169+0000 7fca8339b640 1 -- start start 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c101860 0x7fca7c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c103370 0x7fca7c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c110850 con 0x7fca7c103370 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c10c2a0 con 0x7fca7c101860 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca8339b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1109d0 con 0x7fca7c1029b0 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51500/0 (socket says 172.21.15.96:51500) 2024-09-10T22:56:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 -- 172.21.15.96:0/473198035 learned_addr learned my addr 172.21.15.96:0/473198035 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca81137640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c101860 0x7fca7c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca81938640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c103370 0x7fca7c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 -- 172.21.15.96:0/473198035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c101860 msgr2=0x7fca7c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c101860 0x7fca7c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 -- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c103370 msgr2=0x7fca7c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c103370 0x7fca7c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 -- 172.21.15.96:0/473198035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7c19cac0 con 0x7fca7c1029b0 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.170+0000 7fca80936640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005ab0 tx=0x7fca6c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.171+0000 7fca727fc640 1 -- 172.21.15.96:0/473198035 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c03f070 con 0x7fca7c1029b0 2024-09-10T22:56:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.171+0000 7fca727fc640 1 -- 172.21.15.96:0/473198035 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c031c50 con 0x7fca7c1029b0 2024-09-10T22:56:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.172+0000 7fca727fc640 1 -- 172.21.15.96:0/473198035 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c035030 con 0x7fca7c1029b0 2024-09-10T22:56:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.172+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7c19cd50 con 0x7fca7c1029b0 2024-09-10T22:56:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.173+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7c10c440 con 0x7fca7c1029b0 2024-09-10T22:56:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.175+0000 7fca86247740 1 Processor -- start 2024-09-10T22:56:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.175+0000 7fca86247740 1 -- start start 2024-09-10T22:56:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e52abf6f0 con 0x564e52b4d860 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e52aaeb20 con 0x564e52b50190 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca86247740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564e52ab2930 con 0x564e52b52bd0 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca81938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca81938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48266/0 (socket says 172.21.15.96:48266) 2024-09-10T22:56:24.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca81938640 1 -- 172.21.15.96:0/767198231 learned_addr learned my addr 172.21.15.96:0/767198231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.176+0000 7fca81137640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca80936640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 -- 172.21.15.96:0/767198231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 msgr2=0x564e52b55090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b55090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 -- 172.21.15.96:0/767198231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 msgr2=0x564e52b4fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 -- 172.21.15.96:0/767198231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e52a74530 con 0x564e52b50190 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca727fc640 1 -- 172.21.15.96:0/473198035 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fca6c0317a0 con 0x7fca7c1029b0 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca80936640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81938640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca81137640 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b52580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fca78009c20 tx=0x7fca78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e018dce18204f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca677fe640 1 -- 172.21.15.96:0/767198231 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78009dd0 con 0x564e52b50190 2024-09-10T22:56:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca677fe640 1 -- 172.21.15.96:0/767198231 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca78005020 con 0x564e52b50190 2024-09-10T22:56:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.177+0000 7fca677fe640 1 -- 172.21.15.96:0/767198231 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78004b40 con 0x564e52b50190 2024-09-10T22:56:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca727fc640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 0x7fca680b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca81938640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 0x7fca680b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 -- 172.21.15.96:0/767198231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 msgr2=0x564e52b52580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b52580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fca78009c20 tx=0x7fca78011810 comp rx=0 tx=0).stop 2024-09-10T22:56:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca727fc640 1 -- 172.21.15.96:0/473198035 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca6c106720 con 0x7fca7c1029b0 2024-09-10T22:56:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 -- 172.21.15.96:0/767198231 shutdown_connections 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b52580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 --2- 172.21.15.96:0/767198231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b4fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 -- 172.21.15.96:0/767198231 >> 172.21.15.96:0/767198231 conn(0x564e52b489b0 msgr2=0x564e52b49740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 -- 172.21.15.96:0/767198231 shutdown_connections 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.178+0000 7fca86247740 1 -- 172.21.15.96:0/767198231 wait complete. 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca86247740 1 -- start start 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca81938640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 0x7fca680b7840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca740099a0 tx=0x7fca74009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca86247740 1 Processor -- start 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca86247740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca86247740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.179+0000 7fca86247740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 0x564e52b58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca81137640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51520/0 (socket says 172.21.15.96:51520) 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 -- 172.21.15.96:0/23630676 learned_addr learned my addr 172.21.15.96:0/23630676 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564e529e6ab0 con 0x564e52b4d860 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e52a54730 con 0x564e52b52bd0 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564e52b55bb0 con 0x564e52b50190 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca81938640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 0x564e52b58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca86247740 10 client.0 _ll_register_callbacks cb 0x564e52aa86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 -- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 msgr2=0x564e52b58130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 0x564e52b58130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 -- 172.21.15.96:0/23630676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 msgr2=0x564e52b56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e52810790 con 0x564e52b50190 2024-09-10T22:56:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca81938640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 0x564e52b58130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca80936640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0c94b0 tx=0x7fca6c03b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73bbb0cc7fbb7f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0cc830 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.180+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca6c0cd140 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.181+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca78005020 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.181+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca6c0cc650 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca86247740 10 client.27007 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564e52889620 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca78005020 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca6c0b80f0 con 0x564e52b50190 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca86247740 10 client.27007 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca86247740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.182+0000 7fca86247740 20 client.27007 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.183+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52b64b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.183+0000 7fca86247740 4 client.27007 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.183+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e52ac1640 con 0x564e52b62700 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.183+0000 7fca81137640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52b64b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.184+0000 7fca64ff9640 10 client.27007 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.184+0000 7fca81137640 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52b64b10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9129b041cb411986 server_cookie=92e287abe0ea9dee in_seq=0 out_seq=0 2024-09-10T22:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca64ff9640 1 -- 172.21.15.96:0/23630676 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e52ac1640 con 0x564e52b62700 2024-09-10T22:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca64ff9640 10 client.27007 handle_command_reply: tid=0 2024-09-10T22:56:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 client.27007 shutdown 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 msgr2=0x564e52b57560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0c94b0 tx=0x7fca6c03b690 comp rx=0 tx=0).stop 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 shutdown_connections 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52b64b10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca64ff9640 0 client.27007 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b52bd0 0x564e52b58130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b57560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 --2- 172.21.15.96:0/23630676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 >> 172.21.15.96:0/23630676 conn(0x564e52b489b0 msgr2=0x564e52b47bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 shutdown_connections 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 1 -- 172.21.15.96:0/23630676 wait complete. 2024-09-10T22:56:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.185+0000 7fca86247740 20 client.27007 trim_cache size 0 max 16384 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.206+0000 7fca86247740 1 Processor -- start 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.206+0000 7fca86247740 1 -- start start 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.206+0000 7fca86247740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 0x564e52b58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca86247740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b52fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca86247740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca86247740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a79c0 con 0x564e52b52bd0 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca86247740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a9500 con 0x564e52b4d860 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca86247740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a8090 con 0x564e52b50190 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48276/0 (socket says 172.21.15.96:48276) 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 0x564e52b58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81938640 1 -- 172.21.15.96:0/2092061245 learned_addr learned my addr 172.21.15.96:0/2092061245 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca80936640 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b52fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 -- 172.21.15.96:0/2092061245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 msgr2=0x564e52b52fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b52fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 -- 172.21.15.96:0/2092061245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 msgr2=0x564e52b534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 -- 172.21.15.96:0/2092061245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e52b49450 con 0x564e52b4d860 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81938640 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.207+0000 7fca81137640 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 0x564e52b58a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca78004770 tx=0x7fca78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416c9ed57eaeb5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca66ffd640 1 -- 172.21.15.96:0/2092061245 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780053a0 con 0x564e52b4d860 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca66ffd640 1 -- 172.21.15.96:0/2092061245 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca7802f030 con 0x564e52b4d860 2024-09-10T22:56:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca66ffd640 1 -- 172.21.15.96:0/2092061245 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78024410 con 0x564e52b4d860 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 -- 172.21.15.96:0/2092061245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 msgr2=0x564e52b58a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 0x564e52b58a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca78004770 tx=0x7fca78011810 comp rx=0 tx=0).stop 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 -- 172.21.15.96:0/2092061245 shutdown_connections 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b52bd0 0x564e52b534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b50190 0x564e52b52fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 --2- 172.21.15.96:0/2092061245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b4d860 0x564e52b58a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 -- 172.21.15.96:0/2092061245 >> 172.21.15.96:0/2092061245 conn(0x564e52b489b0 msgr2=0x564e52b4ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.208+0000 7fca86247740 1 -- 172.21.15.96:0/2092061245 shutdown_connections 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.209+0000 7fca86247740 1 -- 172.21.15.96:0/2092061245 wait complete. 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.209+0000 7fca86247740 1 -- start start 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.209+0000 7fca86247740 1 Processor -- start 2024-09-10T22:56:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b5bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a8e30 con 0x564e52b4d860 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a8760 con 0x564e52b50190 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca7c1a59a0 con 0x564e52b52bd0 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca80936640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca80936640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35148/0 (socket says 172.21.15.96:35148) 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca80936640 1 -- 172.21.15.96:0/3272580675 learned_addr learned my addr 172.21.15.96:0/3272580675 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.210+0000 7fca86247740 10 client.0 _ll_register_callbacks cb 0x7fca6c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca81938640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 -- 172.21.15.96:0/3272580675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 msgr2=0x564e52b5cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b5cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 -- 172.21.15.96:0/3272580675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 msgr2=0x564e52b5bc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:56:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca81137640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b5bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b5bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 -- 172.21.15.96:0/3272580675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e52b5d720 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca80936640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0d6630 tx=0x7fca6c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb042fe9d3f5dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.211+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0cc830 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.212+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fca6c0cc140 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.212+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7802f030 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.212+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca6c0c8d80 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.212+0000 7fca86247740 10 client.26945 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.212+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564e52b5eae0 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c0ccee0 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fca6c0ee2f0 con 0x564e52b50190 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 10 client.26945 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 10 client.26945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 20 client.26945 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52bd1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 4 client.26945 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fca7802f030 con 0x564e52b62700 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.213+0000 7fca81137640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52bd1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.214+0000 7fca81137640 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52bd1680 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2da31d65a96f6bf server_cookie=72a019bc0f74622a in_seq=0 out_seq=0 2024-09-10T22:56:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.214+0000 7fca64ff9640 10 client.26945 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.232+0000 7fca64ff9640 1 -- 172.21.15.96:0/3272580675 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fca7802f030 con 0x564e52b62700 2024-09-10T22:56:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.232+0000 7fca64ff9640 10 client.26945 handle_command_reply: tid=0 2024-09-10T22:56:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.232+0000 7fca86247740 1 client.26945 shutdown 2024-09-10T22:56:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.233+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 msgr2=0x564e52b5c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.233+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0d6630 tx=0x7fca6c005c90 comp rx=0 tx=0).stop 2024-09-10T22:56:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 shutdown_connections 2024-09-10T22:56:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564e52b62700 0x564e52bd1680 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca64ff9640 0 client.26945 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e52b50190 0x564e52b5c320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e52b52bd0 0x564e52b5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 --2- 172.21.15.96:0/3272580675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564e52b4d860 0x564e52b5bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.237+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 >> 172.21.15.96:0/3272580675 conn(0x564e52b489b0 msgr2=0x564e52b4c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.243+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 shutdown_connections 2024-09-10T22:56:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.247+0000 7fca86247740 1 -- 172.21.15.96:0/3272580675 wait complete. 2024-09-10T22:56:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.249+0000 7fca86247740 20 client.26945 trim_cache size 0 max 16384 2024-09-10T22:56:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:56:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:56:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:56:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.258+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 msgr2=0x7fca680b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.258+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 0x7fca680b7840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca740099a0 tx=0x7fca74009500 comp rx=0 tx=0).stop 2024-09-10T22:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.258+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 msgr2=0x7fca7c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.258+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca6c005ab0 tx=0x7fca6c003b90 comp rx=0 tx=0).stop 2024-09-10T22:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 shutdown_connections 2024-09-10T22:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fca680b5380 0x7fca680b7840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca7c103370 0x7fca7c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca7c1029b0 0x7fca7c19b5d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 --2- 172.21.15.96:0/473198035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca7c101860 0x7fca7c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.262+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 >> 172.21.15.96:0/473198035 conn(0x7fca7c0fd140 msgr2=0x7fca7c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.266+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 shutdown_connections 2024-09-10T22:56:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:24.270+0000 7fca8339b640 1 -- 172.21.15.96:0/473198035 wait complete. 2024-09-10T22:56:24.501 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:25 smithi096 ceph-mon[30869]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 876 op/s 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 -- 172.21.15.96:0/1221094429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 msgr2=0x7f3aa0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/1221094429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0102070 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009900 tx=0x7f3a9c02c600 comp rx=0 tx=0).stop 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 -- 172.21.15.96:0/1221094429 shutdown_connections 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/1221094429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/1221094429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 0x7f3aa010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/1221094429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0102070 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 -- 172.21.15.96:0/1221094429 >> 172.21.15.96:0/1221094429 conn(0x7f3aa00fd700 msgr2=0x7f3aa00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 -- 172.21.15.96:0/1221094429 shutdown_connections 2024-09-10T22:56:25.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.550+0000 7f3aa73d3640 1 -- 172.21.15.96:0/1221094429 wait complete. 2024-09-10T22:56:25.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 Processor -- start 2024-09-10T22:56:25.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 -- start start 2024-09-10T22:56:25.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 0x7f3aa019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0110c20 con 0x7f3aa0101c90 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0110aa0 con 0x7f3aa01076a0 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa73d3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0110da0 con 0x7f3aa0108020 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48298/0 (socket says 172.21.15.96:48298) 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa496e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.551+0000 7f3aa516f640 1 -- 172.21.15.96:0/2026997976 learned_addr learned my addr 172.21.15.96:0/2026997976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:25.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa5970640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 0x7f3aa019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 -- 172.21.15.96:0/2026997976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 msgr2=0x7f3aa019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 0x7f3aa019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 -- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 msgr2=0x7f3aa0197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 -- 172.21.15.96:0/2026997976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa019d1e0 con 0x7f3aa01076a0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa516f640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.552+0000 7f3aa496e640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0197670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d6d0 tx=0x7f3a9000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.553+0000 7f3a967fc640 1 -- 172.21.15.96:0/2026997976 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x7f3aa01076a0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.553+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa019d4d0 con 0x7f3aa01076a0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.553+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa01a3d60 con 0x7f3aa01076a0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.553+0000 7f3a967fc640 1 -- 172.21.15.96:0/2026997976 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a90009820 con 0x7f3aa01076a0 2024-09-10T22:56:25.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.553+0000 7f3a967fc640 1 -- 172.21.15.96:0/2026997976 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004c40 con 0x7f3aa01076a0 2024-09-10T22:56:25.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.556+0000 7f3aa97a8740 1 Processor -- start 2024-09-10T22:56:25.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.556+0000 7f3aa97a8740 1 -- start start 2024-09-10T22:56:25.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.556+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589f445fea0 con 0x5589f44f5d80 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589f444f2d0 con 0x5589f44e9df0 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589f44530e0 con 0x5589f44e94d0 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3a967fc640 1 -- 172.21.15.96:0/2026997976 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3a90005220 con 0x7f3aa01076a0 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35174/0 (socket says 172.21.15.96:35174) 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 -- 172.21.15.96:0/4148768788 learned_addr learned my addr 172.21.15.96:0/4148768788 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:25.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa5970640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa496e640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 -- 172.21.15.96:0/4148768788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 msgr2=0x5589f44e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 -- 172.21.15.96:0/4148768788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 msgr2=0x5589f44f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 -- 172.21.15.96:0/4148768788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f41ad650 con 0x5589f44e9df0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa496e640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.557+0000 7f3aa516f640 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006010 tx=0x7f3a9c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0118b89df7dbc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3a8b7fe640 1 -- 172.21.15.96:0/4148768788 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c042070 con 0x5589f44e9df0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3a8b7fe640 1 -- 172.21.15.96:0/4148768788 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c003860 con 0x5589f44e9df0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3a8b7fe640 1 -- 172.21.15.96:0/4148768788 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c037040 con 0x5589f44e9df0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3a967fc640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 0x7f3a8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3aa5970640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 0x7f3a8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.558+0000 7f3a967fc640 1 -- 172.21.15.96:0/2026997976 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3a900e51f0 con 0x7f3aa01076a0 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa5970640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 0x7f3a8c0b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3a98005fd0 tx=0x7f3a980096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- 172.21.15.96:0/4148768788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 msgr2=0x5589f44f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006010 tx=0x7f3a9c0045f0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- 172.21.15.96:0/4148768788 shutdown_connections 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44f5840 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/4148768788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- 172.21.15.96:0/4148768788 >> 172.21.15.96:0/4148768788 conn(0x5589f44e7300 msgr2=0x5589f44e84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- 172.21.15.96:0/4148768788 shutdown_connections 2024-09-10T22:56:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- 172.21.15.96:0/4148768788 wait complete. 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.559+0000 7f3aa97a8740 1 -- start start 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.560+0000 7f3aa97a8740 1 Processor -- start 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f44f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589f42b0700 con 0x5589f44e9df0 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589f43ea160 con 0x5589f44e94d0 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f44f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa496e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f44f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35176/0 (socket says 172.21.15.96:35176) 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa516f640 1 -- 172.21.15.96:0/1831986081 learned_addr learned my addr 172.21.15.96:0/1831986081 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.561+0000 7f3aa496e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48322/0 (socket says 172.21.15.96:48322) 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589f44f89a0 con 0x5589f44f5d80 2024-09-10T22:56:25.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa97a8740 10 client.0 _ll_register_callbacks cb 0x5589f44039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa496e640 1 -- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 msgr2=0x5589f44f2f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa496e640 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44f2f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa496e640 1 -- 172.21.15.96:0/1831986081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 msgr2=0x5589f44f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa496e640 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f44f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.562+0000 7f3aa496e640 1 -- 172.21.15.96:0/1831986081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f4417c90 con 0x5589f44e9df0 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.563+0000 7f3aa496e640 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3a900e2de0 tx=0x7f3a900ab270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d26c05ca1d0e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.563+0000 7f3a88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.563+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90020050 con 0x5589f44e9df0 2024-09-10T22:56:25.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.563+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3a900aad60 con 0x5589f44e9df0 2024-09-10T22:56:25.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.563+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c003860 con 0x5589f44e9df0 2024-09-10T22:56:25.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a90005740 con 0x5589f44e9df0 2024-09-10T22:56:25.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 10 client.17358 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:25.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589f4504290 con 0x5589f44e9df0 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c003860 con 0x5589f44e9df0 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3a900b01a0 con 0x5589f44e9df0 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 10 client.17358 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 10 client.17358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 20 client.17358 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f4507b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 4 client.17358 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.564+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589f4461df0 con 0x5589f45057a0 2024-09-10T22:56:25.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.565+0000 7f3aa516f640 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f4507b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.565+0000 7f3aa516f640 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f4507b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fea75fc04265067e server_cookie=b4c341c1ab19cd0a in_seq=0 out_seq=0 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.565+0000 7f3a88ff9640 10 client.17358 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3a88ff9640 1 -- 172.21.15.96:0/1831986081 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589f4461df0 con 0x5589f45057a0 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3a88ff9640 10 client.17358 handle_command_reply: tid=0 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3aa97a8740 1 client.17358 shutdown 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 msgr2=0x5589f44f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3a900e2de0 tx=0x7f3a900ab270 comp rx=0 tx=0).stop 2024-09-10T22:56:25.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.566+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 shutdown_connections 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f4507b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3a88ff9640 0 client.17358 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44f2a10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/1831986081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f44f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 >> 172.21.15.96:0/1831986081 conn(0x5589f44e7300 msgr2=0x5589f44ece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.567+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 shutdown_connections 2024-09-10T22:56:25.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.568+0000 7f3aa97a8740 1 -- 172.21.15.96:0/1831986081 wait complete. 2024-09-10T22:56:25.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.568+0000 7f3aa97a8740 20 client.17358 trim_cache size 0 max 16384 2024-09-10T22:56:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 Processor -- start 2024-09-10T22:56:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 -- start start 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f4501c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44e6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400e390 con 0x5589f44e9df0 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400fed0 con 0x5589f44e94d0 2024-09-10T22:56:25.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa97a8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400ea60 con 0x5589f44f5d80 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa496e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f4501c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa496e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48330/0 (socket says 172.21.15.96:48330) 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.587+0000 7f3aa496e640 1 -- 172.21.15.96:0/3081140859 learned_addr learned my addr 172.21.15.96:0/3081140859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa5970640 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 -- 172.21.15.96:0/3081140859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 msgr2=0x5589f44efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 -- 172.21.15.96:0/3081140859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 msgr2=0x5589f44e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 -- 172.21.15.96:0/3081140859 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f44e7d80 con 0x5589f44e94d0 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f4501c60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5589f45083e0 tx=0x7f3a9c02c600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296d1c1fceeeed65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3a8affd640 1 -- 172.21.15.96:0/3081140859 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c037040 con 0x5589f44e94d0 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3a8affd640 1 -- 172.21.15.96:0/3081140859 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c0054d0 con 0x5589f44e94d0 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.588+0000 7f3a8affd640 1 -- 172.21.15.96:0/3081140859 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c042040 con 0x5589f44e94d0 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3081140859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 msgr2=0x5589f4501c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f4501c60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5589f45083e0 tx=0x7f3a9c02c600 comp rx=0 tx=0).stop 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3081140859 shutdown_connections 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44f5d80 0x5589f44efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44e9df0 0x5589f44e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3081140859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e94d0 0x5589f4501c60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3081140859 >> 172.21.15.96:0/3081140859 conn(0x5589f44e7300 msgr2=0x5589f44e5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3081140859 shutdown_connections 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3081140859 wait complete. 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.589+0000 7f3aa97a8740 1 -- start start 2024-09-10T22:56:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 Processor -- start 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa516f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51588/0 (socket says 172.21.15.96:51588) 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa516f640 1 -- 172.21.15.96:0/3157993161 learned_addr learned my addr 172.21.15.96:0/3157993161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400f800 con 0x5589f44f5d80 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400f130 con 0x5589f44e9df0 2024-09-10T22:56:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa496e640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400c370 con 0x5589f44e94d0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.590+0000 7f3aa5970640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa97a8740 10 client.0 _ll_register_callbacks cb 0x7f3a90009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 -- 172.21.15.96:0/3157993161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 msgr2=0x5589f44fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 -- 172.21.15.96:0/3157993161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 msgr2=0x5589f44facd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44facd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f44e7000 con 0x5589f44e94d0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa496e640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3aa516f640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c033440 tx=0x7f3a9c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8706bc727a6366a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3a88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c042030 con 0x5589f44e94d0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.591+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3a9c004c80 con 0x5589f44e94d0 2024-09-10T22:56:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.592+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c0054d0 con 0x5589f44e94d0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.592+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c037040 con 0x5589f44e94d0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.592+0000 7f3aa97a8740 10 client.27025 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.592+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589f4573340 con 0x5589f44e94d0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a9c0054d0 con 0x5589f44e94d0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3a9c033960 con 0x5589f44e94d0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 10 client.27025 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 10 client.27025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 20 client.27025 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f45755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 4 client.27025 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589f445d8a0 con 0x5589f45057a0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3aa496e640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f45755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.593+0000 7f3a88ff9640 10 client.27025 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:56:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa496e640 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f45755d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a836df64b43fe23 server_cookie=4677c6fd29744b03 in_seq=0 out_seq=0 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3a88ff9640 1 -- 172.21.15.96:0/3157993161 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589f445d8a0 con 0x5589f45057a0 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3a88ff9640 10 client.27025 handle_command_reply: tid=0 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 client.27025 shutdown 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 msgr2=0x5589f44fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c033440 tx=0x7f3a9c005a20 comp rx=0 tx=0).stop 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 shutdown_connections 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x5589f45057a0 0x5589f45755d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3a88ff9640 0 client.27025 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589f44e9df0 0x5589f44fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589f44f5d80 0x5589f44facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 --2- 172.21.15.96:0/3157993161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589f44e94d0 0x5589f44fe2b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.594+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 >> 172.21.15.96:0/3157993161 conn(0x5589f44e7300 msgr2=0x5589f44f4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 shutdown_connections 2024-09-10T22:56:25.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa97a8740 1 -- 172.21.15.96:0/3157993161 wait complete. 2024-09-10T22:56:25.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa97a8740 20 client.27025 trim_cache size 0 max 16384 2024-09-10T22:56:25.601 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 msgr2=0x7f3a8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 0x7f3a8c0b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3a98005fd0 tx=0x7f3a980096a0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 msgr2=0x7f3aa0197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0197670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000d6d0 tx=0x7f3a9000db90 comp rx=0 tx=0).stop 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 shutdown_connections 2024-09-10T22:56:25.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa01076a0 0x7f3aa0197670 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3a8c0b52e0 0x7f3a8c0b77a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.595+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aa0108020 0x7f3aa019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.596+0000 7f3aa73d3640 1 --2- 172.21.15.96:0/2026997976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3aa0101c90 0x7f3aa0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.596+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 >> 172.21.15.96:0/2026997976 conn(0x7f3aa00fd700 msgr2=0x7f3aa00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.596+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 shutdown_connections 2024-09-10T22:56:25.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:25.596+0000 7f3aa73d3640 1 -- 172.21.15.96:0/2026997976 wait complete. 2024-09-10T22:56:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:25 smithi121 ceph-mon[35119]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 876 op/s 2024-09-10T22:56:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:25 smithi117 ceph-mon[35081]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 876 op/s 2024-09-10T22:56:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 530.0 s, 2268.4 tps, lat 44.012 ms stddev 54.446 2024-09-10T22:56:26.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:56:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:27.539 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:27 smithi096 ceph-mon[30869]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 880 op/s 2024-09-10T22:56:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:27 smithi121 ceph-mon[35119]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 880 op/s 2024-09-10T22:56:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:27 smithi117 ceph-mon[35081]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 880 op/s 2024-09-10T22:56:28.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 -- 172.21.15.96:0/3864611377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca810d140 msgr2=0x7faca810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 --2- 172.21.15.96:0/3864611377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca810d140 0x7faca810f5a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca002a410 comp rx=0 tx=0).stop 2024-09-10T22:56:28.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 -- 172.21.15.96:0/3864611377 shutdown_connections 2024-09-10T22:56:28.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 --2- 172.21.15.96:0/3864611377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca8104600 0x7faca810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 --2- 172.21.15.96:0/3864611377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca810d140 0x7faca810f5a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 --2- 172.21.15.96:0/3864611377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 0x7faca81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.806+0000 7facae55b640 1 -- 172.21.15.96:0/3864611377 >> 172.21.15.96:0/3864611377 conn(0x7faca80fd740 msgr2=0x7faca80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.807+0000 7facae55b640 1 -- 172.21.15.96:0/3864611377 shutdown_connections 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.807+0000 7facae55b640 1 -- 172.21.15.96:0/3864611377 wait complete. 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 Processor -- start 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 -- start start 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 0x7faca819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca810d140 0x7faca81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faca8112db0 con 0x7faca8104600 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faca8112c30 con 0x7faca810d140 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7facae55b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faca8112f30 con 0x7faca8103ce0 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7faca7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 0x7faca819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7fac9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50388/0 (socket says 172.21.15.96:50388) 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.808+0000 7fac9f7fe640 1 -- 172.21.15.96:0/210886155 learned_addr learned my addr 172.21.15.96:0/210886155 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7facacaf8640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca810d140 0x7faca81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 -- 172.21.15.96:0/210886155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 msgr2=0x7faca819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 0x7faca819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 -- 172.21.15.96:0/210886155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca810d140 msgr2=0x7faca81a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca810d140 0x7faca81a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 -- 172.21.15.96:0/210886155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a1570 con 0x7faca8104600 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9c0 tx=0x7fac9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7faca5ffb640 1 -- 172.21.15.96:0/210886155 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000ccc0 con 0x7faca8104600 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a1860 con 0x7faca8104600 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7faca5ffb640 1 -- 172.21.15.96:0/210886155 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900035f0 con 0x7faca8104600 2024-09-10T22:56:28.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7faca5ffb640 1 -- 172.21.15.96:0/210886155 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000ce60 con 0x7faca8104600 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.809+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81a7f20 con 0x7faca8104600 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 Processor -- start 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 -- start start 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7faca5ffb640 1 -- 172.21.15.96:0/210886155 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fac90004000 con 0x7faca8104600 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e415840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e418170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55821e37fea0 con 0x55821e415d80 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55821e36f2d0 con 0x55821e4094d0 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.813+0000 7facb0930740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55821e3730e0 con 0x55821e409df0 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.814+0000 7facacaf8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.814+0000 7facacaf8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51296/0 (socket says 172.21.15.96:51296) 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.814+0000 7facacaf8640 1 -- 172.21.15.96:0/2368463639 learned_addr learned my addr 172.21.15.96:0/2368463639 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:28.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.814+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e418170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.814+0000 7faca7fff640 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e415840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.815+0000 7facacaf8640 1 -- 172.21.15.96:0/2368463639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 msgr2=0x55821e415840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.815+0000 7facacaf8640 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e415840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.815+0000 7facacaf8640 1 -- 172.21.15.96:0/2368463639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 msgr2=0x55821e418170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.815+0000 7facacaf8640 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e418170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.815+0000 7facacaf8640 1 -- 172.21.15.96:0/2368463639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55821e0cd650 con 0x55821e4094d0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7facacaf8640 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca0032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3e6c72798c5f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7fac9e7fc640 1 -- 172.21.15.96:0/2368463639 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca003f070 con 0x55821e4094d0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7fac9e7fc640 1 -- 172.21.15.96:0/2368463639 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca0004030 con 0x55821e4094d0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7fac9e7fc640 1 -- 172.21.15.96:0/2368463639 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0035070 con 0x55821e4094d0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7faca5ffb640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 0x7fac940b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.816+0000 7faca5ffb640 1 -- 172.21.15.96:0/210886155 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac900e5ec0 con 0x7faca8104600 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7faca7fff640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 0x7fac940b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7facb0930740 1 -- 172.21.15.96:0/2368463639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 msgr2=0x55821e4098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7facb0930740 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca0032b20 comp rx=0 tx=0).stop 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7facb0930740 1 -- 172.21.15.96:0/2368463639 shutdown_connections 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7facb0930740 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e418170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.817+0000 7facb0930740 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e415840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7facb0930740 1 --2- 172.21.15.96:0/2368463639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e4098b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7facb0930740 1 -- 172.21.15.96:0/2368463639 >> 172.21.15.96:0/2368463639 conn(0x55821e407300 msgr2=0x55821e4084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7facb0930740 1 -- 172.21.15.96:0/2368463639 shutdown_connections 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7facb0930740 1 -- 172.21.15.96:0/2368463639 wait complete. 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7faca7fff640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 0x7fac940b78a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55821e4081e0 tx=0x7fac98009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.818+0000 7facb0930740 1 -- start start 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 Processor -- start 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e415670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e412ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55821e1d0700 con 0x55821e415d80 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55821e30a160 con 0x55821e4094d0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55821e4189a0 con 0x55821e409df0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.819+0000 7facb0930740 10 client.0 _ll_register_callbacks cb 0x55821e3239e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50396/0 (socket says 172.21.15.96:50396) 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 -- 172.21.15.96:0/1068937138 learned_addr learned my addr 172.21.15.96:0/1068937138 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 -- 172.21.15.96:0/1068937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 msgr2=0x55821e412ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7faca7fff640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e415670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e412ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e412ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.820+0000 7facacaf8640 1 -- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 msgr2=0x55821e415670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.821+0000 7facacaf8640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e415670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.821+0000 7facacaf8640 1 -- 172.21.15.96:0/1068937138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55821e337c90 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.821+0000 7faca7fff640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e415670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.821+0000 7facacaf8640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faca00051b0 tx=0x7faca0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c366ff3234018d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.821+0000 7fac877fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0035070 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca0004030 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faca00316a0 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faca003f040 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7facb0930740 10 client.17376 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55821e424330 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.822+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca0004030 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faca0004730 con 0x55821e415d80 2024-09-10T22:56:28.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 10 client.17376 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 10 client.17376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 20 client.17376 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e427a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 4 client.17376 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.823+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55821e381df0 con 0x55821e425680 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.827+0000 7faca7fff640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e427a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.831+0000 7faca7fff640 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e427a70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a9795a1c16383c2 server_cookie=91bf1e816ddd76fa in_seq=0 out_seq=0 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.831+0000 7fac877fe640 10 client.17376 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.833+0000 7fac877fe640 1 -- 172.21.15.96:0/1068937138 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55821e381df0 con 0x55821e425680 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.833+0000 7fac877fe640 10 client.17376 handle_command_reply: tid=0 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.833+0000 7facb0930740 1 client.17376 shutdown 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 msgr2=0x55821e4130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faca00051b0 tx=0x7faca0004610 comp rx=0 tx=0).stop 2024-09-10T22:56:28.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 shutdown_connections 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e427a70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e415d80 0x55821e4130f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7fac877fe640 0 client.17376 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.849+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e409df0 0x55821e412ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.850+0000 7facb0930740 1 --2- 172.21.15.96:0/1068937138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e415670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.850+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 >> 172.21.15.96:0/1068937138 conn(0x55821e407300 msgr2=0x55821e40ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.852+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 shutdown_connections 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.855+0000 7facb0930740 1 -- 172.21.15.96:0/1068937138 wait complete. 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.856+0000 7facb0930740 20 client.17376 trim_cache size 0 max 16384 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.891+0000 7facb0930740 1 Processor -- start 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.891+0000 7facb0930740 1 -- start start 2024-09-10T22:56:28.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.891+0000 7facb0930740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.891+0000 7facb0930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 0x55821e417170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7faca7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7faca7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51304/0 (socket says 172.21.15.96:51304) 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7faca7fff640 1 -- 172.21.15.96:0/114924847 learned_addr learned my addr 172.21.15.96:0/114924847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7facb0930740 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e4176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b05b0 con 0x55821e409df0 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b20f0 con 0x55821e4094d0 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 0x55821e417170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7facacaf8640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e4176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.892+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b0c80 con 0x55821e415d80 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 -- 172.21.15.96:0/114924847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 msgr2=0x55821e4176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e4176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 -- 172.21.15.96:0/114924847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 msgr2=0x55821e406310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 -- 172.21.15.96:0/114924847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55821e410fc0 con 0x55821e409df0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 0x55821e417170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac900a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f533f600788c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7faca7fff640 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9dffb640 1 -- 172.21.15.96:0/114924847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900a8cd0 con 0x55821e409df0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9dffb640 1 -- 172.21.15.96:0/114924847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac900ac190 con 0x55821e409df0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.893+0000 7fac9dffb640 1 -- 172.21.15.96:0/114924847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900abdf0 con 0x55821e409df0 2024-09-10T22:56:28.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.894+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 msgr2=0x55821e417170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.894+0000 7facb0930740 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 0x55821e417170 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac900a86e0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.895+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 shutdown_connections 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.895+0000 7facb0930740 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e4176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.895+0000 7facb0930740 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e409df0 0x55821e417170 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.895+0000 7facb0930740 1 --2- 172.21.15.96:0/114924847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e4094d0 0x55821e406310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.895+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 >> 172.21.15.96:0/114924847 conn(0x55821e407300 msgr2=0x55821e405c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.896+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 shutdown_connections 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.896+0000 7facb0930740 1 -- 172.21.15.96:0/114924847 wait complete. 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.896+0000 7facb0930740 1 -- start start 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 Processor -- start 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 0x55821e4274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 0x55821e427a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e493b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b1a20 con 0x55821e4094d0 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b1350 con 0x55821e409df0 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ae5e0 con 0x55821e415d80 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7facb0930740 10 client.0 _ll_register_callbacks cb 0x7fac900e9750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7faca7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 0x55821e4274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7faca7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 0x55821e4274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50408/0 (socket says 172.21.15.96:50408) 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7faca7fff640 1 -- 172.21.15.96:0/3295183208 learned_addr learned my addr 172.21.15.96:0/3295183208 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.897+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 0x55821e427a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 -- 172.21.15.96:0/3295183208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 msgr2=0x55821e493b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e493b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 -- 172.21.15.96:0/3295183208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 msgr2=0x55821e4274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 0x55821e4274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 -- 172.21.15.96:0/3295183208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55821e494250 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac9f7fe640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 0x55821e427a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac900b5cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6ff638975ec58a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac877fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900abfd0 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.898+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac900ab8e0 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.899+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac900ac190 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.899+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac900ac450 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.899+0000 7facb0930740 10 client.26978 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.899+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55821e49b620 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac90095e50 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac900ab350 con 0x55821e409df0 2024-09-10T22:56:28.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 10 client.26978 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 10 client.26978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 20 client.26978 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e49dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 4 client.26978 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fac900ac190 con 0x55821e425680 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.900+0000 7faca7fff640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e49dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.901+0000 7fac877fe640 10 client.26978 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.901+0000 7faca7fff640 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e49dc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e485d9448d07854 server_cookie=d2b5b5a8647f24d5 in_seq=0 out_seq=0 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.901+0000 7fac877fe640 1 -- 172.21.15.96:0/3295183208 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fac900ac190 con 0x55821e425680 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.901+0000 7fac877fe640 10 client.26978 handle_command_reply: tid=0 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.901+0000 7facb0930740 1 client.26978 shutdown 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 msgr2=0x55821e427a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 0x55821e427a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac900b5cf0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 shutdown_connections 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55821e425680 0x55821e49dc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55821e409df0 0x55821e427a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55821e415d80 0x55821e493b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 --2- 172.21.15.96:0/3295183208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55821e4094d0 0x55821e4274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 >> 172.21.15.96:0/3295183208 conn(0x55821e407300 msgr2=0x55821e414910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.902+0000 7fac877fe640 0 client.26978 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.903+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 shutdown_connections 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.903+0000 7facb0930740 1 -- 172.21.15.96:0/3295183208 wait complete. 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.904+0000 7facb0930740 20 client.26978 trim_cache size 0 max 16384 2024-09-10T22:56:28.322 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.904+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 msgr2=0x7fac940b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.904+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 0x7fac940b78a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55821e4081e0 tx=0x7fac98009500 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.904+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 msgr2=0x7faca819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.904+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9c0 tx=0x7fac9000ee80 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 shutdown_connections 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faca810d140 0x7faca81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac940b53e0 0x7fac940b78a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faca8104600 0x7faca819ba00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 --2- 172.21.15.96:0/210886155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faca8103ce0 0x7faca819b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:28.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 >> 172.21.15.96:0/210886155 conn(0x7faca80fd740 msgr2=0x7faca810f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:28.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 shutdown_connections 2024-09-10T22:56:28.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:27.905+0000 7facae55b640 1 -- 172.21.15.96:0/210886155 wait complete. 2024-09-10T22:56:29.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:56:29.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:29 smithi096 ceph-mon[30869]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 728 op/s 2024-09-10T22:56:29.543 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:29 smithi121 ceph-mon[35119]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 728 op/s 2024-09-10T22:56:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:29 smithi117 ceph-mon[35081]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 728 op/s 2024-09-10T22:56:30.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 -- 172.21.15.96:0/2139164910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 msgr2=0x7fabe0115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 --2- 172.21.15.96:0/2139164910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe0115f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009ca0 tx=0x7fabd802a8d0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 -- 172.21.15.96:0/2139164910 shutdown_connections 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 --2- 172.21.15.96:0/2139164910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe0115f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 --2- 172.21.15.96:0/2139164910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe010edd0 0x7fabe010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 --2- 172.21.15.96:0/2139164910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe01093c0 0x7fabe01097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 -- 172.21.15.96:0/2139164910 >> 172.21.15.96:0/2139164910 conn(0x7fabe0078aa0 msgr2=0x7fabe007aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 -- 172.21.15.96:0/2139164910 shutdown_connections 2024-09-10T22:56:30.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.111+0000 7fabe514f640 1 -- 172.21.15.96:0/2139164910 wait complete. 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.112+0000 7fabe514f640 1 Processor -- start 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 -- start start 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe01093c0 0x7fabe019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe01a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0119380 con 0x7fabe010edd0 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0119200 con 0x7fabe010f750 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabe514f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0119500 con 0x7fabe01093c0 2024-09-10T22:56:30.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50432/0 (socket says 172.21.15.96:50432) 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabde59c640 1 -- 172.21.15.96:0/3730610128 learned_addr learned my addr 172.21.15.96:0/3730610128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabdf59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe01a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.113+0000 7fabded9d640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe01093c0 0x7fabe019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 -- 172.21.15.96:0/3730610128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe01093c0 msgr2=0x7fabe019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe01093c0 0x7fabe019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 -- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 msgr2=0x7fabe019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 -- 172.21.15.96:0/3730610128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe01a58c0 con 0x7fabe010f750 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabdf59e640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe01a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fabd802f3b0 tx=0x7fabd8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabde59c640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabc7fff640 1 -- 172.21.15.96:0/3730610128 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8003b70 con 0x7fabe010f750 2024-09-10T22:56:30.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabc7fff640 1 -- 172.21.15.96:0/3730610128 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd8003d10 con 0x7fabe010f750 2024-09-10T22:56:30.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabc7fff640 1 -- 172.21.15.96:0/3730610128 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8030a40 con 0x7fabe010f750 2024-09-10T22:56:30.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a5b50 con 0x7fabe010f750 2024-09-10T22:56:30.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.114+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01ac270 con 0x7fabe010f750 2024-09-10T22:56:30.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.118+0000 7fabc7fff640 1 -- 172.21.15.96:0/3730610128 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fabd8030be0 con 0x7fabe010f750 2024-09-10T22:56:30.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.120+0000 7fabc7fff640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 0x7fabc80b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.120+0000 7fabc7fff640 1 -- 172.21.15.96:0/3730610128 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabd8106b10 con 0x7fabe010f750 2024-09-10T22:56:30.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.120+0000 7fabded9d640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 0x7fabc80b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 Processor -- start 2024-09-10T22:56:30.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 -- start start 2024-09-10T22:56:30.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabded9d640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 0x7fabc80b78e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabd000bbe0 tx=0x7fabd000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2df423ea0 con 0x55b2df4b99a0 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2df4132d0 con 0x55b2df4ada10 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabe7524740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2df4170e0 con 0x55b2df4ad0f0 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51342/0 (socket says 172.21.15.96:51342) 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.121+0000 7fabde59c640 1 -- 172.21.15.96:0/1432342245 learned_addr learned my addr 172.21.15.96:0/1432342245 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabdf59e640 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 -- 172.21.15.96:0/1432342245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 msgr2=0x55b2df4ad4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ad4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 -- 172.21.15.96:0/1432342245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 msgr2=0x55b2df4bbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4bbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 -- 172.21.15.96:0/1432342245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df171650 con 0x55b2df4ada10 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabde59c640 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009da0 tx=0x7fabd40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201ba2686c18ce4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabc57fa640 1 -- 172.21.15.96:0/1432342245 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4012650 con 0x55b2df4ada10 2024-09-10T22:56:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabc57fa640 1 -- 172.21.15.96:0/1432342245 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd4012ca0 con 0x55b2df4ada10 2024-09-10T22:56:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.122+0000 7fabc57fa640 1 -- 172.21.15.96:0/1432342245 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400fc70 con 0x55b2df4ada10 2024-09-10T22:56:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 -- 172.21.15.96:0/1432342245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 msgr2=0x55b2df4b9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009da0 tx=0x7fabd40117b0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 -- 172.21.15.96:0/1432342245 shutdown_connections 2024-09-10T22:56:30.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b9460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4bbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 --2- 172.21.15.96:0/1432342245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.123+0000 7fabe7524740 1 -- 172.21.15.96:0/1432342245 >> 172.21.15.96:0/1432342245 conn(0x55b2df4aaf20 msgr2=0x55b2df4ac0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.124+0000 7fabe7524740 1 -- 172.21.15.96:0/1432342245 shutdown_connections 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.124+0000 7fabe7524740 1 -- 172.21.15.96:0/1432342245 wait complete. 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.124+0000 7fabe7524740 1 -- start start 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.124+0000 7fabe7524740 1 Processor -- start 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b2df274700 con 0x55b2df4b99a0 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2df3ae160 con 0x55b2df4ada10 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2df4bc620 con 0x55b2df4ad0f0 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51350/0 (socket says 172.21.15.96:51350) 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 -- 172.21.15.96:0/558273284 learned_addr learned my addr 172.21.15.96:0/558273284 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabe7524740 10 client.0 _ll_register_callbacks cb 0x55b2df3c79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 -- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 msgr2=0x55b2df4b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabded9d640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabdf59e640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 -- 172.21.15.96:0/558273284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 msgr2=0x55b2df4b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabde59c640 1 -- 172.21.15.96:0/558273284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df3dbc90 con 0x55b2df4ada10 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabded9d640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.125+0000 7fabdf59e640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabde59c640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fabd401c040 tx=0x7fabd4009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af620fb66065af58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4029020 con 0x55b2df4ada10 2024-09-10T22:56:30.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabd400fc00 con 0x55b2df4ada10 2024-09-10T22:56:30.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd4012ca0 con 0x55b2df4ada10 2024-09-10T22:56:30.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabd4012e50 con 0x55b2df4ada10 2024-09-10T22:56:30.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabe7524740 10 client.26990 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:30.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2df4c7f90 con 0x55b2df4ada10 2024-09-10T22:56:30.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.126+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd4009e90 con 0x55b2df4ada10 2024-09-10T22:56:30.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.127+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fabd4003680 con 0x55b2df4ada10 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 10 client.26990 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 10 client.26990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 20 client.26990 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df4c9340 0x55b2df4cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 4 client.26990 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabd4012ca0 con 0x55b2df4c9340 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabded9d640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df4c9340 0x55b2df4cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabded9d640 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df4c9340 0x55b2df4cb730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2a5100f5af1fb8a server_cookie=6aeb0ebdb07ef131 in_seq=0 out_seq=0 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.128+0000 7fabc2ff5640 10 client.26990 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabc2ff5640 1 -- 172.21.15.96:0/558273284 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fabd4012ca0 con 0x55b2df4c9340 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabc2ff5640 10 client.26990 handle_command_reply: tid=0 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 client.26990 shutdown 2024-09-10T22:56:30.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 msgr2=0x55b2df4b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fabd401c040 tx=0x7fabd4009ac0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 shutdown_connections 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df4c9340 0x55b2df4cb730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b66b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabc2ff5640 0 client.26990 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df4b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 --2- 172.21.15.96:0/558273284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ad0f0 0x55b2df4b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.129+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 >> 172.21.15.96:0/558273284 conn(0x55b2df4aaf20 msgr2=0x55b2df4b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.130+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 shutdown_connections 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.130+0000 7fabe7524740 1 -- 172.21.15.96:0/558273284 wait complete. 2024-09-10T22:56:30.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.130+0000 7fabe7524740 20 client.26990 trim_cache size 0 max 16384 2024-09-10T22:56:30.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.150+0000 7fabe7524740 1 Processor -- start 2024-09-10T22:56:30.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 -- start start 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 0x55b2df4aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00e190 con 0x55b2df4ad0f0 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabde59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51360/0 (socket says 172.21.15.96:51360) 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabde59c640 1 -- 172.21.15.96:0/4078759123 learned_addr learned my addr 172.21.15.96:0/4078759123 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabdf59e640 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 0x55b2df4aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00fcd0 con 0x55b2df4ada10 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.151+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00e860 con 0x55b2df4b99a0 2024-09-10T22:56:30.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.155+0000 7fabded9d640 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.156+0000 7fabdf59e640 1 -- 172.21.15.96:0/4078759123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 msgr2=0x55b2df4b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.156+0000 7fabdf59e640 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.156+0000 7fabdf59e640 1 -- 172.21.15.96:0/4078759123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4ad0f0 msgr2=0x55b2df4ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.156+0000 7fabdf59e640 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.156+0000 7fabdf59e640 1 -- 172.21.15.96:0/4078759123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df4b5610 con 0x55b2df4b99a0 2024-09-10T22:56:30.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.162+0000 7fabdf59e640 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 0x55b2df4aa760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabd802aec0 tx=0x7fabd802a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9074eacc95f4e455 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.167+0000 7fabc4ff9640 1 -- 172.21.15.96:0/4078759123 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80ccde0 con 0x55b2df4b99a0 2024-09-10T22:56:30.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.167+0000 7fabc4ff9640 1 -- 172.21.15.96:0/4078759123 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd8036040 con 0x55b2df4b99a0 2024-09-10T22:56:30.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.167+0000 7fabc4ff9640 1 -- 172.21.15.96:0/4078759123 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80cd530 con 0x55b2df4b99a0 2024-09-10T22:56:30.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.169+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 msgr2=0x55b2df4aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.169+0000 7fabe7524740 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 0x55b2df4aa760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabd802aec0 tx=0x7fabd802a8d0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.173+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 shutdown_connections 2024-09-10T22:56:30.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.173+0000 7fabe7524740 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ada10 0x55b2df4b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.173+0000 7fabe7524740 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4b99a0 0x55b2df4aa760 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.173+0000 7fabe7524740 1 --2- 172.21.15.96:0/4078759123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4ad0f0 0x55b2df4ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.173+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 >> 172.21.15.96:0/4078759123 conn(0x55b2df4aaf20 msgr2=0x55b2df4abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.179+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 shutdown_connections 2024-09-10T22:56:30.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.179+0000 7fabe7524740 1 -- 172.21.15.96:0/4078759123 wait complete. 2024-09-10T22:56:30.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.180+0000 7fabe7524740 1 -- start start 2024-09-10T22:56:30.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.180+0000 7fabe7524740 1 Processor -- start 2024-09-10T22:56:30.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ad0f0 0x55b2df4bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df534ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00f600 con 0x55b2df4b99a0 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00ef30 con 0x55b2df4ad0f0 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc00c220 con 0x55b2df4ada10 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabe7524740 10 client.0 _ll_register_callbacks cb 0x7fabd4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:58850/0 (socket says 172.21.15.96:58850) 2024-09-10T22:56:30.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 -- 172.21.15.96:0/430916230 learned_addr learned my addr 172.21.15.96:0/430916230 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 -- 172.21.15.96:0/430916230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ad0f0 msgr2=0x55b2df4bdfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ad0f0 0x55b2df4bdfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 -- 172.21.15.96:0/430916230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 msgr2=0x55b2df534ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df534ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.181+0000 7fabde59c640 1 -- 172.21.15.96:0/430916230 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df5464a0 con 0x55b2df4ada10 2024-09-10T22:56:30.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.182+0000 7fabde59c640 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fabd4011cb0 tx=0x7fabd400f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5992a9ddf526b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:30.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.190+0000 7fabc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:56:30.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.190+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400fe10 con 0x55b2df4ada10 2024-09-10T22:56:30.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.191+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabd40035f0 con 0x55b2df4ada10 2024-09-10T22:56:30.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.192+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd8036040 con 0x55b2df4ada10 2024-09-10T22:56:30.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.198+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabd40037f0 con 0x55b2df4ada10 2024-09-10T22:56:30.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.199+0000 7fabe7524740 10 client.27049 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:30.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.199+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2df547780 con 0x55b2df4ada10 2024-09-10T22:56:30.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.204+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd8036040 con 0x55b2df4ada10 2024-09-10T22:56:30.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fabd4012880 con 0x55b2df4ada10 2024-09-10T22:56:30.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 10 client.27049 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:30.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 10 client.27049 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:56:30.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 20 client.27049 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:30.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df547e40 0x55b2df54a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:30.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 4 client.27049 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:30.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2df425df0 con 0x55b2df547e40 2024-09-10T22:56:30.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.205+0000 7fabded9d640 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df547e40 0x55b2df54a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:30.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.214+0000 7fabc2ff5640 10 client.27049 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:56:30.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.214+0000 7fabded9d640 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df547e40 0x55b2df54a260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=289f7e0d5b9cca6 server_cookie=2b32383faf4d7076 in_seq=0 out_seq=0 2024-09-10T22:56:30.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.215+0000 7fabc2ff5640 1 -- 172.21.15.96:0/430916230 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2df425df0 con 0x55b2df547e40 2024-09-10T22:56:30.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.215+0000 7fabc2ff5640 10 client.27049 handle_command_reply: tid=0 2024-09-10T22:56:30.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.216+0000 7fabe7524740 1 client.27049 shutdown 2024-09-10T22:56:30.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.218+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 msgr2=0x55b2df4be520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.218+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fabd4011cb0 tx=0x7fabd400f5a0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 shutdown_connections 2024-09-10T22:56:30.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b2df4b99a0 0x55b2df534ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df4ada10 0x55b2df4be520 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55b2df547e40 0x55b2df54a260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 --2- 172.21.15.96:0/430916230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2df4ad0f0 0x55b2df4bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 >> 172.21.15.96:0/430916230 conn(0x55b2df4aaf20 msgr2=0x55b2df4b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabc2ff5640 0 client.27049 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 shutdown_connections 2024-09-10T22:56:30.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.219+0000 7fabe7524740 1 -- 172.21.15.96:0/430916230 wait complete. 2024-09-10T22:56:30.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.220+0000 7fabe7524740 20 client.27049 trim_cache size 0 max 16384 2024-09-10T22:56:30.223 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 msgr2=0x7fabc80b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 0x7fabc80b78e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fabd000bbe0 tx=0x7fabd000b390 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 msgr2=0x7fabe01a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe01a50f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fabd802f3b0 tx=0x7fabd8009780 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 shutdown_connections 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe010f750 0x7fabe01a50f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabc80b5420 0x7fabc80b78e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe010edd0 0x7fabe019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 --2- 172.21.15.96:0/3730610128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe01093c0 0x7fabe019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:30.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 >> 172.21.15.96:0/3730610128 conn(0x7fabe0078aa0 msgr2=0x7fabe007ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:30.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.221+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 shutdown_connections 2024-09-10T22:56:30.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:30.222+0000 7fabe514f640 1 -- 172.21.15.96:0/3730610128 wait complete. 2024-09-10T22:56:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 535.0 s, 2358.2 tps, lat 42.446 ms stddev 51.031 2024-09-10T22:56:31.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:56:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:31 smithi096 ceph-mon[30869]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 828 op/s 2024-09-10T22:56:31.549 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:31 smithi121 ceph-mon[35119]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 828 op/s 2024-09-10T22:56:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:31 smithi117 ceph-mon[35081]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 828 op/s 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 -- 172.21.15.96:0/1361834737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95001087f0 msgr2=0x7f9500108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 --2- 172.21.15.96:0/1361834737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95001087f0 0x7f9500108c30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f94e801a910 tx=0x7f94e803b450 comp rx=0 tx=0).stop 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 -- 172.21.15.96:0/1361834737 shutdown_connections 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 --2- 172.21.15.96:0/1361834737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9500109170 0x7f950010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 --2- 172.21.15.96:0/1361834737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95001087f0 0x7f9500108c30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 --2- 172.21.15.96:0/1361834737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f95001031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 -- 172.21.15.96:0/1361834737 >> 172.21.15.96:0/1361834737 conn(0x7f95000fe870 msgr2=0x7f9500100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 -- 172.21.15.96:0/1361834737 shutdown_connections 2024-09-10T22:56:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.197+0000 7f9504f9d640 1 -- 172.21.15.96:0/1361834737 wait complete. 2024-09-10T22:56:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f9504f9d640 1 Processor -- start 2024-09-10T22:56:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f9504f9d640 1 -- start start 2024-09-10T22:56:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f9504f9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f9500070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f9504f9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51372/0 (socket says 172.21.15.96:51372) 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f94fe59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f9500070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f94fe59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f9500070bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:58854/0 (socket says 172.21.15.96:58854) 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.198+0000 7f9504f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 0x7f9500076490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 -- 172.21.15.96:0/4110385429 learned_addr learned my addr 172.21.15.96:0/4110385429 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f950006f3b0 con 0x7f9500109170 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f950006f230 con 0x7f95001087f0 2024-09-10T22:56:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f950006f530 con 0x7f9500102de0 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94fed9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 0x7f9500076490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 -- 172.21.15.96:0/4110385429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 msgr2=0x7f9500070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f9500070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 -- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 msgr2=0x7f9500076490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 0x7f9500076490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.199+0000 7f94f7fff640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9500076c10 con 0x7f95001087f0 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.200+0000 7f94fed9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 0x7f9500076490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.200+0000 7f94f7fff640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f94e803b970 tx=0x7f94e80052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.200+0000 7f94f77fe640 1 -- 172.21.15.96:0/4110385429 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e804f020 con 0x7f95001087f0 2024-09-10T22:56:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.201+0000 7f94f77fe640 1 -- 172.21.15.96:0/4110385429 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e8040e50 con 0x7f95001087f0 2024-09-10T22:56:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.201+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9500076ea0 con 0x7f95001087f0 2024-09-10T22:56:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.201+0000 7f94f77fe640 1 -- 172.21.15.96:0/4110385429 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e8046070 con 0x7f95001087f0 2024-09-10T22:56:32.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.201+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950019b3a0 con 0x7f95001087f0 2024-09-10T22:56:32.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.205+0000 7f9507372740 1 Processor -- start 2024-09-10T22:56:32.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.205+0000 7f9507372740 1 -- start start 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642e7a2eea0 con 0x5642e7ab8df0 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642e7a1e2d0 con 0x5642e7ab84d0 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f9507372740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642e7a220e0 con 0x5642e7ac4d80 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94f77fe640 1 -- 172.21.15.96:0/4110385429 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f94e80409a0 con 0x7f95001087f0 2024-09-10T22:56:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94fe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94fe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50494/0 (socket says 172.21.15.96:50494) 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94fe59c640 1 -- 172.21.15.96:0/670909835 learned_addr learned my addr 172.21.15.96:0/670909835 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.206+0000 7f94fed9d640 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 -- 172.21.15.96:0/670909835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 msgr2=0x5642e7ac7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 -- 172.21.15.96:0/670909835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 msgr2=0x5642e7ab88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 -- 172.21.15.96:0/670909835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e777c650 con 0x5642e7ab8df0 2024-09-10T22:56:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94f7fff640 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94fe59c640 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009a40 tx=0x7f94f0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640b802d4debc2df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.207+0000 7f94f77fe640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 0x7f94e00b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.208+0000 7f94fed9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 0x7f94e00b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.208+0000 7f94f4ff9640 1 -- 172.21.15.96:0/670909835 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0017070 con 0x5642e7ab8df0 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.208+0000 7f94f4ff9640 1 -- 172.21.15.96:0/670909835 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f0013470 con 0x5642e7ab8df0 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.208+0000 7f94f4ff9640 1 -- 172.21.15.96:0/670909835 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0012630 con 0x5642e7ab8df0 2024-09-10T22:56:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.208+0000 7f94f77fe640 1 -- 172.21.15.96:0/4110385429 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f94e8117280 con 0x7f95001087f0 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f94fed9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 0x7f94e00b7970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 -- 172.21.15.96:0/670909835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 msgr2=0x5642e7ac4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009a40 tx=0x7f94f0009f00 comp rx=0 tx=0).stop 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 -- 172.21.15.96:0/670909835 shutdown_connections 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac4840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 --2- 172.21.15.96:0/670909835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.209+0000 7f9507372740 1 -- 172.21.15.96:0/670909835 >> 172.21.15.96:0/670909835 conn(0x5642e7ab6300 msgr2=0x5642e7ab74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.210+0000 7f9507372740 1 -- 172.21.15.96:0/670909835 shutdown_connections 2024-09-10T22:56:32.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.210+0000 7f9507372740 1 -- 172.21.15.96:0/670909835 wait complete. 2024-09-10T22:56:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.210+0000 7f9507372740 1 -- start start 2024-09-10T22:56:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.210+0000 7f9507372740 1 Processor -- start 2024-09-10T22:56:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ac45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642e787f700 con 0x5642e7ab8df0 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642e79b9160 con 0x5642e7ab84d0 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642e7ac7aa0 con 0x5642e7ac4d80 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50508/0 (socket says 172.21.15.96:50508) 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f94f7fff640 1 -- 172.21.15.96:0/3280884587 learned_addr learned my addr 172.21.15.96:0/3280884587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f94fe59c640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ac45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.211+0000 7f9507372740 10 client.0 _ll_register_callbacks cb 0x5642e79d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 -- 172.21.15.96:0/3280884587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 msgr2=0x5642e7ac1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 -- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 msgr2=0x5642e7ac45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ac45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 -- 172.21.15.96:0/3280884587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e79e6c90 con 0x5642e7ab8df0 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94fe59c640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ac45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94f7fff640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f94e80da010 tx=0x7f94e80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca844cc83d8bea08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94edffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80d98e0 con 0x5642e7ab8df0 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f94e8004260 con 0x5642e7ab8df0 2024-09-10T22:56:32.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.212+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f0013470 con 0x5642e7ab8df0 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.214+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f94e80dd1b0 con 0x5642e7ab8df0 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.214+0000 7f9507372740 10 client.17406 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.214+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e7ad3320 con 0x5642e7ab8df0 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e8004480 con 0x5642e7ab8df0 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f94e80d9cb0 con 0x5642e7ab8df0 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 10 client.17406 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 20 client.17406 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:32.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7ad4910 0x5642e7ad6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 4 client.17406 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f94f0013470 con 0x5642e7ad4910 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.215+0000 7f94fe59c640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7ad4910 0x5642e7ad6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.216+0000 7f94fe59c640 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7ad4910 0x5642e7ad6d90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce53191da9e2bf27 server_cookie=e8cbdf711e492a97 in_seq=0 out_seq=0 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.216+0000 7f94edffb640 10 client.17406 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.217+0000 7f94edffb640 1 -- 172.21.15.96:0/3280884587 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f94f0013470 con 0x5642e7ad4910 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.217+0000 7f94edffb640 10 client.17406 handle_command_reply: tid=0 2024-09-10T22:56:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.217+0000 7f9507372740 1 client.17406 shutdown 2024-09-10T22:56:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.217+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 msgr2=0x5642e7ac1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.217+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f94e80da010 tx=0x7f94e80048a0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 shutdown_connections 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7ad4910 0x5642e7ad6d90 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac1a10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 --2- 172.21.15.96:0/3280884587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ac45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 >> 172.21.15.96:0/3280884587 conn(0x5642e7ab6300 msgr2=0x5642e7abbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f94edffb640 0 client.17406 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 shutdown_connections 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 1 -- 172.21.15.96:0/3280884587 wait complete. 2024-09-10T22:56:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.218+0000 7f9507372740 20 client.17406 trim_cache size 0 max 16384 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 Processor -- start 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 -- start start 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ab8df0 0x5642e7ac0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b4820 con 0x5642e7ac4d80 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b6360 con 0x5642e7ab84d0 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.245+0000 7f9507372740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b4ef0 con 0x5642e7ab8df0 2024-09-10T22:56:32.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.246+0000 7f94fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.246+0000 7f94fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.246+0000 7f94fed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50516/0 (socket says 172.21.15.96:50516) 2024-09-10T22:56:32.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.247+0000 7f94fed9d640 1 -- 172.21.15.96:0/2433517688 learned_addr learned my addr 172.21.15.96:0/2433517688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:32.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.248+0000 7f94fed9d640 1 -- 172.21.15.96:0/2433517688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ab8df0 msgr2=0x5642e7ac0f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:32.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.248+0000 7f94fed9d640 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ab8df0 0x5642e7ac0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.248+0000 7f94fed9d640 1 -- 172.21.15.96:0/2433517688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 msgr2=0x5642e7ab5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:32.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.248+0000 7f94fed9d640 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.248+0000 7f94fed9d640 1 -- 172.21.15.96:0/2433517688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e7ac22e0 con 0x5642e7ac4d80 2024-09-10T22:56:32.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.250+0000 7f94fed9d640 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f94f80107c0 tx=0x7f94f8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d429663b034b282b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.252+0000 7f94effff640 1 -- 172.21.15.96:0/2433517688 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801e8c0 con 0x5642e7ac4d80 2024-09-10T22:56:32.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.252+0000 7f94effff640 1 -- 172.21.15.96:0/2433517688 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f801f430 con 0x5642e7ac4d80 2024-09-10T22:56:32.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.252+0000 7f94effff640 1 -- 172.21.15.96:0/2433517688 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801c680 con 0x5642e7ac4d80 2024-09-10T22:56:32.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.253+0000 7f9507372740 1 -- 172.21.15.96:0/2433517688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 msgr2=0x5642e7ac0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.253+0000 7f9507372740 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f94f80107c0 tx=0x7f94f8014040 comp rx=0 tx=0).stop 2024-09-10T22:56:32.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.254+0000 7f9507372740 1 -- 172.21.15.96:0/2433517688 shutdown_connections 2024-09-10T22:56:32.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.254+0000 7f9507372740 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ac4d80 0x5642e7ac0290 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.254+0000 7f9507372740 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ab8df0 0x5642e7ac0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.254+0000 7f9507372740 1 --2- 172.21.15.96:0/2433517688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7ab5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.254+0000 7f9507372740 1 -- 172.21.15.96:0/2433517688 >> 172.21.15.96:0/2433517688 conn(0x5642e7ab6300 msgr2=0x5642e7ab7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.259+0000 7f9507372740 1 -- 172.21.15.96:0/2433517688 shutdown_connections 2024-09-10T22:56:32.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.259+0000 7f9507372740 1 -- 172.21.15.96:0/2433517688 wait complete. 2024-09-10T22:56:32.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.259+0000 7f9507372740 1 -- start start 2024-09-10T22:56:32.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.259+0000 7f9507372740 1 Processor -- start 2024-09-10T22:56:32.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.260+0000 7f9507372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7acd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.260+0000 7f9507372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.260+0000 7f9507372740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.260+0000 7f9507372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b5c90 con 0x5642e7ab8df0 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f9507372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b55c0 con 0x5642e7ab84d0 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f9507372740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95001b2820 con 0x5642e7ac4d80 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f9507372740 10 client.0 _ll_register_callbacks cb 0x7f94e8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50522/0 (socket says 172.21.15.96:50522) 2024-09-10T22:56:32.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 -- 172.21.15.96:0/2571271722 learned_addr learned my addr 172.21.15.96:0/2571271722 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:32.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 -- 172.21.15.96:0/2571271722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 msgr2=0x5642e7ac9100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:32.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac9100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 -- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 msgr2=0x5642e7acd600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:32.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7acd600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.261+0000 7f94f7fff640 1 -- 172.21.15.96:0/2571271722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e7b408a0 con 0x5642e7ab8df0 2024-09-10T22:56:32.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.262+0000 7f94f7fff640 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d3ac7bee76126a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:32.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.265+0000 7f94edffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:32.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.265+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80dd390 con 0x5642e7ab8df0 2024-09-10T22:56:32.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.265+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f94e80dceb0 con 0x5642e7ab8df0 2024-09-10T22:56:32.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.265+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f801f430 con 0x5642e7ab8df0 2024-09-10T22:56:32.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.267+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f94e80dd810 con 0x5642e7ab8df0 2024-09-10T22:56:32.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.268+0000 7f9507372740 10 client.17412 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:32.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.268+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e7b419f0 con 0x5642e7ab8df0 2024-09-10T22:56:32.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.268+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e8004480 con 0x5642e7ab8df0 2024-09-10T22:56:32.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.268+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f94e80dc710 con 0x5642e7ab8df0 2024-09-10T22:56:32.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 10 client.17412 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:32.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:32.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 20 client.17412 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:32.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7b41f30 0x5642e7b44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:32.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 4 client.17412 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:56:32.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f94f801f430 con 0x5642e7b41f30 2024-09-10T22:56:32.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.269+0000 7f94fe59c640 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7b41f30 0x5642e7b44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:32.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.270+0000 7f94fe59c640 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7b41f30 0x5642e7b44350 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce36081741df248 server_cookie=ec907edb52b2c2be in_seq=0 out_seq=0 2024-09-10T22:56:32.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.270+0000 7f94edffb640 10 client.17412 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:32.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.270+0000 7f94edffb640 1 -- 172.21.15.96:0/2571271722 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f94f801f430 con 0x5642e7b41f30 2024-09-10T22:56:32.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.270+0000 7f94edffb640 10 client.17412 handle_command_reply: tid=0 2024-09-10T22:56:32.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.272+0000 7f9507372740 1 client.17412 shutdown 2024-09-10T22:56:32.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.273+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 msgr2=0x5642e7ac8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.273+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8005a70 comp rx=0 tx=0).stop 2024-09-10T22:56:32.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.273+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 shutdown_connections 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.273+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5642e7b41f30 0x5642e7b44350 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.274+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e7ac4d80 0x5642e7ac9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.274+0000 7f94edffb640 0 client.17412 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.274+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e7ab8df0 0x5642e7ac8ad0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.274+0000 7f9507372740 1 --2- 172.21.15.96:0/2571271722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e7ab84d0 0x5642e7acd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.274+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 >> 172.21.15.96:0/2571271722 conn(0x5642e7ab6300 msgr2=0x5642e7ac3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.275+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 shutdown_connections 2024-09-10T22:56:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.275+0000 7f9507372740 1 -- 172.21.15.96:0/2571271722 wait complete. 2024-09-10T22:56:32.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.275+0000 7f9507372740 20 client.17412 trim_cache size 0 max 16384 2024-09-10T22:56:32.278 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:56:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 msgr2=0x7f94e00b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 0x7f94e00b7970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f8009500 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 msgr2=0x7f95000710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f94e803b970 tx=0x7f94e80052a0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 shutdown_connections 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95001087f0 0x7f95000710f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f94e00b54b0 0x7f94e00b7970 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9500109170 0x7f9500076490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 --2- 172.21.15.96:0/4110385429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9500102de0 0x7f9500070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 >> 172.21.15.96:0/4110385429 conn(0x7f95000fe870 msgr2=0x7f9500100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 shutdown_connections 2024-09-10T22:56:32.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:32.277+0000 7f9504f9d640 1 -- 172.21.15.96:0/4110385429 wait complete. 2024-09-10T22:56:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:33 smithi096 ceph-mon[30869]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 714 op/s 2024-09-10T22:56:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:33 smithi121 ceph-mon[35119]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 714 op/s 2024-09-10T22:56:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:33 smithi117 ceph-mon[35081]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 714 op/s 2024-09-10T22:56:34.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:56:34.056 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:56:34.467 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:35.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 -- 172.21.15.96:0/1299327953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010aa70 msgr2=0x7f73701178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 --2- 172.21.15.96:0/1299327953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010aa70 0x7f73701178c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7364009890 tx=0x7f736402a410 comp rx=0 tx=0).stop 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 -- 172.21.15.96:0/1299327953 shutdown_connections 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 --2- 172.21.15.96:0/1299327953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010aa70 0x7f73701178c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 --2- 172.21.15.96:0/1299327953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 0x7f737011a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 --2- 172.21.15.96:0/1299327953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010a150 0x7f737010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.079+0000 7f7376979640 1 -- 172.21.15.96:0/1299327953 >> 172.21.15.96:0/1299327953 conn(0x7f737006dfc0 msgr2=0x7f737006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.080+0000 7f7376979640 1 -- 172.21.15.96:0/1299327953 shutdown_connections 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.080+0000 7f7376979640 1 -- 172.21.15.96:0/1299327953 wait complete. 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 Processor -- start 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 -- start start 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010aa70 0x7f73701a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 0x7f73701a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f736ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f736ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51422/0 (socket says 172.21.15.96:51422) 2024-09-10T22:56:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f736ffff640 1 -- 172.21.15.96:0/1470052366 learned_addr learned my addr 172.21.15.96:0/1470052366 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f737011d7b0 con 0x7f737010aa70 2024-09-10T22:56:35.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f737011d630 con 0x7f737010a150 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.081+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f737011d930 con 0x7f7370117e00 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f7374f16640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 0x7f73701a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736f7fe640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010aa70 0x7f73701a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 -- 172.21.15.96:0/1470052366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 msgr2=0x7f73701a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 0x7f73701a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 -- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010aa70 msgr2=0x7f73701a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010aa70 0x7f73701a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73701a9d90 con 0x7f737010a150 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736ffff640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f736800cf30 tx=0x7f736800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736d7fa640 1 -- 172.21.15.96:0/1470052366 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368023070 con 0x7f737010a150 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73701aa080 con 0x7f737010a150 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737010d640 con 0x7f737010a150 2024-09-10T22:56:35.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736d7fa640 1 -- 172.21.15.96:0/1470052366 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736801c9d0 con 0x7f737010a150 2024-09-10T22:56:35.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.082+0000 7f736d7fa640 1 -- 172.21.15.96:0/1470052366 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736801e940 con 0x7f737010a150 2024-09-10T22:56:35.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.086+0000 7f736d7fa640 1 -- 172.21.15.96:0/1470052366 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f736800b040 con 0x7f737010a150 2024-09-10T22:56:35.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 Processor -- start 2024-09-10T22:56:35.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 -- start start 2024-09-10T22:56:35.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f736d7fa640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 0x7f73580b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd87b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd887250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f736d7fa640 1 -- 172.21.15.96:0/1470052366 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f73680ee560 con 0x7f737010a150 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 0x55c6cd889b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd7f1e60 con 0x55c6cd87b800 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd7e1290 con 0x55c6cd87aee0 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.087+0000 7f7378d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd7e50a0 con 0x55c6cd887790 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd887250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd887250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50548/0 (socket says 172.21.15.96:50548) 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f736f7fe640 1 -- 172.21.15.96:0/3486198236 learned_addr learned my addr 172.21.15.96:0/3486198236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 0x55c6cd889b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 0x7f73580b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 -- 172.21.15.96:0/3486198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 msgr2=0x55c6cd87b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd87b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 -- 172.21.15.96:0/3486198236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 msgr2=0x55c6cd887250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd887250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.088+0000 7f7374f16640 1 -- 172.21.15.96:0/3486198236 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cd53f930 con 0x55c6cd887790 2024-09-10T22:56:35.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.089+0000 7f7374f16640 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 0x55c6cd889b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7360010a30 tx=0x7f7360010ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206b4a834625376b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.089+0000 7f7374f16640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 0x7f73580b78e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f736000f850 tx=0x7f7360018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.089+0000 7f73567fc640 1 -- 172.21.15.96:0/3486198236 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360019040 con 0x55c6cd887790 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.089+0000 7f73567fc640 1 -- 172.21.15.96:0/3486198236 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360010430 con 0x55c6cd887790 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.089+0000 7f73567fc640 1 -- 172.21.15.96:0/3486198236 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736001ebd0 con 0x55c6cd887790 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 -- 172.21.15.96:0/3486198236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 msgr2=0x55c6cd889b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 0x55c6cd889b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7360010a30 tx=0x7f7360010ef0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 -- 172.21.15.96:0/3486198236 shutdown_connections 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd887790 0x55c6cd889b80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd887250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 --2- 172.21.15.96:0/3486198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd87b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 -- 172.21.15.96:0/3486198236 >> 172.21.15.96:0/3486198236 conn(0x55c6cd878d10 msgr2=0x55c6cd879eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 -- 172.21.15.96:0/3486198236 shutdown_connections 2024-09-10T22:56:35.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.090+0000 7f7378d4e740 1 -- 172.21.15.96:0/3486198236 wait complete. 2024-09-10T22:56:35.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.091+0000 7f7378d4e740 1 -- start start 2024-09-10T22:56:35.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.091+0000 7f7378d4e740 1 Processor -- start 2024-09-10T22:56:35.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd884db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd887790 0x55c6cd8852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd642890 con 0x55c6cd887790 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd77bd40 con 0x55c6cd87aee0 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd88a3e0 con 0x55c6cd87b800 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7378d4e740 10 client.0 _ll_register_callbacks cb 0x55c6cd7dfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51444/0 (socket says 172.21.15.96:51444) 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 -- 172.21.15.96:0/2434878367 learned_addr learned my addr 172.21.15.96:0/2434878367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736f7fe640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd884db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f7374f16640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd887790 0x55c6cd8852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 -- 172.21.15.96:0/2434878367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 msgr2=0x55c6cd884db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd884db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 -- 172.21.15.96:0/2434878367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd887790 msgr2=0x55c6cd8852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd887790 0x55c6cd8852f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.092+0000 7f736ffff640 1 -- 172.21.15.96:0/2434878367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cd8939a0 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f736ffff640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73680096c0 tx=0x7f73680b4490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23086d30f05008c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680ee310 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f736801ebc0 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7360010430 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73680b4d20 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 10 client.27014 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c6cd896310 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360010430 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f736809abd0 con 0x55c6cd87aee0 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 10 client.27014 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 10 client.27014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 20 client.27014 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd897810 0x55c6cd899c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 4 client.27014 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.093+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cd7f3db0 con 0x55c6cd897810 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.094+0000 7f736f7fe640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd897810 0x55c6cd899c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.094+0000 7f734f7fe640 10 client.27014 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.094+0000 7f736f7fe640 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd897810 0x55c6cd899c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2891fe9271349d server_cookie=e1518de4dd32aace in_seq=0 out_seq=0 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f734f7fe640 1 -- 172.21.15.96:0/2434878367 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6cd7f3db0 con 0x55c6cd897810 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f734f7fe640 10 client.27014 handle_command_reply: tid=0 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 client.27014 shutdown 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 msgr2=0x55c6cd884870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73680096c0 tx=0x7f73680b4490 comp rx=0 tx=0).stop 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 shutdown_connections 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd897810 0x55c6cd899c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f734f7fe640 0 client.27014 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd887790 0x55c6cd8852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd884db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 --2- 172.21.15.96:0/2434878367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd87aee0 0x55c6cd884870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 >> 172.21.15.96:0/2434878367 conn(0x55c6cd878d10 msgr2=0x55c6cd877f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 shutdown_connections 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 1 -- 172.21.15.96:0/2434878367 wait complete. 2024-09-10T22:56:35.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.095+0000 7f7378d4e740 20 client.27014 trim_cache size 0 max 16384 2024-09-10T22:56:35.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.123+0000 7f7378d4e740 1 Processor -- start 2024-09-10T22:56:35.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 -- start start 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd892da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd881aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00e240 con 0x55c6cd87aee0 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00fd80 con 0x55c6cd878d10 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7378d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00e910 con 0x55c6cd87b800 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f736ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd892da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50562/0 (socket says 172.21.15.96:50562) 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f7374f16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd881aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.124+0000 7f736f7fe640 1 -- 172.21.15.96:0/1532259877 learned_addr learned my addr 172.21.15.96:0/1532259877 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:35.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 -- 172.21.15.96:0/1532259877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 msgr2=0x55c6cd892da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd892da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 -- 172.21.15.96:0/1532259877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 msgr2=0x55c6cd8932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 -- 172.21.15.96:0/1532259877 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cd87a3c0 con 0x55c6cd87b800 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f736f7fe640 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7374f16640 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd881aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f736000a010 tx=0x7f7360009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18146a43e847de02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7355ffb640 1 -- 172.21.15.96:0/1532259877 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360034020 con 0x55c6cd87b800 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7355ffb640 1 -- 172.21.15.96:0/1532259877 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736000ed30 con 0x55c6cd87b800 2024-09-10T22:56:35.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.125+0000 7f7355ffb640 1 -- 172.21.15.96:0/1532259877 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360005400 con 0x55c6cd87b800 2024-09-10T22:56:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 -- 172.21.15.96:0/1532259877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 msgr2=0x55c6cd881aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd881aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f736000a010 tx=0x7f7360009a80 comp rx=0 tx=0).stop 2024-09-10T22:56:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 -- 172.21.15.96:0/1532259877 shutdown_connections 2024-09-10T22:56:35.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87b800 0x55c6cd881aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87aee0 0x55c6cd8932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1532259877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd892da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 -- 172.21.15.96:0/1532259877 >> 172.21.15.96:0/1532259877 conn(0x55c6cd886ad0 msgr2=0x55c6cd892690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 -- 172.21.15.96:0/1532259877 shutdown_connections 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.126+0000 7f7378d4e740 1 -- 172.21.15.96:0/1532259877 wait complete. 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.127+0000 7f7378d4e740 1 -- start start 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.127+0000 7f7378d4e740 1 Processor -- start 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd88ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 0x55c6cd88c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00f6b0 con 0x55c6cd87b800 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00efe0 con 0x55c6cd878d10 2024-09-10T22:56:35.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f735c00c220 con 0x55c6cd87aee0 2024-09-10T22:56:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7378d4e740 10 client.0 _ll_register_callbacks cb 0x7f73680b4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.128+0000 7f7374f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50564/0 (socket says 172.21.15.96:50564) 2024-09-10T22:56:35.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 -- 172.21.15.96:0/1102679279 learned_addr learned my addr 172.21.15.96:0/1102679279 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f736ffff640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd88ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 -- 172.21.15.96:0/1102679279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 msgr2=0x55c6cd88c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f736f7fe640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 0x55c6cd88c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 0x55c6cd88c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 -- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 msgr2=0x55c6cd88ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd88ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f7374f16640 1 -- 172.21.15.96:0/1102679279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cd9046a0 con 0x55c6cd87b800 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f736ffff640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd88ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.129+0000 7f736f7fe640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 0x55c6cd88c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f7374f16640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7360010f30 tx=0x7f736001e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3521bd8d9ffe373 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f734f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000eb90 con 0x55c6cd87b800 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7360005340 con 0x55c6cd87b800 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736000ed30 con 0x55c6cd87b800 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7360005540 con 0x55c6cd87b800 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f7378d4e740 10 client.17430 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:35.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c6cd905860 con 0x55c6cd87b800 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.130+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736000ed30 con 0x55c6cd87b800 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7360003b20 con 0x55c6cd87b800 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 10 client.17430 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 20 client.17430 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd887790 0x55c6cd907ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 4 client.17430 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c6cd7ef860 con 0x55c6cd887790 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f736ffff640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd887790 0x55c6cd907ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f734f7fe640 10 client.17430 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:35.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.131+0000 7f736ffff640 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd887790 0x55c6cd907ef0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc662dce5efd4b58 server_cookie=1b12e78a7ff5bb8 in_seq=0 out_seq=0 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f734f7fe640 1 -- 172.21.15.96:0/1102679279 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c6cd7ef860 con 0x55c6cd887790 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f734f7fe640 10 client.17430 handle_command_reply: tid=0 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f7378d4e740 1 client.17430 shutdown 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 msgr2=0x55c6cd88cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7360010f30 tx=0x7f736001e6c0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 shutdown_connections 2024-09-10T22:56:35.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.133+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c6cd887790 0x55c6cd907ef0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6cd87b800 0x55c6cd88cb60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c6cd87aee0 0x55c6cd88c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 --2- 172.21.15.96:0/1102679279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6cd878d10 0x55c6cd88ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 >> 172.21.15.96:0/1102679279 conn(0x55c6cd886ad0 msgr2=0x55c6cd892eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f734f7fe640 0 client.17430 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 shutdown_connections 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 1 -- 172.21.15.96:0/1102679279 wait complete. 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.134+0000 7f7378d4e740 20 client.17430 trim_cache size 0 max 16384 2024-09-10T22:56:35.138 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:56:35.139 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:56:35.139 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "cc4a82d5-9a16-48fd-980f-de2071c2da43", 2024-09-10T22:56:35.139 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:56:35.139 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 msgr2=0x7f73580b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 0x7f73580b78e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f736000f850 tx=0x7f7360018000 comp rx=0 tx=0).stop 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 msgr2=0x7f73701a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f736800cf30 tx=0x7f736800b730 comp rx=0 tx=0).stop 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 shutdown_connections 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.136+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73580b5420 0x7f73580b78e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7370117e00 0x7f73701a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f737010aa70 0x7f73701a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 --2- 172.21.15.96:0/1470052366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f737010a150 0x7f73701a3ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:35.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 >> 172.21.15.96:0/1470052366 conn(0x7f737006dfc0 msgr2=0x7f73701182e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:35.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 shutdown_connections 2024-09-10T22:56:35.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:35.137+0000 7f7376979640 1 -- 172.21.15.96:0/1470052366 wait complete. 2024-09-10T22:56:35.660 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:35 smithi096 ceph-mon[30869]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 736 op/s 2024-09-10T22:56:35.660 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:35 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:56:35.660 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:35 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:35.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:35 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:56:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:35 smithi121 ceph-mon[35119]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 736 op/s 2024-09-10T22:56:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:35 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:56:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:35 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:35.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:35 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:56:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:35 smithi117 ceph-mon[35081]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 736 op/s 2024-09-10T22:56:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:35 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:56:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:35 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:35 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:56:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 540.0 s, 2020.6 tps, lat 49.412 ms stddev 62.827 2024-09-10T22:56:36.075 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:56:36.464 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:56:36.634 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:36 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:56:36.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:36 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:36.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:36 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:36.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:36 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:36.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:36 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:56:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:36 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:36 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:36.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:36 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:36 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:56:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:36 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:36.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:36 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:56:36.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:36 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:56:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.073+0000 7f37527fc640 1 -- 172.21.15.96:0/2975619154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c02f310 con 0x7f375410d120 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 -- 172.21.15.96:0/2975619154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 msgr2=0x7f375410f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 --2- 172.21.15.96:0/2975619154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375410f580 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f374c009ca0 tx=0x7f374c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 -- 172.21.15.96:0/2975619154 shutdown_connections 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 --2- 172.21.15.96:0/2975619154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375410f580 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 --2- 172.21.15.96:0/2975619154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37541045e0 0x7f375410cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 --2- 172.21.15.96:0/2975619154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754103cc0 0x7f37541040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 -- 172.21.15.96:0/2975619154 >> 172.21.15.96:0/2975619154 conn(0x7f37540fd700 msgr2=0x7f37540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 -- 172.21.15.96:0/2975619154 shutdown_connections 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.074+0000 7f375934f640 1 -- 172.21.15.96:0/2975619154 wait complete. 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.075+0000 7f375934f640 1 Processor -- start 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.075+0000 7f375934f640 1 -- start start 2024-09-10T22:56:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.076+0000 7f375934f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3754103cc0 0x7f37541992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.077+0000 7f375934f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f375934f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52492/0 (socket says 172.21.15.96:52492) 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f375934f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3754112de0 con 0x7f375410d120 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f375934f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3754112c60 con 0x7f37541045e0 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f375934f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3754112f60 con 0x7f3754103cc0 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 -- 172.21.15.96:0/641335136 learned_addr learned my addr 172.21.15.96:0/641335136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f37537fe640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 -- 172.21.15.96:0/641335136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3754103cc0 msgr2=0x7f37541992e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3754103cc0 0x7f37541992e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 -- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 msgr2=0x7f375419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f374bfff640 1 -- 172.21.15.96:0/641335136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375419f390 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.078+0000 7f37537fe640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f374bfff640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f373c00e9c0 tx=0x7f373c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f3750ff9640 1 -- 172.21.15.96:0/641335136 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00ccc0 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f3750ff9640 1 -- 172.21.15.96:0/641335136 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0035f0 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f3750ff9640 1 -- 172.21.15.96:0/641335136 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00ce60 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375419f680 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.079+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3754101850 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.082+0000 7f3750ff9640 1 -- 172.21.15.96:0/641335136 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f373c004000 con 0x7f37541045e0 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.082+0000 7f375b724740 1 Processor -- start 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.082+0000 7f375b724740 1 -- start start 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x563513324410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513326d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56351328ee50 con 0x5635133180a0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56351327e280 con 0x563513324950 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f375b724740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563513282090 con 0x5635133189c0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f37537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f37537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58032/0 (socket says 172.21.15.96:58032) 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f37537fe640 1 -- 172.21.15.96:0/1530952553 learned_addr learned my addr 172.21.15.96:0/1530952553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f374bfff640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513326d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.083+0000 7f3752ffd640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x563513324410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 -- 172.21.15.96:0/1530952553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 msgr2=0x563513324410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x563513324410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 -- 172.21.15.96:0/1530952553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 msgr2=0x563513326d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513326d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 -- 172.21.15.96:0/1530952553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563512fdc650 con 0x5635133180a0 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f374bfff640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513326d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37537fe640 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f374c005fa0 tx=0x7f374c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eced854207edbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f3750ff9640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 0x7f37340b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37497fa640 1 -- 172.21.15.96:0/1530952553 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c031cf0 con 0x5635133180a0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37497fa640 1 -- 172.21.15.96:0/1530952553 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f374c036040 con 0x5635133180a0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f37497fa640 1 -- 172.21.15.96:0/1530952553 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c003e60 con 0x5635133180a0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f3750ff9640 1 -- 172.21.15.96:0/641335136 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f373c0e5dc0 con 0x7f37541045e0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.084+0000 7f3752ffd640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 0x7f37340b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.085+0000 7f375b724740 1 -- 172.21.15.96:0/1530952553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 msgr2=0x563513318480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.085+0000 7f375b724740 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f374c005fa0 tx=0x7f374c009800 comp rx=0 tx=0).stop 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.085+0000 7f3752ffd640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 0x7f37340b7820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x563513316db0 tx=0x7f374400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.086+0000 7f375b724740 1 -- 172.21.15.96:0/1530952553 shutdown_connections 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.086+0000 7f375b724740 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513326d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.086+0000 7f375b724740 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x563513324410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.086+0000 7f375b724740 1 --2- 172.21.15.96:0/1530952553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x563513318480 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.086+0000 7f375b724740 1 -- 172.21.15.96:0/1530952553 >> 172.21.15.96:0/1530952553 conn(0x563513315ed0 msgr2=0x563513317070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.087+0000 7f375b724740 1 -- 172.21.15.96:0/1530952553 shutdown_connections 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.087+0000 7f375b724740 1 -- 172.21.15.96:0/1530952553 wait complete. 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.087+0000 7f375b724740 1 -- start start 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.087+0000 7f375b724740 1 Processor -- start 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.088+0000 7f375b724740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x563513324210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.088+0000 7f375b724740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 0x563513321660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.088+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x563513324210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.088+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x563513324210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:56646/0 (socket says 172.21.15.96:56646) 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.088+0000 7f375b724740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513321c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f3752ffd640 1 -- 172.21.15.96:0/3261642111 learned_addr learned my addr 172.21.15.96:0/3261642111 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 0x563513321660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5635130df700 con 0x5635133189c0 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563513219110 con 0x563513324950 2024-09-10T22:56:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563513327510 con 0x5635133180a0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f37537fe640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513321c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f375b724740 10 client.0 _ll_register_callbacks cb 0x563513232990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 -- 172.21.15.96:0/3261642111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 msgr2=0x563513324210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x563513324210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 -- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 msgr2=0x563513321c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513321c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.089+0000 7f374bfff640 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563513246c40 con 0x5635133189c0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f374bfff640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 0x563513321660 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f373c0e39b0 tx=0x7f373c0ac500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55beccad6c1cf6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f37427fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0acd20 con 0x5635133189c0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f373c004320 con 0x5635133189c0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c036040 con 0x5635133189c0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f373c0ac620 con 0x5635133189c0 2024-09-10T22:56:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f375b724740 10 client.17442 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.090+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563513332e30 con 0x5635133189c0 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0b1100 con 0x5635133189c0 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f373c0a7f70 con 0x5635133189c0 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 10 client.17442 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 20 client.17442 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563513334180 0x563513336570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 4 client.17442 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f374c036040 con 0x563513334180 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.091+0000 7f3752ffd640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563513334180 0x563513336570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.092+0000 7f37427fc640 10 client.17442 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.092+0000 7f3752ffd640 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563513334180 0x563513336570 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6570a190f53d9736 server_cookie=943001006665ef31 in_seq=0 out_seq=0 2024-09-10T22:56:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.093+0000 7f37427fc640 1 -- 172.21.15.96:0/3261642111 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f374c036040 con 0x563513334180 2024-09-10T22:56:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.093+0000 7f37427fc640 10 client.17442 handle_command_reply: tid=0 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.094+0000 7f375b724740 1 client.17442 shutdown 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.094+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 msgr2=0x563513321660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.094+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 0x563513321660 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f373c0e39b0 tx=0x7f373c0ac500 comp rx=0 tx=0).stop 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 shutdown_connections 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563513334180 0x563513336570 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513321c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f37427fc640 0 client.17442 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133189c0 0x563513321660 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 --2- 172.21.15.96:0/3261642111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x563513324210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 >> 172.21.15.96:0/3261642111 conn(0x563513315ed0 msgr2=0x56351331ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 shutdown_connections 2024-09-10T22:56:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.095+0000 7f375b724740 1 -- 172.21.15.96:0/3261642111 wait complete. 2024-09-10T22:56:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.096+0000 7f375b724740 20 client.17442 trim_cache size 0 max 16384 2024-09-10T22:56:37.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.128+0000 7f375b724740 1 Processor -- start 2024-09-10T22:56:37.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.128+0000 7f375b724740 1 -- start start 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x5635133307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563513324950 0x56351331edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f374bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f374bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52508/0 (socket says 172.21.15.96:52508) 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f374bfff640 1 -- 172.21.15.96:0/4219041786 learned_addr learned my addr 172.21.15.96:0/4219041786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37541ac090 con 0x563513324950 2024-09-10T22:56:37.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37541adbd0 con 0x5635133189c0 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f37537fe640 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563513324950 0x56351331edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.129+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37541ac760 con 0x5635133180a0 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f3752ffd640 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x5635133307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f374bfff640 1 -- 172.21.15.96:0/4219041786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 msgr2=0x5635133307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f374bfff640 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x5635133307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f374bfff640 1 -- 172.21.15.96:0/4219041786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563513324950 msgr2=0x56351331edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f374bfff640 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563513324950 0x56351331edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.130+0000 7f374bfff640 1 -- 172.21.15.96:0/4219041786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635133270b0 con 0x5635133189c0 2024-09-10T22:56:37.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.132+0000 7f374bfff640 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f373c009660 tx=0x7f373c0ac1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4c71d7605b961e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.132+0000 7f3748ff9640 1 -- 172.21.15.96:0/4219041786 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0a84c0 con 0x5635133189c0 2024-09-10T22:56:37.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.132+0000 7f3748ff9640 1 -- 172.21.15.96:0/4219041786 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0acd20 con 0x5635133189c0 2024-09-10T22:56:37.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.132+0000 7f3748ff9640 1 -- 172.21.15.96:0/4219041786 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0a7f70 con 0x5635133189c0 2024-09-10T22:56:37.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.133+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 msgr2=0x56351331e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.133+0000 7f375b724740 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f373c009660 tx=0x7f373c0ac1f0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.135+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 shutdown_connections 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.135+0000 7f375b724740 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635133189c0 0x56351331e870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.135+0000 7f375b724740 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563513324950 0x56351331edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.135+0000 7f375b724740 1 --2- 172.21.15.96:0/4219041786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133180a0 0x5635133307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.135+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 >> 172.21.15.96:0/4219041786 conn(0x563513315ed0 msgr2=0x56351331c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.136+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 shutdown_connections 2024-09-10T22:56:37.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.136+0000 7f375b724740 1 -- 172.21.15.96:0/4219041786 wait complete. 2024-09-10T22:56:37.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.136+0000 7f375b724740 1 -- start start 2024-09-10T22:56:37.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f375b724740 1 Processor -- start 2024-09-10T22:56:37.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f375b724740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x5635133ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f375b724740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x5635133ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f375b724740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513329010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x5635133ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.137+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x5635133ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58060/0 (socket says 172.21.15.96:58060) 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f375b724740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37541ad500 con 0x5635133180a0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f3752ffd640 1 -- 172.21.15.96:0/3813074494 learned_addr learned my addr 172.21.15.96:0/3813074494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37541ace30 con 0x563513324950 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37541aa070 con 0x5635133189c0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f375b724740 10 client.0 _ll_register_callbacks cb 0x7f373c003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513329010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f374bfff640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x5635133ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 -- 172.21.15.96:0/3813074494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 msgr2=0x5635133ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x5635133ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 -- 172.21.15.96:0/3813074494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 msgr2=0x5635133ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x5635133ab390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563513329820 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37537fe640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513329010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f374c005e30 tx=0x7f374c003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f921a122396144 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37427fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.138+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c059020 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c0acd20 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f374c031860 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f374c004b40 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 10 client.27032 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5635133b2c90 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c0acd20 con 0x563513324950 2024-09-10T22:56:37.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f374c05e070 con 0x563513324950 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 10 client.27032 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 10 client.27032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 20 client.27032 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5635133b2fd0 0x5635133b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 4 client.27032 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563513290da0 con 0x5635133b2fd0 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.139+0000 7f3752ffd640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5635133b2fd0 0x5635133b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f37427fc640 10 client.27032 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f3752ffd640 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5635133b2fd0 0x5635133b53c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed543fd4b722a1f server_cookie=473431e2ca9ea29b in_seq=0 out_seq=0 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f37427fc640 1 -- 172.21.15.96:0/3813074494 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x563513290da0 con 0x5635133b2fd0 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f37427fc640 10 client.27032 handle_command_reply: tid=0 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 client.27032 shutdown 2024-09-10T22:56:37.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 msgr2=0x563513329010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513329010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f374c005e30 tx=0x7f374c003c00 comp rx=0 tx=0).stop 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 shutdown_connections 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5635133b2fd0 0x5635133b53c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563513324950 0x563513329010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635133189c0 0x5635133ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 --2- 172.21.15.96:0/3813074494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635133180a0 0x5635133ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 >> 172.21.15.96:0/3813074494 conn(0x563513315ed0 msgr2=0x563513322c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.140+0000 7f37427fc640 0 client.27032 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.141+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 shutdown_connections 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.141+0000 7f375b724740 1 -- 172.21.15.96:0/3813074494 wait complete. 2024-09-10T22:56:37.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.141+0000 7f375b724740 20 client.27032 trim_cache size 0 max 16384 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "cc4a82d5-9a16-48fd-980f-de2071c2da43", 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:56:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:56:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:56:37.151 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:56:37.153 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 msgr2=0x7f37340b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 0x7f37340b7820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x563513316db0 tx=0x7f374400a250 comp rx=0 tx=0).stop 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 msgr2=0x7f3754199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f373c00e9c0 tx=0x7f373c00ee80 comp rx=0 tx=0).stop 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 shutdown_connections 2024-09-10T22:56:37.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37541045e0 0x7f3754199820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f37340b5360 0x7f37340b7820 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f375410d120 0x7f375419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 --2- 172.21.15.96:0/641335136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3754103cc0 0x7f37541992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 >> 172.21.15.96:0/641335136 conn(0x7f37540fd700 msgr2=0x7f37540fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 shutdown_connections 2024-09-10T22:56:37.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:56:37.142+0000 7f375934f640 1 -- 172.21.15.96:0/641335136 wait complete. 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 -- 172.21.15.96:0/3379017983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40103370 msgr2=0x7f3b40109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 --2- 172.21.15.96:0/3379017983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40103370 0x7f3b40109cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009900 tx=0x7f3b3802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 -- 172.21.15.96:0/3379017983 shutdown_connections 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 --2- 172.21.15.96:0/3379017983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40103370 0x7f3b40109cf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 --2- 172.21.15.96:0/3379017983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 0x7f3b40102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 --2- 172.21.15.96:0/3379017983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40101860 0x7f3b40101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 -- 172.21.15.96:0/3379017983 >> 172.21.15.96:0/3379017983 conn(0x7f3b400fd140 msgr2=0x7f3b400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 -- 172.21.15.96:0/3379017983 shutdown_connections 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.416+0000 7f3b45216640 1 -- 172.21.15.96:0/3379017983 wait complete. 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 Processor -- start 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 -- start start 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 0x7f3b40199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 0x7f3b40199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010e600 con 0x7f3b401029b0 2024-09-10T22:56:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010e480 con 0x7f3b40101860 2024-09-10T22:56:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b45216640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010e780 con 0x7f3b40103370 2024-09-10T22:56:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52536/0 (socket says 172.21.15.96:52536) 2024-09-10T22:56:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b3ed9d640 1 -- 172.21.15.96:0/887869547 learned_addr learned my addr 172.21.15.96:0/887869547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 0x7f3b40199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.417+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 0x7f3b40199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 -- 172.21.15.96:0/887869547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 msgr2=0x7f3b40199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 0x7f3b40199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 -- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 msgr2=0x7f3b40199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 0x7f3b40199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 -- 172.21.15.96:0/887869547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4019a850 con 0x7f3b40101860 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 0x7f3b40199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c890 tx=0x7f3b2800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b37fff640 1 -- 172.21.15.96:0/887869547 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004220 con 0x7f3b40101860 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.418+0000 7f3b37fff640 1 -- 172.21.15.96:0/887869547 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b280043c0 con 0x7f3b40101860 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.419+0000 7f3b37fff640 1 -- 172.21.15.96:0/887869547 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004a00 con 0x7f3b40101860 2024-09-10T22:56:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.419+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4019ab40 con 0x7f3b40101860 2024-09-10T22:56:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.419+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4010a780 con 0x7f3b40101860 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.422+0000 7f3b47fe3740 1 Processor -- start 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.422+0000 7f3b47fe3740 1 -- start start 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 0x55e6ca363c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca366580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca369090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca2d36f0 con 0x55e6ca364190 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca2c2b20 con 0x55e6ca366bd0 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca2c6930 con 0x55e6ca361860 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3f59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 0x55e6ca363c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3f59e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 0x55e6ca363c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:56688/0 (socket says 172.21.15.96:56688) 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b37fff640 1 -- 172.21.15.96:0/887869547 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3b280079b0 con 0x7f3b40101860 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3f59e640 1 -- 172.21.15.96:0/2402778023 learned_addr learned my addr 172.21.15.96:0/2402778023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca369090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca366580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2402778023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 msgr2=0x55e6ca363c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 0x55e6ca363c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2402778023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 msgr2=0x55e6ca366580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca366580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2402778023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ca288530 con 0x55e6ca366bd0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.423+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca366580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.424+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca369090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009fa0 tx=0x7f3b300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bae5accd8ddd65d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.424+0000 7f3b357fa640 1 -- 172.21.15.96:0/2402778023 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003c30 con 0x55e6ca366bd0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.424+0000 7f3b357fa640 1 -- 172.21.15.96:0/2402778023 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b30003dd0 con 0x55e6ca366bd0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.424+0000 7f3b357fa640 1 -- 172.21.15.96:0/2402778023 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000fb20 con 0x55e6ca366bd0 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.424+0000 7f3b37fff640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 0x7f3b2c0b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.425+0000 7f3b37fff640 1 -- 172.21.15.96:0/887869547 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3b280e5820 con 0x7f3b40101860 2024-09-10T22:56:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.425+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2402778023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 msgr2=0x55e6ca369090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.425+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca369090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009fa0 tx=0x7f3b300118d0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.425+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 0x7f3b2c0b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.425+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2402778023 shutdown_connections 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca369090 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca366580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2402778023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca361860 0x55e6ca363c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2402778023 >> 172.21.15.96:0/2402778023 conn(0x55e6ca35c9b0 msgr2=0x55e6ca35d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2402778023 shutdown_connections 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2402778023 wait complete. 2024-09-10T22:56:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 -- start start 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 0x7f3b2c0b7860 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3b38031490 tx=0x7f3b38004400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 Processor -- start 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.426+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca36aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca366bd0 0x55e6ca36c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca1faab0 con 0x55e6ca364190 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca268730 con 0x55e6ca361860 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e6ca369bb0 con 0x55e6ca366bd0 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58114/0 (socket says 172.21.15.96:58114) 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 -- 172.21.15.96:0/3157210183 learned_addr learned my addr 172.21.15.96:0/3157210183 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b47fe3740 10 client.0 _ll_register_callbacks cb 0x55e6ca2bc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca366bd0 0x55e6ca36c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca36aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 -- 172.21.15.96:0/3157210183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca366bd0 msgr2=0x55e6ca36c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca366bd0 0x55e6ca36c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 -- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 msgr2=0x55e6ca36aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca36aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 -- 172.21.15.96:0/3157210183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ca024790 con 0x55e6ca364190 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca36aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b30009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beed7a22b6288b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.427+0000 7f3b267fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30027070 con 0x55e6ca364190 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3b3000fc70 con 0x55e6ca364190 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30003dd0 con 0x55e6ca364190 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b30004000 con 0x55e6ca364190 2024-09-10T22:56:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b47fe3740 10 client.17457 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6ca09d620 con 0x55e6ca364190 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b30033070 con 0x55e6ca364190 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.428+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3b30005360 con 0x55e6ca364190 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 10 client.17457 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 20 client.17457 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca378990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 4 client.17457 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6ca2d5640 con 0x55e6ca376540 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca378990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b267fc640 10 client.17457 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.429+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca378990 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39689d4d20bcdf17 server_cookie=bce8a2dca698f9d9 in_seq=0 out_seq=0 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.430+0000 7f3b267fc640 1 -- 172.21.15.96:0/3157210183 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6ca2d5640 con 0x55e6ca376540 2024-09-10T22:56:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.430+0000 7f3b267fc640 10 client.17457 handle_command_reply: tid=0 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.430+0000 7f3b47fe3740 1 client.17457 shutdown 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 msgr2=0x55e6ca36b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b30009d20 comp rx=0 tx=0).stop 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 shutdown_connections 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca378990 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca366bd0 0x55e6ca36c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b267fc640 0 client.17457 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca364190 0x55e6ca36b570 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3157210183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca36aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 >> 172.21.15.96:0/3157210183 conn(0x55e6ca35c9b0 msgr2=0x55e6ca35bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 shutdown_connections 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3157210183 wait complete. 2024-09-10T22:56:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.431+0000 7f3b47fe3740 20 client.17457 trim_cache size 0 max 16384 2024-09-10T22:56:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.453+0000 7f3b47fe3740 1 Processor -- start 2024-09-10T22:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.453+0000 7f3b47fe3740 1 -- start start 2024-09-10T22:56:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.454+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca361860 0x55e6ca360cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.454+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.454+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.454+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ac060 con 0x55e6ca361860 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.454+0000 7f3b47fe3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401adba0 con 0x55e6ca366bd0 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca361860 0x55e6ca360cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52554/0 (socket says 172.21.15.96:52554) 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:56698/0 (socket says 172.21.15.96:56698) 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3f59e640 1 -- 172.21.15.96:0/2735717134 learned_addr learned my addr 172.21.15.96:0/2735717134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ac730 con 0x55e6ca364190 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2735717134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 msgr2=0x55e6ca35d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2735717134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca361860 msgr2=0x55e6ca360cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca361860 0x55e6ca360cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 -- 172.21.15.96:0/2735717134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ca360940 con 0x55e6ca364190 2024-09-10T22:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:56:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.455+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009ef0 tx=0x7f3b30005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4bab67329aae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.456+0000 7f3b34ff9640 1 -- 172.21.15.96:0/2735717134 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30027040 con 0x55e6ca364190 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.456+0000 7f3b34ff9640 1 -- 172.21.15.96:0/2735717134 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b30030030 con 0x55e6ca364190 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.456+0000 7f3b34ff9640 1 -- 172.21.15.96:0/2735717134 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30005440 con 0x55e6ca364190 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 msgr2=0x55e6ca361200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009ef0 tx=0x7f3b30005af0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 shutdown_connections 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca366bd0 0x55e6ca35d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca361200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/2735717134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca361860 0x55e6ca360cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.457+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 >> 172.21.15.96:0/2735717134 conn(0x55e6ca35c9b0 msgr2=0x55e6ca366fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.458+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 shutdown_connections 2024-09-10T22:56:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.458+0000 7f3b47fe3740 1 -- 172.21.15.96:0/2735717134 wait complete. 2024-09-10T22:56:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.458+0000 7f3b47fe3740 1 -- start start 2024-09-10T22:56:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.458+0000 7f3b47fe3740 1 Processor -- start 2024-09-10T22:56:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.459+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca372560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.459+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b47fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 0x55e6ca371330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:56710/0 (socket says 172.21.15.96:56710) 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca372560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca372560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:52570/0 (socket says 172.21.15.96:52570) 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3e59c640 1 -- 172.21.15.96:0/3219700907 learned_addr learned my addr 172.21.15.96:0/3219700907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ad4d0 con 0x55e6ca366bd0 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 0x55e6ca371330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ace00 con 0x55e6ca361860 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.460+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401aa060 con 0x55e6ca364190 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.461+0000 7f3b3f59e640 1 -- 172.21.15.96:0/3219700907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 msgr2=0x55e6ca372c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.461+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.461+0000 7f3b3f59e640 1 -- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 msgr2=0x55e6ca372560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.461+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca372560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.461+0000 7f3b3f59e640 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ca371b60 con 0x55e6ca366bd0 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.462+0000 7f3b3f59e640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 0x55e6ca371330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55e6ca361830 tx=0x7f3b38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5687b391741ebbea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.462+0000 7f3b267fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.462+0000 7f3b47fe3740 10 client.0 _ll_register_callbacks cb 0x7f3b30009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.462+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b30030030 con 0x55e6ca366bd0 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.462+0000 7f3b3e59c640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:56:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.463+0000 7f3b267fc640 1 -- 172.21.15.96:0/3219700907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035050 con 0x55e6ca366bd0 2024-09-10T22:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.463+0000 7f3b267fc640 1 -- 172.21.15.96:0/3219700907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3b38031c10 con 0x55e6ca366bd0 2024-09-10T22:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.463+0000 7f3b267fc640 1 -- 172.21.15.96:0/3219700907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b38031e30 con 0x55e6ca366bd0 2024-09-10T22:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.463+0000 7f3b47fe3740 10 client.17463 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.463+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e6ca370d50 con 0x55e6ca366bd0 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b267fc640 1 -- 172.21.15.96:0/3219700907 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3b3805d470 con 0x55e6ca366bd0 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 10 client.17463 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 20 client.17463 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca3e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 4 client.17463 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6ca2d10f0 con 0x55e6ca376540 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca3e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b267fc640 10 client.17463 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.464+0000 7f3b3ed9d640 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca3e5950 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c57084f5a8f78a server_cookie=8c35ca900e3698fd in_seq=0 out_seq=0 2024-09-10T22:56:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b267fc640 1 -- 172.21.15.96:0/3219700907 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6ca2d10f0 con 0x55e6ca376540 2024-09-10T22:56:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b267fc640 10 client.17463 handle_command_reply: tid=0 2024-09-10T22:56:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 client.17463 shutdown 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 msgr2=0x55e6ca371330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 0x55e6ca371330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55e6ca361830 tx=0x7f3b38009260 comp rx=0 tx=0).stop 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 shutdown_connections 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e6ca376540 0x55e6ca3e5950 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b267fc640 0 client.17463 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.489+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ca366bd0 0x55e6ca371330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6ca364190 0x55e6ca372c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 1 --2- 172.21.15.96:0/3219700907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6ca361860 0x55e6ca372560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 >> 172.21.15.96:0/3219700907 conn(0x55e6ca35c9b0 msgr2=0x55e6ca36aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 shutdown_connections 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 1 -- 172.21.15.96:0/3219700907 wait complete. 2024-09-10T22:56:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b47fe3740 20 client.17463 trim_cache size 0 max 16384 2024-09-10T22:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:56:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 msgr2=0x7f3b2c0b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 0x7f3b2c0b7860 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3b38031490 tx=0x7f3b38004400 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 msgr2=0x7f3b40198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c890 tx=0x7f3b2800cd50 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 shutdown_connections 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b2c0b53a0 0x7f3b2c0b7860 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b40103370 0x7f3b40199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b401029b0 0x7f3b40199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 --2- 172.21.15.96:0/887869547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b40101860 0x7f3b40198d10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.490+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 >> 172.21.15.96:0/887869547 conn(0x7f3b400fd140 msgr2=0x7f3b40105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.491+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 shutdown_connections 2024-09-10T22:56:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:37.491+0000 7f3b45216640 1 -- 172.21.15.96:0/887869547 wait complete. 2024-09-10T22:56:37.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:37 smithi096 ceph-mon[30869]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 712 op/s 2024-09-10T22:56:37.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:37.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:37.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:56:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:37 smithi121 ceph-mon[35119]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 712 op/s 2024-09-10T22:56:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:37.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:56:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:37 smithi117 ceph-mon[35081]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 712 op/s 2024-09-10T22:56:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:56:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:56:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:56:37.886 INFO:tasks.cephfs.filesystem:scrub status for tag:cc4a82d5-9a16-48fd-980f-de2071c2da43 - {'path': '/', 'tag': 'cc4a82d5-9a16-48fd-980f-de2071c2da43', 'options': 'recursive,force'} 2024-09-10T22:56:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:39 smithi096 ceph-mon[30869]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 636 op/s 2024-09-10T22:56:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:39 smithi121 ceph-mon[35119]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 636 op/s 2024-09-10T22:56:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:39 smithi117 ceph-mon[35081]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 636 op/s 2024-09-10T22:56:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 545.0 s, 2248.8 tps, lat 43.541 ms stddev 50.647 2024-09-10T22:56:41.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:41 smithi096 ceph-mon[30869]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 810 op/s 2024-09-10T22:56:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:41 smithi121 ceph-mon[35119]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 810 op/s 2024-09-10T22:56:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:41 smithi117 ceph-mon[35081]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 810 op/s 2024-09-10T22:56:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:43 smithi096 ceph-mon[30869]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T22:56:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:43 smithi121 ceph-mon[35119]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T22:56:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:43 smithi117 ceph-mon[35081]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T22:56:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:44 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:56:44.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:44 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:56:44.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:44 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:56:45.677 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:45 smithi117 ceph-mon[35081]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 29 MiB/s wr, 1.10k op/s 2024-09-10T22:56:45.738 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:45 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:56:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:45 smithi096 ceph-mon[30869]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 29 MiB/s wr, 1.10k op/s 2024-09-10T22:56:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:45 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:56:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:45 smithi121 ceph-mon[35119]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 29 MiB/s wr, 1.10k op/s 2024-09-10T22:56:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:45 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:56:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 550.0 s, 1943.2 tps, lat 52.572 ms stddev 94.206 2024-09-10T22:56:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:47 smithi096 ceph-mon[30869]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 45 MiB/s wr, 1.47k op/s 2024-09-10T22:56:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:47 smithi121 ceph-mon[35119]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 45 MiB/s wr, 1.47k op/s 2024-09-10T22:56:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:47 smithi117 ceph-mon[35081]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 45 MiB/s wr, 1.47k op/s 2024-09-10T22:56:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:49 smithi096 ceph-mon[30869]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 43 MiB/s wr, 1.39k op/s 2024-09-10T22:56:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:49 smithi121 ceph-mon[35119]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 43 MiB/s wr, 1.39k op/s 2024-09-10T22:56:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:49 smithi117 ceph-mon[35081]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 43 MiB/s wr, 1.39k op/s 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.609+0000 7fac6d1f8640 1 -- 172.21.15.96:0/3909189310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68103370 msgr2=0x7fac68109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.609+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/3909189310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68103370 0x7fac68109cf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fac60009cd0 tx=0x7fac6002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 -- 172.21.15.96:0/3909189310 shutdown_connections 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/3909189310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac681029b0 0x7fac68102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/3909189310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68103370 0x7fac68109cf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/3909189310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac68101860 0x7fac68101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 -- 172.21.15.96:0/3909189310 >> 172.21.15.96:0/3909189310 conn(0x7fac680fd140 msgr2=0x7fac680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 -- 172.21.15.96:0/3909189310 shutdown_connections 2024-09-10T22:56:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.610+0000 7fac6d1f8640 1 -- 172.21.15.96:0/3909189310 wait complete. 2024-09-10T22:56:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 Processor -- start 2024-09-10T22:56:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 -- start start 2024-09-10T22:56:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101860 0x7fac68105670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac68103370 0x7fac68104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810d4d0 con 0x7fac68101860 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810d350 con 0x7fac68103370 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.611+0000 7fac6d1f8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810d650 con 0x7fac681029b0 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40700/0 (socket says 172.21.15.96:40700) 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 -- 172.21.15.96:0/1224704986 learned_addr learned my addr 172.21.15.96:0/1224704986 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101860 0x7fac68105670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 -- 172.21.15.96:0/1224704986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac68103370 msgr2=0x7fac68104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac68103370 0x7fac68104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 -- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101860 msgr2=0x7fac68105670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101860 0x7fac68105670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 -- 172.21.15.96:0/1224704986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac68104950 con 0x7fac681029b0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d950 tx=0x7fac5000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.612+0000 7fac64d99640 1 -- 172.21.15.96:0/1224704986 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50017030 con 0x7fac681029b0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.613+0000 7fac64d99640 1 -- 172.21.15.96:0/1224704986 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac50004020 con 0x7fac681029b0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.613+0000 7fac64d99640 1 -- 172.21.15.96:0/1224704986 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500047a0 con 0x7fac681029b0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.613+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac68104c40 con 0x7fac681029b0 2024-09-10T22:56:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.613+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681a7700 con 0x7fac681029b0 2024-09-10T22:56:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.616+0000 7fac64d99640 1 -- 172.21.15.96:0/1224704986 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fac50009a30 con 0x7fac681029b0 2024-09-10T22:56:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac6ffca740 1 Processor -- start 2024-09-10T22:56:50.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac6ffca740 1 -- start start 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882746fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x558827472580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x558827475090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.617+0000 7fac64d99640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 0x7fac540b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6ffca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5588273df6f0 con 0x558827472bd0 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6ffca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588273ceb20 con 0x558827470190 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6ffca740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5588273d2930 con 0x55882746d860 2024-09-10T22:56:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 0x7fac540b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x558827475090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x558827475090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46716/0 (socket says 172.21.15.96:46716) 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882746fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac5ffff640 1 -- 172.21.15.96:0/1185310478 learned_addr learned my addr 172.21.15.96:0/1185310478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x558827472580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac64d99640 1 -- 172.21.15.96:0/1224704986 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac500e55c0 con 0x7fac681029b0 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 -- 172.21.15.96:0/1185310478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 msgr2=0x558827472580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x558827472580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 -- 172.21.15.96:0/1185310478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 msgr2=0x558827475090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x558827475090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.618+0000 7fac6759e640 1 -- 172.21.15.96:0/1185310478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558827394530 con 0x55882746d860 2024-09-10T22:56:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.619+0000 7fac6759e640 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882746fc50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fac60009ca0 tx=0x7fac60044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1091e12171519f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.619+0000 7fac6759e640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 0x7fac540b78a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac6002ae00 tx=0x7fac60009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.619+0000 7fac5d7fa640 1 -- 172.21.15.96:0/1185310478 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60031430 con 0x55882746d860 2024-09-10T22:56:50.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.619+0000 7fac5d7fa640 1 -- 172.21.15.96:0/1185310478 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6004e070 con 0x55882746d860 2024-09-10T22:56:50.624 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.619+0000 7fac5d7fa640 1 -- 172.21.15.96:0/1185310478 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac600057c0 con 0x55882746d860 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- 172.21.15.96:0/1185310478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 msgr2=0x55882746fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882746fc50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fac60009ca0 tx=0x7fac60044000 comp rx=0 tx=0).stop 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- 172.21.15.96:0/1185310478 shutdown_connections 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x558827472580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x558827475090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1185310478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882746fc50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- 172.21.15.96:0/1185310478 >> 172.21.15.96:0/1185310478 conn(0x5588274689b0 msgr2=0x558827469740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- 172.21.15.96:0/1185310478 shutdown_connections 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- 172.21.15.96:0/1185310478 wait complete. 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 -- start start 2024-09-10T22:56:50.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.620+0000 7fac6ffca740 1 Processor -- start 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827470190 0x5588274775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827472bd0 0x558827478180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558827306ab0 con 0x55882746d860 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558827374730 con 0x558827472bd0 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558827475b50 con 0x558827470190 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46728/0 (socket says 172.21.15.96:46728) 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 -- 172.21.15.96:0/1042548526 learned_addr learned my addr 172.21.15.96:0/1042548526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827470190 0x5588274775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6759e640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827472bd0 0x558827478180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac6ffca740 10 client.0 _ll_register_callbacks cb 0x5588273c86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 -- 172.21.15.96:0/1042548526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827470190 msgr2=0x5588274775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827470190 0x5588274775b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 -- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827472bd0 msgr2=0x558827478180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827472bd0 0x558827478180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.621+0000 7fac66d9d640 1 -- 172.21.15.96:0/1042548526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558827130790 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.622+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fac58012740 tx=0x7fac58012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362146392c8e73f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.622+0000 7fac4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.622+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5800f920 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.622+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac5800fac0 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.622+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6004e070 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac5800fd00 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac6ffca740 10 client.17475 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588271a9620 con 0x55882746d860 2024-09-10T22:56:50.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac580108e0 con 0x55882746d860 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac5801c540 con 0x55882746d860 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac6ffca740 10 client.17475 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.623+0000 7fac6ffca740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac6ffca740 20 client.17475 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558827482830 0x558827484c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac6ffca740 4 client.17475 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:56:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac6004e070 con 0x558827482830 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558827482830 0x558827484c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac4e7fc640 10 client.17475 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.624+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558827482830 0x558827484c40 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b2c9f431088c16 server_cookie=4715db323b7d41f7 in_seq=0 out_seq=0 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1042548526 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fac6004e070 con 0x558827482830 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac4e7fc640 10 client.17475 handle_command_reply: tid=0 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 client.17475 shutdown 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 msgr2=0x558827476f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fac58012740 tx=0x7fac58012b40 comp rx=0 tx=0).stop 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 shutdown_connections 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558827482830 0x558827484c40 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827472bd0 0x558827478180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827470190 0x5588274775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac4e7fc640 0 client.17475 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1042548526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827476f10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.625+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 >> 172.21.15.96:0/1042548526 conn(0x5588274689b0 msgr2=0x558827467bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.626+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 shutdown_connections 2024-09-10T22:56:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.626+0000 7fac6ffca740 1 -- 172.21.15.96:0/1042548526 wait complete. 2024-09-10T22:56:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.626+0000 7fac6ffca740 20 client.17475 trim_cache size 0 max 16384 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.646+0000 7fac6ffca740 1 Processor -- start 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.646+0000 7fac6ffca740 1 -- start start 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.646+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827473c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.646+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.646+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6ffca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b03c0 con 0x55882746d860 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6ffca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b1f00 con 0x558827470190 2024-09-10T22:56:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6ffca740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b0a90 con 0x558827472bd0 2024-09-10T22:56:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40724/0 (socket says 172.21.15.96:40724) 2024-09-10T22:56:50.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.647+0000 7fac6759e640 1 -- 172.21.15.96:0/721841745 learned_addr learned my addr 172.21.15.96:0/721841745 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 -- 172.21.15.96:0/721841745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 msgr2=0x55882747ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747ea80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 -- 172.21.15.96:0/721841745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 msgr2=0x558827473c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827473c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 -- 172.21.15.96:0/721841745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558827469c10 con 0x558827472bd0 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.648+0000 7fac6759e640 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f80 tx=0x7fac60031950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09969c6ad7c7ced server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.649+0000 7fac5cff9640 1 -- 172.21.15.96:0/721841745 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60031b30 con 0x558827472bd0 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.650+0000 7fac5cff9640 1 -- 172.21.15.96:0/721841745 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6004e040 con 0x558827472bd0 2024-09-10T22:56:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.650+0000 7fac5cff9640 1 -- 172.21.15.96:0/721841745 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60004f90 con 0x558827472bd0 2024-09-10T22:56:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.651+0000 7fac6ffca740 1 -- 172.21.15.96:0/721841745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 msgr2=0x558827474170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.656 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.651+0000 7fac6ffca740 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac60005f80 tx=0x7fac60031950 comp rx=0 tx=0).stop 2024-09-10T22:56:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.662+0000 7fac6ffca740 1 -- 172.21.15.96:0/721841745 shutdown_connections 2024-09-10T22:56:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.662+0000 7fac6ffca740 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.662+0000 7fac6ffca740 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558827472bd0 0x558827474170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.662+0000 7fac6ffca740 1 --2- 172.21.15.96:0/721841745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55882746d860 0x558827473c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.662+0000 7fac6ffca740 1 -- 172.21.15.96:0/721841745 >> 172.21.15.96:0/721841745 conn(0x5588274689b0 msgr2=0x558827472fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.668+0000 7fac6ffca740 1 -- 172.21.15.96:0/721841745 shutdown_connections 2024-09-10T22:56:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.672+0000 7fac6ffca740 1 -- 172.21.15.96:0/721841745 wait complete. 2024-09-10T22:56:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.672+0000 7fac6ffca740 1 -- start start 2024-09-10T22:56:50.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.672+0000 7fac6ffca740 1 Processor -- start 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882747e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x55882747d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b1830 con 0x558827472bd0 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b1160 con 0x558827470190 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac681ae3a0 con 0x55882746d860 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.676+0000 7fac6ffca740 10 client.0 _ll_register_callbacks cb 0x7fac58009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:56:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.677+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882747e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.681+0000 7fac66d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882747e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40726/0 (socket says 172.21.15.96:40726) 2024-09-10T22:56:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.681+0000 7fac66d9d640 1 -- 172.21.15.96:0/1655533988 learned_addr learned my addr 172.21.15.96:0/1655533988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:56:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.681+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.681+0000 7fac6759e640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x55882747d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.685+0000 7fac5ffff640 1 -- 172.21.15.96:0/1655533988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 msgr2=0x55882747e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.685+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882747e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.685+0000 7fac5ffff640 1 -- 172.21.15.96:0/1655533988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 msgr2=0x55882747d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.685+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x55882747d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.685+0000 7fac5ffff640 1 -- 172.21.15.96:0/1655533988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588274728f0 con 0x558827470190 2024-09-10T22:56:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.686+0000 7fac5ffff640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac500a8ff0 tx=0x7fac50021ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a58ce1b8f42040 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.688+0000 7fac4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.688+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500ab890 con 0x558827470190 2024-09-10T22:56:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.688+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac50009d50 con 0x558827470190 2024-09-10T22:56:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.690+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6004e040 con 0x558827470190 2024-09-10T22:56:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.698+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac500ac480 con 0x558827470190 2024-09-10T22:56:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.703+0000 7fac6ffca740 10 client.27062 fetch_fsmap learned FSMap version 32 2024-09-10T22:56:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.703+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55882747cbb0 con 0x558827470190 2024-09-10T22:56:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.704+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac50097020 con 0x558827470190 2024-09-10T22:56:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.706+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac500a7ff0 con 0x558827470190 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 10 client.27062 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 10 client.27062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 20 client.27062 populate_metadata read hostname 'smithi096' 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5588274efca0 0x5588274f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 4 client.27062 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:56:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.708+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588273e1640 con 0x5588274efca0 2024-09-10T22:56:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.710+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5588274efca0 0x5588274f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:56:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.712+0000 7fac66d9d640 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5588274efca0 0x5588274f20c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b705dc1fe758ca2d server_cookie=98b024b3c5b52032 in_seq=0 out_seq=0 2024-09-10T22:56:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.713+0000 7fac4e7fc640 10 client.27062 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.767+0000 7fac4e7fc640 1 -- 172.21.15.96:0/1655533988 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588273e1640 con 0x5588274efca0 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.767+0000 7fac4e7fc640 10 client.27062 handle_command_reply: tid=0 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 client.27062 shutdown 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 msgr2=0x55882747d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747d3f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac500a8ff0 tx=0x7fac50021ec0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 shutdown_connections 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5588274efca0 0x5588274f20c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558827470190 0x55882747d3f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558827472bd0 0x55882747d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 --2- 172.21.15.96:0/1655533988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55882746d860 0x55882747e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 >> 172.21.15.96:0/1655533988 conn(0x5588274689b0 msgr2=0x5588274673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac4e7fc640 0 client.27062 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 shutdown_connections 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 1 -- 172.21.15.96:0/1655533988 wait complete. 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.768+0000 7fac6ffca740 20 client.27062 trim_cache size 0 max 16384 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:56:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:56:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:56:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 msgr2=0x7fac540b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 0x7fac540b78a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac6002ae00 tx=0x7fac60009800 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 msgr2=0x7fac68103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d950 tx=0x7fac5000de10 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 shutdown_connections 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac68103370 0x7fac68104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac540b53e0 0x7fac540b78a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac681029b0 0x7fac68103ed0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 --2- 172.21.15.96:0/1224704986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac68101860 0x7fac68105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:56:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 >> 172.21.15.96:0/1224704986 conn(0x7fac680fd140 msgr2=0x7fac6806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:56:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 shutdown_connections 2024-09-10T22:56:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:56:50.769+0000 7fac6d1f8640 1 -- 172.21.15.96:0/1224704986 wait complete. 2024-09-10T22:56:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 555.0 s, 2198.0 tps, lat 45.512 ms stddev 57.270 2024-09-10T22:56:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:51 smithi096 ceph-mon[30869]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 45 MiB/s wr, 1.55k op/s 2024-09-10T22:56:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:51 smithi121 ceph-mon[35119]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 45 MiB/s wr, 1.55k op/s 2024-09-10T22:56:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:51 smithi117 ceph-mon[35081]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 45 MiB/s wr, 1.55k op/s 2024-09-10T22:56:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:53 smithi096 ceph-mon[30869]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 1.47k op/s 2024-09-10T22:56:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:53 smithi121 ceph-mon[35119]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 1.47k op/s 2024-09-10T22:56:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:53 smithi117 ceph-mon[35081]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 43 MiB/s wr, 1.47k op/s 2024-09-10T22:56:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:55 smithi096 ceph-mon[30869]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-09-10T22:56:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:55 smithi121 ceph-mon[35119]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-09-10T22:56:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:55 smithi117 ceph-mon[35081]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-09-10T22:56:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 560.0 s, 2442.8 tps, lat 41.009 ms stddev 49.799 2024-09-10T22:56:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:57 smithi096 ceph-mon[30869]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.29k op/s 2024-09-10T22:56:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:57 smithi121 ceph-mon[35119]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.29k op/s 2024-09-10T22:56:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:57 smithi117 ceph-mon[35081]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.29k op/s 2024-09-10T22:56:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:56:59 smithi096 ceph-mon[30869]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-09-10T22:56:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:56:59 smithi121 ceph-mon[35119]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-09-10T22:56:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:56:59 smithi117 ceph-mon[35081]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 826 op/s 2024-09-10T22:57:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 565.0 s, 2472.6 tps, lat 40.201 ms stddev 45.200 2024-09-10T22:57:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:01 smithi096 ceph-mon[30869]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 950 op/s 2024-09-10T22:57:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:01 smithi121 ceph-mon[35119]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 950 op/s 2024-09-10T22:57:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:01 smithi117 ceph-mon[35081]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 950 op/s 2024-09-10T22:57:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:03 smithi096 ceph-mon[30869]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-09-10T22:57:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:03 smithi121 ceph-mon[35119]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-09-10T22:57:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:03 smithi117 ceph-mon[35081]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 817 op/s 2024-09-10T22:57:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 -- 172.21.15.96:0/734520377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32406b3d0 msgr2=0x7fc3241055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 --2- 172.21.15.96:0/734520377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32406b3d0 0x7fc3241055d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc314009860 tx=0x7fc31402a340 comp rx=0 tx=0).stop 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 -- 172.21.15.96:0/734520377 shutdown_connections 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 --2- 172.21.15.96:0/734520377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 0x7fc32410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 --2- 172.21.15.96:0/734520377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc324105b10 0x7fc324107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 --2- 172.21.15.96:0/734520377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32406b3d0 0x7fc3241055d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.891+0000 7fc32a95d640 1 -- 172.21.15.96:0/734520377 >> 172.21.15.96:0/734520377 conn(0x7fc3240fd140 msgr2=0x7fc3240ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- 172.21.15.96:0/734520377 shutdown_connections 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- 172.21.15.96:0/734520377 wait complete. 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 Processor -- start 2024-09-10T22:57:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- start start 2024-09-10T22:57:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 0x7fc324199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 0x7fc32419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410e5d0 con 0x7fc32406b3d0 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410e450 con 0x7fc324105b10 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.892+0000 7fc32a95d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410e750 con 0x7fc324108680 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39242/0 (socket says 172.21.15.96:39242) 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 -- 172.21.15.96:0/3410784595 learned_addr learned my addr 172.21.15.96:0/3410784595 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc3237fe640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 0x7fc324199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc328efa640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 0x7fc32419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 -- 172.21.15.96:0/3410784595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 msgr2=0x7fc32419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 0x7fc32419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 -- 172.21.15.96:0/3410784595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 msgr2=0x7fc324199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 0x7fc324199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 -- 172.21.15.96:0/3410784595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32419a900 con 0x7fc32406b3d0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc3237fe640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 0x7fc324199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc323fff640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc314005940 tx=0x7fc314030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc3217fa640 1 -- 172.21.15.96:0/3410784595 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314040020 con 0x7fc32406b3d0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32419ab90 con 0x7fc32406b3d0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3241a12c0 con 0x7fc32406b3d0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc3217fa640 1 -- 172.21.15.96:0/3410784595 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc314030750 con 0x7fc32406b3d0 2024-09-10T22:57:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.893+0000 7fc3217fa640 1 -- 172.21.15.96:0/3410784595 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314037040 con 0x7fc32406b3d0 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 Processor -- start 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 -- start start 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc3217fa640 1 -- 172.21.15.96:0/3410784595 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc314003b30 con 0x7fc32406b3d0 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8d534f6f0 con 0x55f8d53db7d0 2024-09-10T22:57:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f8d533eb20 con 0x55f8d53e0c50 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.897+0000 7fc32d803740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f8d5342930 con 0x55f8d53dc110 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38518/0 (socket says 172.21.15.96:38518) 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 -- 172.21.15.96:0/3064531876 learned_addr learned my addr 172.21.15.96:0/3064531876 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc3237fe640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc328efa640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 -- 172.21.15.96:0/3064531876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 msgr2=0x55f8d53e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 -- 172.21.15.96:0/3064531876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 msgr2=0x55f8d53dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc323fff640 1 -- 172.21.15.96:0/3064531876 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d5304530 con 0x55f8d53dc110 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.898+0000 7fc3237fe640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc323fff640 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc314109d70 tx=0x7fc314109e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5b9755f0ec15c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc3217fa640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 0x7fc30c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc30a7fc640 1 -- 172.21.15.96:0/3064531876 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314115070 con 0x55f8d53dc110 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc328efa640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 0x7fc30c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc3217fa640 1 -- 172.21.15.96:0/3410784595 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc314105e50 con 0x7fc32406b3d0 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc30a7fc640 1 -- 172.21.15.96:0/3064531876 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3140cc120 con 0x55f8d53dc110 2024-09-10T22:57:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.899+0000 7fc30a7fc640 1 -- 172.21.15.96:0/3064531876 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140cbd80 con 0x55f8d53dc110 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- 172.21.15.96:0/3064531876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 msgr2=0x55f8d53e0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc328efa640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 0x7fc30c0b78a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc324199b40 tx=0x7fc31c0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc314109d70 tx=0x7fc314109e60 comp rx=0 tx=0).stop 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- 172.21.15.96:0/3064531876 shutdown_connections 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53e0710 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 --2- 172.21.15.96:0/3064531876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- 172.21.15.96:0/3064531876 >> 172.21.15.96:0/3064531876 conn(0x55f8d53d8930 msgr2=0x55f8d53d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- 172.21.15.96:0/3064531876 shutdown_connections 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- 172.21.15.96:0/3064531876 wait complete. 2024-09-10T22:57:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.900+0000 7fc32d803740 1 -- start start 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 Processor -- start 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53fee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f8d5276ab0 con 0x55f8d53db7d0 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f8d52e4730 con 0x55f8d53e0c50 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f8d53e3980 con 0x55f8d53dc110 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc32d803740 10 client.0 _ll_register_callbacks cb 0x55f8d53386b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc323fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc323fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39258/0 (socket says 172.21.15.96:39258) 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc323fff640 1 -- 172.21.15.96:0/1455162231 learned_addr learned my addr 172.21.15.96:0/1455162231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc3237fe640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc328efa640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc323fff640 1 -- 172.21.15.96:0/1455162231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 msgr2=0x55f8d53fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.901+0000 7fc323fff640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc323fff640 1 -- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 msgr2=0x55f8d53fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc323fff640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc323fff640 1 -- 172.21.15.96:0/1455162231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d50a0790 con 0x55f8d53db7d0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc323fff640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc3140c8be0 tx=0x7fc314109400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23374ca87a3709a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc3037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314115070 con 0x55f8d53db7d0 2024-09-10T22:57:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc314103a60 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3140cc120 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc3140c4de0 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc32d803740 10 client.17490 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.902+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8d5119620 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3140cc120 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc3140cc870 con 0x55f8d53db7d0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 10 client.17490 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 20 client.17490 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d54031c0 0x55f8d54055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 4 client.17490 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d5351640 con 0x55f8d54031c0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc3237fe640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d54031c0 0x55f8d54055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc3237fe640 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d54031c0 0x55f8d54055b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd6acfcb095e48e server_cookie=88861f1598af544c in_seq=0 out_seq=0 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.903+0000 7fc3037fe640 10 client.17490 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.904+0000 7fc3037fe640 1 -- 172.21.15.96:0/1455162231 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8d5351640 con 0x55f8d54031c0 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.904+0000 7fc3037fe640 10 client.17490 handle_command_reply: tid=0 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.904+0000 7fc32d803740 1 client.17490 shutdown 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 msgr2=0x55f8d53fa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc3140c8be0 tx=0x7fc314109400 comp rx=0 tx=0).stop 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 shutdown_connections 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d54031c0 0x55f8d54055b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53dc110 0x55f8d53fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 --2- 172.21.15.96:0/1455162231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53db7d0 0x55f8d53fa120 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 >> 172.21.15.96:0/1455162231 conn(0x55f8d53d8930 msgr2=0x55f8d53d7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc3037fe640 0 client.17490 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 shutdown_connections 2024-09-10T22:57:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 1 -- 172.21.15.96:0/1455162231 wait complete. 2024-09-10T22:57:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.905+0000 7fc32d803740 20 client.17490 trim_cache size 0 max 16384 2024-09-10T22:57:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 Processor -- start 2024-09-10T22:57:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 -- start start 2024-09-10T22:57:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53db7d0 0x55f8d53deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000e2e0 con 0x55f8d53dc110 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000fe20 con 0x55f8d53e0c50 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc32d803740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000e9b0 con 0x55f8d53db7d0 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc3237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc3237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39266/0 (socket says 172.21.15.96:39266) 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.922+0000 7fc3237fe640 1 -- 172.21.15.96:0/4009709401 learned_addr learned my addr 172.21.15.96:0/4009709401 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc328efa640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc323fff640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53db7d0 0x55f8d53deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 -- 172.21.15.96:0/4009709401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53db7d0 msgr2=0x55f8d53deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53db7d0 0x55f8d53deea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 -- 172.21.15.96:0/4009709401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 msgr2=0x55f8d53d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53d73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 -- 172.21.15.96:0/4009709401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d5405be0 con 0x55f8d53dc110 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc328efa640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55f8d5405e10 tx=0x7fc318004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d563f15853677875 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc309ffb640 1 -- 172.21.15.96:0/4009709401 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318005220 con 0x55f8d53dc110 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc309ffb640 1 -- 172.21.15.96:0/4009709401 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc318003f60 con 0x55f8d53dc110 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.923+0000 7fc309ffb640 1 -- 172.21.15.96:0/4009709401 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3180117c0 con 0x55f8d53dc110 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- 172.21.15.96:0/4009709401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 msgr2=0x55f8d53df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55f8d5405e10 tx=0x7fc318004610 comp rx=0 tx=0).stop 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- 172.21.15.96:0/4009709401 shutdown_connections 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53e0c50 0x55f8d53d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d53df3e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 --2- 172.21.15.96:0/4009709401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53db7d0 0x55f8d53deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- 172.21.15.96:0/4009709401 >> 172.21.15.96:0/4009709401 conn(0x55f8d53d8930 msgr2=0x55f8d53d95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- 172.21.15.96:0/4009709401 shutdown_connections 2024-09-10T22:57:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- 172.21.15.96:0/4009709401 wait complete. 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.924+0000 7fc32d803740 1 -- start start 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 Processor -- start 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53db7d0 0x55f8d540b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53e0c50 0x55f8d54163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000f750 con 0x55f8d53dc110 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000f080 con 0x55f8d53db7d0 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc31000c2c0 con 0x55f8d53e0c50 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc32d803740 10 client.0 _ll_register_callbacks cb 0x7fc31c009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39276/0 (socket says 172.21.15.96:39276) 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 -- 172.21.15.96:0/4020299931 learned_addr learned my addr 172.21.15.96:0/4020299931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc328efa640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53e0c50 0x55f8d54163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc323fff640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53db7d0 0x55f8d540b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 -- 172.21.15.96:0/4020299931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53e0c50 msgr2=0x55f8d54163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53e0c50 0x55f8d54163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 -- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53db7d0 msgr2=0x55f8d540b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53db7d0 0x55f8d540b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.925+0000 7fc3237fe640 1 -- 172.21.15.96:0/4020299931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d5416c00 con 0x55f8d53dc110 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3237fe640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc318026040 tx=0x7fc3180042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22887a42d2040571 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318031020 con 0x55f8d53dc110 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc3180046c0 con 0x55f8d53dc110 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8d534d0f0 con 0x55f8d53dc110 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc3180048c0 con 0x55f8d53dc110 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 10 client.17496 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8d54701e0 con 0x55f8d53dc110 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f8d534d0f0 con 0x55f8d53dc110 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc31802d440 con 0x55f8d53dc110 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 10 client.17496 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 20 client.17496 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d5470bc0 0x55f8d5472fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 4 client.17496 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.926+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8d534a520 con 0x55f8d5470bc0 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.927+0000 7fc323fff640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d5470bc0 0x55f8d5472fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.927+0000 7fc3037fe640 10 client.17496 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.927+0000 7fc323fff640 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d5470bc0 0x55f8d5472fe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b78be741a5dbd8b server_cookie=189ca79fc70faf26 in_seq=0 out_seq=0 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc3037fe640 1 -- 172.21.15.96:0/4020299931 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8d534a520 con 0x55f8d5470bc0 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc3037fe640 10 client.17496 handle_command_reply: tid=0 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 client.17496 shutdown 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 msgr2=0x55f8d5415da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc318026040 tx=0x7fc3180042e0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 shutdown_connections 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f8d5470bc0 0x55f8d5472fe0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc3037fe640 0 client.17496 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f8d53e0c50 0x55f8d54163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f8d53dc110 0x55f8d5415da0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 --2- 172.21.15.96:0/4020299931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8d53db7d0 0x55f8d540b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 >> 172.21.15.96:0/4020299931 conn(0x55f8d53d8930 msgr2=0x55f8d53dee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 shutdown_connections 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 1 -- 172.21.15.96:0/4020299931 wait complete. 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.947+0000 7fc32d803740 20 client.17496 trim_cache size 0 max 16384 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:57:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 msgr2=0x7fc30c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 0x7fc30c0b78a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc324199b40 tx=0x7fc31c0096a0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 msgr2=0x7fc324198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc314005940 tx=0x7fc314030e00 comp rx=0 tx=0).stop 2024-09-10T22:57:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 shutdown_connections 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc324105b10 0x7fc324199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc30c0b53e0 0x7fc30c0b78a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc324108680 0x7fc32419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 --2- 172.21.15.96:0/3410784595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc32406b3d0 0x7fc324198dc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 >> 172.21.15.96:0/3410784595 conn(0x7fc3240fd140 msgr2=0x7fc324103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 shutdown_connections 2024-09-10T22:57:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:03.948+0000 7fc32a95d640 1 -- 172.21.15.96:0/3410784595 wait complete. 2024-09-10T22:57:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:05 smithi096 ceph-mon[30869]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-09-10T22:57:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:05 smithi121 ceph-mon[35119]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-09-10T22:57:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:05 smithi117 ceph-mon[35081]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 763 op/s 2024-09-10T22:57:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 570.0 s, 1926.2 tps, lat 52.279 ms stddev 61.060 2024-09-10T22:57:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:07 smithi096 ceph-mon[30869]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-09-10T22:57:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:07 smithi121 ceph-mon[35119]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-09-10T22:57:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:07 smithi117 ceph-mon[35081]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-09-10T22:57:07.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:08.244 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 -- 172.21.15.96:0/1969727912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac108020 msgr2=0x7f92ac10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 --2- 172.21.15.96:0/1969727912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac108020 0x7f92ac10e820 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009ca0 tx=0x7f92a402a8d0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 -- 172.21.15.96:0/1969727912 shutdown_connections 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 --2- 172.21.15.96:0/1969727912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac108020 0x7f92ac10e820 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 --2- 172.21.15.96:0/1969727912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 0x7f92ac107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 --2- 172.21.15.96:0/1969727912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac101c90 0x7f92ac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 -- 172.21.15.96:0/1969727912 >> 172.21.15.96:0/1969727912 conn(0x7f92ac0fd700 msgr2=0x7f92ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 -- 172.21.15.96:0/1969727912 shutdown_connections 2024-09-10T22:57:08.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.823+0000 7f92b36b6640 1 -- 172.21.15.96:0/1969727912 wait complete. 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 Processor -- start 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 -- start start 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 0x7f92ac10b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac108020 0x7f92ac10b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac110c20 con 0x7f92ac1076a0 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac110aa0 con 0x7f92ac108020 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b36b6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac110da0 con 0x7f92ac101c90 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b0c51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 0x7f92ac10b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40200/0 (socket says 172.21.15.96:40200) 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 -- 172.21.15.96:0/2157867910 learned_addr learned my addr 172.21.15.96:0/2157867910 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1c53640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac108020 0x7f92ac10b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 -- 172.21.15.96:0/2157867910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac108020 msgr2=0x7f92ac10b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac108020 0x7f92ac10b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 -- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 msgr2=0x7f92ac10b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.824+0000 7f92b1452640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 0x7f92ac10b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.825+0000 7f92b1452640 1 -- 172.21.15.96:0/2157867910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac074cc0 con 0x7f92ac101c90 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.825+0000 7f92b1452640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b4a0 tx=0x7f92a800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.825+0000 7f92a27fc640 1 -- 172.21.15.96:0/2157867910 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8014070 con 0x7f92ac101c90 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.825+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac074fb0 con 0x7f92ac101c90 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.825+0000 7f92a27fc640 1 -- 172.21.15.96:0/2157867910 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a8010ac0 con 0x7f92ac101c90 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.826+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac1b4bc0 con 0x7f92ac101c90 2024-09-10T22:57:08.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.826+0000 7f92a27fc640 1 -- 172.21.15.96:0/2157867910 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8017c30 con 0x7f92ac101c90 2024-09-10T22:57:08.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.830+0000 7f92a27fc640 1 -- 172.21.15.96:0/2157867910 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f92a8010610 con 0x7f92ac101c90 2024-09-10T22:57:08.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.831+0000 7f92b5a8b740 1 Processor -- start 2024-09-10T22:57:08.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.831+0000 7f92b5a8b740 1 -- start start 2024-09-10T22:57:08.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92a27fc640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 0x7f92980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 0x5589b84c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b8432e50 con 0x5589b84bc9c0 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b8422280 con 0x5589b84bc0a0 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b8426090 con 0x5589b84c8950 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92a27fc640 1 -- 172.21.15.96:0/2157867910 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f92a80e54c0 con 0x7f92ac101c90 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b0c51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 0x5589b84c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38388/0 (socket says 172.21.15.96:38388) 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b1c53640 1 -- 172.21.15.96:0/2219597914 learned_addr learned my addr 172.21.15.96:0/2219597914 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:08.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b1c53640 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.832+0000 7f92b1452640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 0x7f92980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 -- 172.21.15.96:0/2219597914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 msgr2=0x5589b84cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 -- 172.21.15.96:0/2219597914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 msgr2=0x5589b84bc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 -- 172.21.15.96:0/2219597914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b8180650 con 0x5589b84bc9c0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b1c53640 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b0c51640 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 0x5589b84c8410 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bbc0 tx=0x7f929c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a94d5bb188818b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92b1452640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 0x7f92980b7820 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f92a800be60 tx=0x7f92a8017650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92977fe640 1 -- 172.21.15.96:0/2219597914 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c004030 con 0x5589b84bc9c0 2024-09-10T22:57:08.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92977fe640 1 -- 172.21.15.96:0/2219597914 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929c0035f0 con 0x5589b84bc9c0 2024-09-10T22:57:08.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.833+0000 7f92977fe640 1 -- 172.21.15.96:0/2219597914 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0116a0 con 0x5589b84bc9c0 2024-09-10T22:57:08.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 -- 172.21.15.96:0/2219597914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 msgr2=0x5589b84c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 0x5589b84c8410 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bbc0 tx=0x7f929c0107a0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 -- 172.21.15.96:0/2219597914 shutdown_connections 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc9c0 0x5589b84c8410 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/2219597914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 -- 172.21.15.96:0/2219597914 >> 172.21.15.96:0/2219597914 conn(0x5589b84b9ed0 msgr2=0x5589b84bb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 -- 172.21.15.96:0/2219597914 shutdown_connections 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.834+0000 7f92b5a8b740 1 -- 172.21.15.96:0/2219597914 wait complete. 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 -- start start 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 Processor -- start 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84c8950 0x5589b84c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b8283700 con 0x5589b84c8950 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b83bd110 con 0x5589b84bc0a0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b84cb5d0 con 0x5589b84bc9c0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b5a8b740 10 client.0 _ll_register_callbacks cb 0x5589b83d6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.835+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38398/0 (socket says 172.21.15.96:38398) 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 -- 172.21.15.96:0/3044620113 learned_addr learned my addr 172.21.15.96:0/3044620113 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84c8950 0x5589b84c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b0c51640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 -- 172.21.15.96:0/3044620113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 msgr2=0x5589b84c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 -- 172.21.15.96:0/3044620113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84c8950 msgr2=0x5589b84c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84c8950 0x5589b84c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 -- 172.21.15.96:0/3044620113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b83eac40 con 0x5589b84bc0a0 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b0c51640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f92b1c53640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92a402aec0 tx=0x7f92a402f470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fafdf2128060cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.836+0000 7f9294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:08.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.839+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4032ed0 con 0x5589b84bc0a0 2024-09-10T22:57:08.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.839+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f92a4003db0 con 0x5589b84bc0a0 2024-09-10T22:57:08.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.839+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c0035f0 con 0x5589b84bc0a0 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.840+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92a4004040 con 0x5589b84bc0a0 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.840+0000 7f92b5a8b740 10 client.27089 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.840+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b84d6fd0 con 0x5589b84bc0a0 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.840+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929c0035f0 con 0x5589b84bc0a0 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f92a402f9a0 con 0x5589b84bc0a0 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 10 client.27089 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 10 client.27089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 20 client.27089 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b84d8490 0x5589b84da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 4 client.27089 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b8434da0 con 0x5589b84d8490 2024-09-10T22:57:08.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.841+0000 7f92b0c51640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b84d8490 0x5589b84da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f92b0c51640 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b84d8490 0x5589b84da880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d0a15112dc5f8c server_cookie=73be3bf54dadb46f in_seq=0 out_seq=0 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f9294ff9640 10 client.27089 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f9294ff9640 1 -- 172.21.15.96:0/3044620113 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589b8434da0 con 0x5589b84d8490 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f9294ff9640 10 client.27089 handle_command_reply: tid=0 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f92b5a8b740 1 client.27089 shutdown 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 msgr2=0x5589b84c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.842+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92a402aec0 tx=0x7f92a402f470 comp rx=0 tx=0).stop 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 shutdown_connections 2024-09-10T22:57:08.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b84d8490 0x5589b84da880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84c8950 0x5589b84c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/3044620113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc0a0 0x5589b84c82c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 >> 172.21.15.96:0/3044620113 conn(0x5589b84b9ed0 msgr2=0x5589b84bfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f9294ff9640 0 client.27089 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 shutdown_connections 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 1 -- 172.21.15.96:0/3044620113 wait complete. 2024-09-10T22:57:08.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.843+0000 7f92b5a8b740 20 client.27089 trim_cache size 0 max 16384 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.872+0000 7f92b5a8b740 1 Processor -- start 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.872+0000 7f92b5a8b740 1 -- start start 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.872+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 0x5589b84c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1bd270 con 0x5589b84bc0a0 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1bedb0 con 0x5589b84bc9c0 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1bd940 con 0x5589b84c8950 2024-09-10T22:57:08.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b0c51640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 0x5589b84c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40218/0 (socket says 172.21.15.96:40218) 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b0c51640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 0x5589b84c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38402/0 (socket says 172.21.15.96:38402) 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 -- 172.21.15.96:0/4104295538 learned_addr learned my addr 172.21.15.96:0/4104295538 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 -- 172.21.15.96:0/4104295538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 msgr2=0x5589b84c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 0x5589b84c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 -- 172.21.15.96:0/4104295538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 msgr2=0x5589b84d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.873+0000 7f92b1c53640 1 -- 172.21.15.96:0/4104295538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b84c54c0 con 0x5589b84c8950 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.874+0000 7f92b1c53640 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92a4031880 tx=0x7f92a402a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5410c9ab640e6ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.874+0000 7f9296ffd640 1 -- 172.21.15.96:0/4104295538 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4003db0 con 0x5589b84c8950 2024-09-10T22:57:08.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.874+0000 7f9296ffd640 1 -- 172.21.15.96:0/4104295538 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a4036040 con 0x5589b84c8950 2024-09-10T22:57:08.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.874+0000 7f9296ffd640 1 -- 172.21.15.96:0/4104295538 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4030ad0 con 0x5589b84c8950 2024-09-10T22:57:08.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 -- 172.21.15.96:0/4104295538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 msgr2=0x5589b84ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f92a4031880 tx=0x7f92a402a8d0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 -- 172.21.15.96:0/4104295538 shutdown_connections 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84bc9c0 0x5589b84c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84c8950 0x5589b84ca280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/4104295538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 -- 172.21.15.96:0/4104295538 >> 172.21.15.96:0/4104295538 conn(0x5589b84b9ed0 msgr2=0x5589b84c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 -- 172.21.15.96:0/4104295538 shutdown_connections 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.875+0000 7f92b5a8b740 1 -- 172.21.15.96:0/4104295538 wait complete. 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 -- start start 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 Processor -- start 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 0x5589b84cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1be6e0 con 0x5589b84bc0a0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1be010 con 0x5589b84c8950 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1bb270 con 0x5589b84bc9c0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.876+0000 7f92b5a8b740 10 client.0 _ll_register_callbacks cb 0x7f92a4009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.877+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.877+0000 7f92b1c53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52814/0 (socket says 172.21.15.96:52814) 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.877+0000 7f92b1c53640 1 -- 172.21.15.96:0/1954523549 learned_addr learned my addr 172.21.15.96:0/1954523549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.877+0000 7f92b0c51640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.877+0000 7f92b1c53640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 0x5589b84cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 -- 172.21.15.96:0/1954523549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 msgr2=0x5589b84cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 -- 172.21.15.96:0/1954523549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 msgr2=0x5589b84cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:08.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 -- 172.21.15.96:0/1954523549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b8545460 con 0x5589b84c8950 2024-09-10T22:57:08.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f92b1c53640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 0x5589b84cc350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92a4002230 tx=0x7f92a4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f547b1f896ec21b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:08.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.878+0000 7f9294ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.879+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4032eb0 con 0x5589b84c8950 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.879+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f92a402f620 con 0x5589b84c8950 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.879+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a4036040 con 0x5589b84c8950 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.879+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92a402f820 con 0x5589b84c8950 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.880+0000 7f92b5a8b740 10 client.27095 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:08.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.880+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b8546770 con 0x5589b84c8950 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a402f9c0 con 0x5589b84c8950 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f92a4030b20 con 0x5589b84c8950 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 10 client.27095 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 20 client.27095 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8546dd0 0x5589b85491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 4 client.27095 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:08.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.881+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f92a4036040 con 0x5589b8546dd0 2024-09-10T22:57:08.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.882+0000 7f92b0c51640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8546dd0 0x5589b85491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.883+0000 7f9294ff9640 10 client.27095 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.883+0000 7f92b0c51640 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8546dd0 0x5589b85491c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81fd715444430f29 server_cookie=224838775dc5dddc in_seq=0 out_seq=0 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.883+0000 7f9294ff9640 1 -- 172.21.15.96:0/1954523549 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x7f92a4036040 con 0x5589b8546dd0 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.883+0000 7f9294ff9640 10 client.27095 handle_command_reply: tid=0 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.883+0000 7f92b5a8b740 1 client.27095 shutdown 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 msgr2=0x5589b84cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 0x5589b84cc350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92a4002230 tx=0x7f92a4005fd0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 shutdown_connections 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8546dd0 0x5589b85491c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b84c8950 0x5589b84cc350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b84bc9c0 0x5589b84cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f9294ff9640 0 client.27095 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 --2- 172.21.15.96:0/1954523549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b84bc0a0 0x5589b84cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 >> 172.21.15.96:0/1954523549 conn(0x5589b84b9ed0 msgr2=0x5589b84b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 shutdown_connections 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 1 -- 172.21.15.96:0/1954523549 wait complete. 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.884+0000 7f92b5a8b740 20 client.27095 trim_cache size 0 max 16384 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:57:08.887 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:57:08.889 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.891 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.892 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:08.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:08.895 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:08.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:08.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:08.895 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 msgr2=0x7f92980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 0x7f92980b7820 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f92a800be60 tx=0x7f92a8017650 comp rx=0 tx=0).stop 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 msgr2=0x7f92ac10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b4a0 tx=0x7f92a800b960 comp rx=0 tx=0).stop 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 shutdown_connections 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92ac108020 0x7f92ac10b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f92980b5360 0x7f92980b7820 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92ac1076a0 0x7f92ac10b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 --2- 172.21.15.96:0/2157867910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92ac101c90 0x7f92ac10aaf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:08.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 >> 172.21.15.96:0/2157867910 conn(0x7f92ac0fd700 msgr2=0x7f92ac0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:08.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 shutdown_connections 2024-09-10T22:57:08.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:08.885+0000 7f92b36b6640 1 -- 172.21.15.96:0/2157867910 wait complete. 2024-09-10T22:57:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:09 smithi096 ceph-mon[30869]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.9 MiB/s wr, 627 op/s 2024-09-10T22:57:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:09 smithi121 ceph-mon[35119]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.9 MiB/s wr, 627 op/s 2024-09-10T22:57:09.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:09 smithi117 ceph-mon[35081]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.9 MiB/s wr, 627 op/s 2024-09-10T22:57:10.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:57:10.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:57:10.399 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 575.0 s, 2598.6 tps, lat 38.227 ms stddev 42.460 2024-09-10T22:57:10.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 -- 172.21.15.96:0/2275330567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 msgr2=0x7fe798103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/2275330567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe798103f10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe788009900 tx=0x7fe78802a410 comp rx=0 tx=0).stop 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 -- 172.21.15.96:0/2275330567 shutdown_connections 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/2275330567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe79810d0e0 0x7fe79810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/2275330567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe798104530 0x7fe79810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/2275330567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe798103f10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 -- 172.21.15.96:0/2275330567 >> 172.21.15.96:0/2275330567 conn(0x7fe7980fd700 msgr2=0x7fe7980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 -- 172.21.15.96:0/2275330567 shutdown_connections 2024-09-10T22:57:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.972+0000 7fe79c9ab640 1 -- 172.21.15.96:0/2275330567 wait complete. 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 Processor -- start 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 -- start start 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe798104530 0x7fe798101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 0x7fe798102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe798112db0 con 0x7fe798103b30 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe798112c30 con 0x7fe798104530 2024-09-10T22:57:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79c9ab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe798112f30 con 0x7fe79810d0e0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52834/0 (socket says 172.21.15.96:52834) 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe79659c640 1 -- 172.21.15.96:0/286474460 learned_addr learned my addr 172.21.15.96:0/286474460 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe796d9d640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 0x7fe798102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.973+0000 7fe795d9b640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe798104530 0x7fe798101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 -- 172.21.15.96:0/286474460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe798104530 msgr2=0x7fe798101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe798104530 0x7fe798101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 -- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 msgr2=0x7fe7981019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 -- 172.21.15.96:0/286474460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7981a76e0 con 0x7fe79810d0e0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe79659c640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe796d9d640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 0x7fe798102430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe79000da10 tx=0x7fe79000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000bd70 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe790003e20 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981a79d0 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900106c0 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.974+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a7ec0 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.975+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe798103f10 con 0x7fe79810d0e0 2024-09-10T22:57:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.977+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe790024060 con 0x7fe79810d0e0 2024-09-10T22:57:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.977+0000 7fe7877fe640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 0x7fe7800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.978+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe7900e5b90 con 0x7fe79810d0e0 2024-09-10T22:57:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.978+0000 7fe79659c640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 0x7fe7800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.978+0000 7fe79659c640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 0x7fe7800b7820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7880096c0 tx=0x7fe788005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:10.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:10.981+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe7900aca50 con 0x7fe79810d0e0 2024-09-10T22:57:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.306+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79802e5c0 con 0x7fe79810d0e0 2024-09-10T22:57:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.308+0000 7fe7877fe640 1 -- 172.21.15.96:0/286474460 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fe7900b5070 con 0x7fe79810d0e0 2024-09-10T22:57:11.310 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:57:11.310 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:57:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:57:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 msgr2=0x7fe7800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 0x7fe7800b7820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7880096c0 tx=0x7fe788005a20 comp rx=0 tx=0).stop 2024-09-10T22:57:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 msgr2=0x7fe798102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 0x7fe798102430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe79000da10 tx=0x7fe79000ded0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 shutdown_connections 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.311+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe798104530 0x7fe798101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe7800b5360 0x7fe7800b7820 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe79810d0e0 0x7fe798102430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 --2- 172.21.15.96:0/286474460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe798103b30 0x7fe7981019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 >> 172.21.15.96:0/286474460 conn(0x7fe7980fd700 msgr2=0x7fe7980fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 shutdown_connections 2024-09-10T22:57:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:11.312+0000 7fe79c9ab640 1 -- 172.21.15.96:0/286474460 wait complete. 2024-09-10T22:57:11.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:11 smithi096 ceph-mon[30869]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:57:11.716 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:11 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/286474460' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:11 smithi121 ceph-mon[35119]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:57:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:11 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/286474460' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:11 smithi117 ceph-mon[35081]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T22:57:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:11 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/286474460' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:57:12.127 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:57:12.502 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:13.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 -- 172.21.15.96:0/3160071340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128010a530 msgr2=0x7f128010a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3160071340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128010a530 0x7f128010a930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1278009be0 tx=0x7f127802a790 comp rx=0 tx=0).stop 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 -- 172.21.15.96:0/3160071340 shutdown_connections 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3160071340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128010a530 0x7f128010a930 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1278009be0 tx=0x7f127802a790 comp rx=0 tx=0).stop 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3160071340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f128011c0a0 0x7f128011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3160071340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 -- 172.21.15.96:0/3160071340 >> 172.21.15.96:0/3160071340 conn(0x7f128006dfc0 msgr2=0x7f128006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.041+0000 7f1285e4b640 1 -- 172.21.15.96:0/3160071340 shutdown_connections 2024-09-10T22:57:13.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.042+0000 7f1285e4b640 1 -- 172.21.15.96:0/3160071340 wait complete. 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.042+0000 7f1285e4b640 1 Processor -- start 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 -- start start 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801a8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 0x7f12801a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12801adb40 0x7f12801adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1280121b20 con 0x7f12801adb40 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12801219a0 con 0x7f128011c0a0 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f1285e4b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1280121ca0 con 0x7f128010a150 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f127effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 0x7f12801a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f127effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 0x7f12801a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38452/0 (socket says 172.21.15.96:38452) 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f127effd640 1 -- 172.21.15.96:0/3716764071 learned_addr learned my addr 172.21.15.96:0/3716764071 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.043+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.044+0000 7f127f7fe640 1 -- 172.21.15.96:0/3716764071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 msgr2=0x7f12801a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.044+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 0x7f12801a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.044+0000 7f127f7fe640 1 -- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12801adb40 msgr2=0x7f12801adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.044+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12801adb40 0x7f12801adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.044+0000 7f127f7fe640 1 -- 172.21.15.96:0/3716764071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12801ae6d0 con 0x7f128010a150 2024-09-10T22:57:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.045+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801a8260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f127400e9f0 tx=0x7f127400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.045+0000 7f127cff9640 1 -- 172.21.15.96:0/3716764071 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127400ccc0 con 0x7f128010a150 2024-09-10T22:57:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.045+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128007b3b0 con 0x7f128010a150 2024-09-10T22:57:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.045+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128007b910 con 0x7f128010a150 2024-09-10T22:57:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.047+0000 7f127cff9640 1 -- 172.21.15.96:0/3716764071 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12740108e0 con 0x7f128010a150 2024-09-10T22:57:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f127cff9640 1 -- 172.21.15.96:0/3716764071 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127400ccc0 con 0x7f128010a150 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 Processor -- start 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 -- start start 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c9128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c921170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c49c888ea0 con 0x55c49c9124d0 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c49c8782d0 con 0x55c49c91ed80 2024-09-10T22:57:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.049+0000 7f1288220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c49c87c0e0 con 0x55c49c912df0 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40268/0 (socket says 172.21.15.96:40268) 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 -- 172.21.15.96:0/3198838238 learned_addr learned my addr 172.21.15.96:0/3198838238 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127effd640 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c921170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 -- 172.21.15.96:0/3198838238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 msgr2=0x55c49c921170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c921170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 -- 172.21.15.96:0/3198838238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 msgr2=0x55c49c9128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c9128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.050+0000 7f127f7fe640 1 -- 172.21.15.96:0/3198838238 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49c5d6650 con 0x55c49c912df0 2024-09-10T22:57:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.051+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f127401a040 tx=0x7f1274018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f75253009e4f371 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.058+0000 7f127cff9640 1 -- 172.21.15.96:0/3716764071 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1274004890 con 0x7f128010a150 2024-09-10T22:57:13.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.060+0000 7f127cff9640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 0x7f12680b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.060+0000 7f127cff9640 1 -- 172.21.15.96:0/3716764071 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f127400cac0 con 0x7f128010a150 2024-09-10T22:57:13.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.060+0000 7f1265ffb640 1 -- 172.21.15.96:0/3198838238 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740aad00 con 0x55c49c912df0 2024-09-10T22:57:13.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.061+0000 7f1265ffb640 1 -- 172.21.15.96:0/3198838238 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12740108e0 con 0x55c49c912df0 2024-09-10T22:57:13.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.061+0000 7f1265ffb640 1 -- 172.21.15.96:0/3198838238 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740aaea0 con 0x55c49c912df0 2024-09-10T22:57:13.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.061+0000 7f127ffff640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 0x7f12680b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.062+0000 7f1288220740 1 -- 172.21.15.96:0/3198838238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 msgr2=0x55c49c91e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.062+0000 7f1288220740 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f127401a040 tx=0x7f1274018420 comp rx=0 tx=0).stop 2024-09-10T22:57:13.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.063+0000 7f127ffff640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 0x7f12680b7b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1270009510 tx=0x7f1270009480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.064+0000 7f1288220740 1 -- 172.21.15.96:0/3198838238 shutdown_connections 2024-09-10T22:57:13.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.064+0000 7f1288220740 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c921170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.064+0000 7f1288220740 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91e840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.064+0000 7f1288220740 1 --2- 172.21.15.96:0/3198838238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c9128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.064+0000 7f1288220740 1 -- 172.21.15.96:0/3198838238 >> 172.21.15.96:0/3198838238 conn(0x55c49c910300 msgr2=0x55c49c9114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.065+0000 7f1288220740 1 -- 172.21.15.96:0/3198838238 shutdown_connections 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.065+0000 7f1288220740 1 -- 172.21.15.96:0/3198838238 wait complete. 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.065+0000 7f1288220740 1 -- start start 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.065+0000 7f1288220740 1 Processor -- start 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c91e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c91c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c49c6d9700 con 0x55c49c9124d0 2024-09-10T22:57:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c49c813160 con 0x55c49c91ed80 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c49c9218b0 con 0x55c49c912df0 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f1288220740 10 client.0 _ll_register_callbacks cb 0x55c49c82c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f127effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f127f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c91e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f127effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40276/0 (socket says 172.21.15.96:40276) 2024-09-10T22:57:13.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.066+0000 7f127effd640 1 -- 172.21.15.96:0/872991873 learned_addr learned my addr 172.21.15.96:0/872991873 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:13.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127ffff640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c91c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127effd640 1 -- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 msgr2=0x55c49c91c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127effd640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c91c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127effd640 1 -- 172.21.15.96:0/872991873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 msgr2=0x55c49c91e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127effd640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c91e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127effd640 1 -- 172.21.15.96:0/872991873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49c840c90 con 0x55c49c912df0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.067+0000 7f127ffff640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c91c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.068+0000 7f127effd640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1278009b30 tx=0x7f1278032e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce39e35dbc17f58d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.068+0000 7f125effd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.068+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278042070 con 0x55c49c912df0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.069+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12740108e0 con 0x55c49c912df0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.069+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1278003970 con 0x55c49c912df0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.069+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1278036040 con 0x55c49c912df0 2024-09-10T22:57:13.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.069+0000 7f1288220740 10 client.27169 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:13.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.069+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c49c923480 con 0x55c49c912df0 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12740108e0 con 0x55c49c912df0 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f12780041d0 con 0x55c49c912df0 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 10 client.27169 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 10 client.27169 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 20 client.27169 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c924d50 0x55c49c927190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 4 client.27169 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:13.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c49c88adf0 con 0x55c49c924d50 2024-09-10T22:57:13.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.072+0000 7f127f7fe640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c924d50 0x55c49c927190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.074+0000 7f125effd640 10 client.27169 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.074+0000 7f127f7fe640 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c924d50 0x55c49c927190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d15726081bc2988 server_cookie=2d3be59839cdd7a2 in_seq=0 out_seq=0 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f125effd640 1 -- 172.21.15.96:0/872991873 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c49c88adf0 con 0x55c49c924d50 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f125effd640 10 client.27169 handle_command_reply: tid=0 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 client.27169 shutdown 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 msgr2=0x55c49c91ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1278009b30 tx=0x7f1278032e00 comp rx=0 tx=0).stop 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 shutdown_connections 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c924d50 0x55c49c927190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c91c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c91ba80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 --2- 172.21.15.96:0/872991873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c91e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 >> 172.21.15.96:0/872991873 conn(0x55c49c910300 msgr2=0x55c49c915ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.075+0000 7f125effd640 0 client.27169 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:57:13.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.076+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 shutdown_connections 2024-09-10T22:57:13.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.076+0000 7f1288220740 1 -- 172.21.15.96:0/872991873 wait complete. 2024-09-10T22:57:13.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.076+0000 7f1288220740 20 client.27169 trim_cache size 0 max 16384 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 Processor -- start 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 -- start start 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c9124d0 0x55c49c921000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c916d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1280119a70 con 0x55c49c91ed80 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f128007d150 con 0x55c49c9124d0 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f1288220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f128007bd00 con 0x55c49c912df0 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.095+0000 7f127ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52892/0 (socket says 172.21.15.96:52892) 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c916d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 -- 172.21.15.96:0/2491407425 learned_addr learned my addr 172.21.15.96:0/2491407425 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127f7fe640 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c9124d0 0x55c49c921000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 -- 172.21.15.96:0/2491407425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 msgr2=0x55c49c916d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c916d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 -- 172.21.15.96:0/2491407425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c9124d0 msgr2=0x55c49c921000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c9124d0 0x55c49c921000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 -- 172.21.15.96:0/2491407425 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49c910d80 con 0x55c49c91ed80 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f127ffff640 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1270004700 tx=0x7f1270011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e74fe588bfb2cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f12657fa640 1 -- 172.21.15.96:0/2491407425 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270025430 con 0x55c49c91ed80 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f12657fa640 1 -- 172.21.15.96:0/2491407425 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f127001c8d0 con 0x55c49c91ed80 2024-09-10T22:57:13.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.096+0000 7f12657fa640 1 -- 172.21.15.96:0/2491407425 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127001b7d0 con 0x55c49c91ed80 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 -- 172.21.15.96:0/2491407425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 msgr2=0x55c49c917270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1270004700 tx=0x7f1270011000 comp rx=0 tx=0).stop 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 -- 172.21.15.96:0/2491407425 shutdown_connections 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c91ed80 0x55c49c917270 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c916d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 --2- 172.21.15.96:0/2491407425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c9124d0 0x55c49c921000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 -- 172.21.15.96:0/2491407425 >> 172.21.15.96:0/2491407425 conn(0x55c49c910300 msgr2=0x55c49c91b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 -- 172.21.15.96:0/2491407425 shutdown_connections 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.097+0000 7f1288220740 1 -- 172.21.15.96:0/2491407425 wait complete. 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.098+0000 7f1288220740 1 -- start start 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.098+0000 7f1288220740 1 Processor -- start 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c929940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c92a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f128007ca80 con 0x55c49c9124d0 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f128007c3b0 con 0x55c49c91ed80 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1280117a50 con 0x55c49c912df0 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f1288220740 10 client.0 _ll_register_callbacks cb 0x7f1274005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f127f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f127f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52904/0 (socket says 172.21.15.96:52904) 2024-09-10T22:57:13.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f127f7fe640 1 -- 172.21.15.96:0/3810848333 learned_addr learned my addr 172.21.15.96:0/3810848333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.099+0000 7f127effd640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c929940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127ffff640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c92a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 -- 172.21.15.96:0/3810848333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 msgr2=0x55c49c929940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c929940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 -- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 msgr2=0x55c49c92a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c92a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 -- 172.21.15.96:0/3810848333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49c910130 con 0x55c49c9124d0 2024-09-10T22:57:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127ffff640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c92a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.100+0000 7f127f7fe640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f12740ab3b0 tx=0x7f127400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cafc4bd3c7e438b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:13.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.101+0000 7f125effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.101+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740ab070 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.101+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f127001c8d0 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.101+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f12740d50f0 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f12740ab210 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 10 client.17529 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c49c9bdc20 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f127001c8d0 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1274004890 con 0x55c49c9124d0 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 10 client.17529 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 10 client.17529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 20 client.17529 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c9bdeb0 0x55c49c9c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 4 client.17529 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.102+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c49c8868a0 con 0x55c49c9bdeb0 2024-09-10T22:57:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.103+0000 7f127effd640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c9bdeb0 0x55c49c9c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:13.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f127effd640 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c9bdeb0 0x55c49c9c02a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=856f0f50e6f2f32e server_cookie=f8953f93bab0ede7 in_seq=0 out_seq=0 2024-09-10T22:57:13.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f125effd640 10 client.17529 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:57:13.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f125effd640 1 -- 172.21.15.96:0/3810848333 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c49c8868a0 con 0x55c49c9bdeb0 2024-09-10T22:57:13.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f125effd640 10 client.17529 handle_command_reply: tid=0 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f1288220740 1 client.17529 shutdown 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 msgr2=0x55c49c92d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.104+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f12740ab3b0 tx=0x7f127400cf40 comp rx=0 tx=0).stop 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 shutdown_connections 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c49c9bdeb0 0x55c49c9c02a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c49c91ed80 0x55c49c92a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c49c912df0 0x55c49c929940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 --2- 172.21.15.96:0/3810848333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c49c9124d0 0x55c49c92d460 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 >> 172.21.15.96:0/3810848333 conn(0x55c49c910300 msgr2=0x55c49c916d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f125effd640 0 client.17529 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.105+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 shutdown_connections 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.106+0000 7f1288220740 1 -- 172.21.15.96:0/3810848333 wait complete. 2024-09-10T22:57:13.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.106+0000 7f1288220740 20 client.17529 trim_cache size 0 max 16384 2024-09-10T22:57:13.110 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 msgr2=0x7f12680b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 0x7f12680b7b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1270009510 tx=0x7f1270009480 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 msgr2=0x7f12801a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801a8260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f127400e9f0 tx=0x7f127400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 shutdown_connections 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f128011c0a0 0x7f12801a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f12680b56d0 0x7f12680b7b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12801adb40 0x7f12801adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 --2- 172.21.15.96:0/3716764071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f128010a150 0x7f12801a8260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:13.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 >> 172.21.15.96:0/3716764071 conn(0x7f128006dfc0 msgr2=0x7f128011c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:13.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.107+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 shutdown_connections 2024-09-10T22:57:13.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:13.108+0000 7f1285e4b640 1 -- 172.21.15.96:0/3716764071 wait complete. 2024-09-10T22:57:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:13 smithi096 ceph-mon[30869]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2024-09-10T22:57:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:13 smithi121 ceph-mon[35119]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2024-09-10T22:57:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:13 smithi117 ceph-mon[35081]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2024-09-10T22:57:14.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:57:15.007 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:15.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 -- 172.21.15.96:0/3643160751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f601087f0 msgr2=0x7f5f60108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 --2- 172.21.15.96:0/3643160751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f601087f0 0x7f5f60108c30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009860 tx=0x7f5f4802a340 comp rx=0 tx=0).stop 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 -- 172.21.15.96:0/3643160751 shutdown_connections 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 --2- 172.21.15.96:0/3643160751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 --2- 172.21.15.96:0/3643160751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f601087f0 0x7f5f60108c30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 --2- 172.21.15.96:0/3643160751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f60102de0 0x7f5f601031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 -- 172.21.15.96:0/3643160751 >> 172.21.15.96:0/3643160751 conn(0x7f5f600fe830 msgr2=0x7f5f60100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 -- 172.21.15.96:0/3643160751 shutdown_connections 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.520+0000 7f5f66159640 1 -- 172.21.15.96:0/3643160751 wait complete. 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 Processor -- start 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 -- start start 2024-09-10T22:57:15.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 0x7f5f601997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60112dc0 con 0x7f5f60102de0 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60112c40 con 0x7f5f601087f0 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f66159640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60112f40 con 0x7f5f60109170 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52918/0 (socket says 172.21.15.96:52918) 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3867270705 learned_addr learned my addr 172.21.15.96:0/3867270705 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 0x7f5f601997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3867270705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 msgr2=0x7f5f6019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3867270705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 msgr2=0x7f5f601997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 0x7f5f601997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3867270705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6019f340 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 0x7f5f601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.521+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000e9c0 tx=0x7f5f5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f5d7fa640 1 -- 172.21.15.96:0/3867270705 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5000ccc0 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6019f630 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f5d7fa640 1 -- 172.21.15.96:0/3867270705 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f500054d0 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f5d7fa640 1 -- 172.21.15.96:0/3867270705 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50010430 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.522+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f601a5ec0 con 0x7f5f60102de0 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.526+0000 7f5f6852e740 1 Processor -- start 2024-09-10T22:57:15.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.526+0000 7f5f6852e740 1 -- start start 2024-09-10T22:57:15.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93edc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93edf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc93e46ea0 con 0x55fc93ed0df0 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc93e362d0 con 0x55fc93ed04d0 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f6852e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fc93e3a0e0 con 0x55fc93edcd80 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93edf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38518/0 (socket says 172.21.15.96:38518) 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 -- 172.21.15.96:0/1454642073 learned_addr learned my addr 172.21.15.96:0/1454642073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 -- 172.21.15.96:0/1454642073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 msgr2=0x55fc93edf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93edf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 -- 172.21.15.96:0/1454642073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 msgr2=0x55fc93edc840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:57:15.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93edc840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 -- 172.21.15.96:0/1454642073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc93b94650 con 0x55fc93ed04d0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.527+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009fd0 tx=0x7f5f580108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8abc0bbe54948ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.528+0000 7f5f55ffb640 1 -- 172.21.15.96:0/1454642073 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58011760 con 0x55fc93ed04d0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.528+0000 7f5f55ffb640 1 -- 172.21.15.96:0/1454642073 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58011db0 con 0x55fc93ed04d0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.528+0000 7f5f55ffb640 1 -- 172.21.15.96:0/1454642073 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800e3c0 con 0x55fc93ed04d0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.528+0000 7f5f5d7fa640 1 -- 172.21.15.96:0/3867270705 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5f50004f80 con 0x7f5f60102de0 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.528+0000 7f5f6852e740 1 -- 172.21.15.96:0/1454642073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 msgr2=0x55fc93ed08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009fd0 tx=0x7f5f580108e0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 -- 172.21.15.96:0/1454642073 shutdown_connections 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93edf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93edc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 --2- 172.21.15.96:0/1454642073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93ed08b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f6852e740 1 -- 172.21.15.96:0/1454642073 >> 172.21.15.96:0/1454642073 conn(0x55fc93ece300 msgr2=0x55fc93ecf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f5d7fa640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 0x7f5f4c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.529+0000 7f5f5d7fa640 1 -- 172.21.15.96:0/3867270705 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f500e5ff0 con 0x7f5f60102de0 2024-09-10T22:57:15.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.530+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 0x7f5f4c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.535+0000 7f5f6852e740 1 -- 172.21.15.96:0/1454642073 shutdown_connections 2024-09-10T22:57:15.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.539+0000 7f5f6852e740 1 -- 172.21.15.96:0/1454642073 wait complete. 2024-09-10T22:57:15.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.539+0000 7f5f6852e740 1 -- start start 2024-09-10T22:57:15.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.539+0000 7f5f6852e740 1 Processor -- start 2024-09-10T22:57:15.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.540+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 0x7f5f4c0b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f4802a930 tx=0x7f5f4802c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93edc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ed9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc93c97700 con 0x55fc93ed0df0 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc93dd1160 con 0x55fc93ed04d0 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fc93ee3eb0 con 0x55fc93edcd80 2024-09-10T22:57:15.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.542+0000 7f5f6852e740 10 client.0 _ll_register_callbacks cb 0x55fc93dea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:15.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.543+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.543+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52928/0 (socket says 172.21.15.96:52928) 2024-09-10T22:57:15.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.543+0000 7f5f57fff640 1 -- 172.21.15.96:0/423010618 learned_addr learned my addr 172.21.15.96:0/423010618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:15.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.546+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ed9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.547+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93edc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.550+0000 7f5f57fff640 1 -- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 msgr2=0x55fc93ed9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.550+0000 7f5f57fff640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ed9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.550+0000 7f5f57fff640 1 -- 172.21.15.96:0/423010618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 msgr2=0x55fc93edc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.550+0000 7f5f57fff640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93edc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.550+0000 7f5f57fff640 1 -- 172.21.15.96:0/423010618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc93dfec90 con 0x55fc93ed0df0 2024-09-10T22:57:15.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.551+0000 7f5f57fff640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009860 tx=0x7f5f4802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9df51070106ccdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.556+0000 7f5f3effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:15.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.556+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48037040 con 0x55fc93ed0df0 2024-09-10T22:57:15.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.556+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f58011db0 con 0x55fc93ed0df0 2024-09-10T22:57:15.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.556+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f480327f0 con 0x55fc93ed0df0 2024-09-10T22:57:15.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.559+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f480329f0 con 0x55fc93ed0df0 2024-09-10T22:57:15.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.559+0000 7f5f6852e740 10 client.17538 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:15.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.559+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc93f007c0 con 0x55fc93ed0df0 2024-09-10T22:57:15.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.560+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58011db0 con 0x55fc93ed0df0 2024-09-10T22:57:15.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.560+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f4804a460 con 0x55fc93ed0df0 2024-09-10T22:57:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 10 client.17538 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 10 client.17538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:57:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 20 client.17538 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f01c80 0x55fc93f04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 4 client.17538 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc93e48df0 con 0x55fc93f01c80 2024-09-10T22:57:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.563+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f01c80 0x55fc93f04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.568+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f01c80 0x55fc93f04070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=457145670943129a server_cookie=ac2f5cce02340f3 in_seq=0 out_seq=0 2024-09-10T22:57:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.568+0000 7f5f3effd640 10 client.17538 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:57:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.568+0000 7f5f3effd640 1 -- 172.21.15.96:0/423010618 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc93e48df0 con 0x55fc93f01c80 2024-09-10T22:57:15.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.568+0000 7f5f3effd640 10 client.17538 handle_command_reply: tid=0 2024-09-10T22:57:15.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.569+0000 7f5f6852e740 1 client.17538 shutdown 2024-09-10T22:57:15.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.569+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 msgr2=0x55fc93ed9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.569+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009860 tx=0x7f5f4802a340 comp rx=0 tx=0).stop 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 shutdown_connections 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f01c80 0x55fc93f04070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ed9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ed9a50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 --2- 172.21.15.96:0/423010618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93edc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 >> 172.21.15.96:0/423010618 conn(0x55fc93ece300 msgr2=0x55fc93ed3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f3effd640 0 client.17538 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 shutdown_connections 2024-09-10T22:57:15.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 1 -- 172.21.15.96:0/423010618 wait complete. 2024-09-10T22:57:15.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.570+0000 7f5f6852e740 20 client.17538 trim_cache size 0 max 16384 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.589+0000 7f5f6852e740 1 Processor -- start 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.589+0000 7f5f6852e740 1 -- start start 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed04d0 0x55fc93edb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93edcd80 0x55fc93ede6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60074eb0 con 0x55fc93ed04d0 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f600769f0 con 0x55fc93edcd80 2024-09-10T22:57:15.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f6852e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60075580 con 0x55fc93ed0df0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed04d0 0x55fc93edb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40340/0 (socket says 172.21.15.96:40340) 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 -- 172.21.15.96:0/3280890109 learned_addr learned my addr 172.21.15.96:0/3280890109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93edcd80 0x55fc93ede6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 -- 172.21.15.96:0/3280890109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93edcd80 msgr2=0x55fc93ede6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93edcd80 0x55fc93ede6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 -- 172.21.15.96:0/3280890109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed04d0 msgr2=0x55fc93edb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed04d0 0x55fc93edb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.590+0000 7f5f57fff640 1 -- 172.21.15.96:0/3280890109 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc93eda8c0 con 0x55fc93ed0df0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.591+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f480057c0 tx=0x7f5f480096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646f7c7a14ee1888 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.591+0000 7f5f557fa640 1 -- 172.21.15.96:0/3280890109 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48037070 con 0x55fc93ed0df0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.591+0000 7f5f557fa640 1 -- 172.21.15.96:0/3280890109 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4804f070 con 0x55fc93ed0df0 2024-09-10T22:57:15.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.591+0000 7f5f557fa640 1 -- 172.21.15.96:0/3280890109 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48032df0 con 0x55fc93ed0df0 2024-09-10T22:57:15.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- 172.21.15.96:0/3280890109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 msgr2=0x55fc93ede170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f480057c0 tx=0x7f5f480096f0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- 172.21.15.96:0/3280890109 shutdown_connections 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93edcd80 0x55fc93ede6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93ed0df0 0x55fc93ede170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3280890109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed04d0 0x55fc93edb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- 172.21.15.96:0/3280890109 >> 172.21.15.96:0/3280890109 conn(0x55fc93ece300 msgr2=0x55fc93edb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- 172.21.15.96:0/3280890109 shutdown_connections 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- 172.21.15.96:0/3280890109 wait complete. 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.592+0000 7f5f6852e740 1 -- start start 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 Processor -- start 2024-09-10T22:57:15.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ee9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ee9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38562/0 (socket says 172.21.15.96:38562) 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ee9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ee9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52954/0 (socket says 172.21.15.96:52954) 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3501744222 learned_addr learned my addr 172.21.15.96:0/3501744222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60076320 con 0x55fc93ed0df0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60075c50 con 0x55fc93ed04d0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60072e90 con 0x55fc93edcd80 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f6852e740 10 client.0 _ll_register_callbacks cb 0x7f5f580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5ffff640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ee9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 msgr2=0x55fc93ee9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ee9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3501744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 msgr2=0x55fc93ee9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ee9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.593+0000 7f5f5f7fe640 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc93edf6b0 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f5f7fe640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f500058c0 tx=0x7f5f500ac080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03a7ac51dbdb206 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f3effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500ac850 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5f500a8ac0 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f4804f070 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.594+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f500b1330 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 10 client.27134 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc93f734e0 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4804f070 con 0x55fc93ed04d0 2024-09-10T22:57:15.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5f500adb80 con 0x55fc93ed04d0 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 10 client.27134 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 10 client.27134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 20 client.27134 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f73b60 0x55fc93f75f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 4 client.27134 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc93e448a0 con 0x55fc93f73b60 2024-09-10T22:57:15.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.595+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f73b60 0x55fc93f75f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:15.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f57fff640 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f73b60 0x55fc93f75f80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=309cbb64c1338f5c server_cookie=aa7096c9e1b7b069 in_seq=0 out_seq=0 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f3effd640 10 client.27134 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f3effd640 1 -- 172.21.15.96:0/3501744222 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc93e448a0 con 0x55fc93f73b60 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f3effd640 10 client.27134 handle_command_reply: tid=0 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 client.27134 shutdown 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 msgr2=0x55fc93f0a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5f500058c0 tx=0x7f5f500ac080 comp rx=0 tx=0).stop 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 shutdown_connections 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55fc93f73b60 0x55fc93f75f80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc93edcd80 0x55fc93ee9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc93ed0df0 0x55fc93ee9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f3effd640 0 client.27134 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 --2- 172.21.15.96:0/3501744222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc93ed04d0 0x55fc93f0a200 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 >> 172.21.15.96:0/3501744222 conn(0x55fc93ece300 msgr2=0x55fc93edb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 shutdown_connections 2024-09-10T22:57:15.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 1 -- 172.21.15.96:0/3501744222 wait complete. 2024-09-10T22:57:15.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.596+0000 7f5f6852e740 20 client.27134 trim_cache size 0 max 16384 2024-09-10T22:57:15.604 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 msgr2=0x7f5f4c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 0x7f5f4c0b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f4802a930 tx=0x7f5f4802c040 comp rx=0 tx=0).stop 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 msgr2=0x7f5f60199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000e9c0 tx=0x7f5f5000ee80 comp rx=0 tx=0).stop 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 shutdown_connections 2024-09-10T22:57:15.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f601087f0 0x7f5f601997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5f4c0b52e0 0x7f5f4c0b77a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f60109170 0x7f5f6019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 --2- 172.21.15.96:0/3867270705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f60102de0 0x7f5f60199290 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 >> 172.21.15.96:0/3867270705 conn(0x7f5f600fe830 msgr2=0x7f5f60100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 shutdown_connections 2024-09-10T22:57:15.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:15.597+0000 7f5f66159640 1 -- 172.21.15.96:0/3867270705 wait complete. 2024-09-10T22:57:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:15 smithi096 ceph-mon[30869]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-10T22:57:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:15 smithi121 ceph-mon[35119]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-10T22:57:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:15 smithi117 ceph-mon[35081]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-10T22:57:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 580.0 s, 2355.4 tps, lat 42.835 ms stddev 49.058 2024-09-10T22:57:16.708 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 -- 172.21.15.96:0/2331126579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 msgr2=0x7f8f54102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/2331126579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54102e30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d940 tx=0x7f8f4802c3a0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 -- 172.21.15.96:0/2331126579 shutdown_connections 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/2331126579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54103370 0x7f8f54109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/2331126579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54102e30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/2331126579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54101860 0x7f8f54101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 -- 172.21.15.96:0/2331126579 >> 172.21.15.96:0/2331126579 conn(0x7f8f540fd140 msgr2=0x7f8f540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 -- 172.21.15.96:0/2331126579 shutdown_connections 2024-09-10T22:57:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.082+0000 7f8f59a4f640 1 -- 172.21.15.96:0/2331126579 wait complete. 2024-09-10T22:57:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 Processor -- start 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 -- start start 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 0x7f8f54196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 0x7f8f54197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c490 con 0x7f8f541029b0 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c310 con 0x7f8f54103370 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f59a4f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c610 con 0x7f8f54101860 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.083+0000 7f8f527fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f527fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50820/0 (socket says 172.21.15.96:50820) 2024-09-10T22:57:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f527fc640 1 -- 172.21.15.96:0/1044485317 learned_addr learned my addr 172.21.15.96:0/1044485317 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 0x7f8f54196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f537fe640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 0x7f8f54197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 -- 172.21.15.96:0/1044485317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 msgr2=0x7f8f54197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 0x7f8f54197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 -- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 msgr2=0x7f8f54197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 -- 172.21.15.96:0/1044485317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f54198750 con 0x7f8f54101860 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f537fe640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 0x7f8f54197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.084+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 0x7f8f54196c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b3f0 tx=0x7f8f4400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.085+0000 7f8f58a4d640 1 -- 172.21.15.96:0/1044485317 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44015020 con 0x7f8f54101860 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.085+0000 7f8f58a4d640 1 -- 172.21.15.96:0/1044485317 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f44011ce0 con 0x7f8f54101860 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.085+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f54198a40 con 0x7f8f54101860 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.085+0000 7f8f58a4d640 1 -- 172.21.15.96:0/1044485317 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44010d80 con 0x7f8f54101860 2024-09-10T22:57:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.085+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5419f110 con 0x7f8f54101860 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f58a4d640 1 -- 172.21.15.96:0/1044485317 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f8f44011830 con 0x7f8f54101860 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 Processor -- start 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 -- start start 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb369a0 0x55b9cbb38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9cba9d6f0 con 0x55b9cbb393e0 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b9cba8cb20 con 0x55b9cbb369a0 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b9cba90930 con 0x55b9cbb34070 2024-09-10T22:57:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44750/0 (socket says 172.21.15.96:44750) 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f537fe640 1 -- 172.21.15.96:0/4124974177 learned_addr learned my addr 172.21.15.96:0/4124974177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb369a0 0x55b9cbb38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.088+0000 7f8f527fc640 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 -- 172.21.15.96:0/4124974177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb369a0 msgr2=0x55b9cbb38d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb369a0 0x55b9cbb38d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 -- 172.21.15.96:0/4124974177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 msgr2=0x55b9cbb3b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 -- 172.21.15.96:0/4124974177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9cba52530 con 0x55b9cbb34070 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f537fe640 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009f60 tx=0x7f8f4c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66746d0631734712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f58a4d640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 0x7f8f3c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f39ffb640 1 -- 172.21.15.96:0/4124974177 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c011640 con 0x55b9cbb34070 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f39ffb640 1 -- 172.21.15.96:0/4124974177 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f4c011c90 con 0x55b9cbb34070 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f39ffb640 1 -- 172.21.15.96:0/4124974177 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c01bcb0 con 0x55b9cbb34070 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f58a4d640 1 -- 172.21.15.96:0/1044485317 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8f440e5560 con 0x7f8f54101860 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.089+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 0x7f8f3c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 -- 172.21.15.96:0/4124974177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 msgr2=0x55b9cbb36460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009f60 tx=0x7f8f4c009f90 comp rx=0 tx=0).stop 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 -- 172.21.15.96:0/4124974177 shutdown_connections 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb369a0 0x55b9cbb38d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/4124974177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb36460 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.091+0000 7f8f5c81d740 1 -- 172.21.15.96:0/4124974177 >> 172.21.15.96:0/4124974177 conn(0x55b9cbb2f1c0 msgr2=0x55b9cbb2ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.092+0000 7f8f5c81d740 1 -- 172.21.15.96:0/4124974177 shutdown_connections 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.092+0000 7f8f52ffd640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 0x7f8f3c0b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f440eb060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.092+0000 7f8f5c81d740 1 -- 172.21.15.96:0/4124974177 wait complete. 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.093+0000 7f8f5c81d740 1 -- start start 2024-09-10T22:57:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.093+0000 7f8f5c81d740 1 Processor -- start 2024-09-10T22:57:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 0x55b9cbb3de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9cb9c4ab0 con 0x55b9cbb393e0 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb3d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49980/0 (socket says 172.21.15.96:49980) 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f537fe640 1 -- 172.21.15.96:0/28850259 learned_addr learned my addr 172.21.15.96:0/28850259 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 0x55b9cbb3de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b9cba32730 con 0x55b9cbb34070 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b9cbb3c3c0 con 0x55b9cbb369a0 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f537fe640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 -- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 msgr2=0x55b9cbb3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 -- 172.21.15.96:0/28850259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 msgr2=0x55b9cbb3e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.094+0000 7f8f527fc640 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9cb7ee790 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f5c81d740 10 client.0 _ll_register_callbacks cb 0x55b9cba866b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f527fc640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 0x55b9cbb3de00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d910 tx=0x7f8f4802cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b9fc7c75fa09b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f32ffd640 1 -- 172.21.15.96:0/28850259 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48043070 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c011c90 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f32ffd640 1 -- 172.21.15.96:0/28850259 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8f48004020 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f32ffd640 1 -- 172.21.15.96:0/28850259 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f4804e050 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f5c81d740 10 client.27205 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.095+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b9cb867620 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f32ffd640 1 -- 172.21.15.96:0/28850259 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8f48004220 con 0x55b9cbb369a0 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 10 client.27205 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 20 client.27205 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbb48d30 0x55b9cbb4b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 4 client.27205 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9cba9f640 con 0x55b9cbb48d30 2024-09-10T22:57:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.096+0000 7f8f537fe640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbb48d30 0x55b9cbb4b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.097+0000 7f8f537fe640 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbb48d30 0x55b9cbb4b1c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02778d05e3f03c6 server_cookie=8cd7d999aac0cee2 in_seq=0 out_seq=0 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.097+0000 7f8f32ffd640 10 client.27205 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.097+0000 7f8f32ffd640 1 -- 172.21.15.96:0/28850259 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b9cba9f640 con 0x55b9cbb48d30 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.097+0000 7f8f32ffd640 10 client.27205 handle_command_reply: tid=0 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 client.27205 shutdown 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 msgr2=0x55b9cbb3de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 0x55b9cbb3de00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d910 tx=0x7f8f4802cdf0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 shutdown_connections 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbb48d30 0x55b9cbb4b1c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb3e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb369a0 0x55b9cbb3de00 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d910 tx=0x7f8f4802cdf0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/28850259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 >> 172.21.15.96:0/28850259 conn(0x55b9cbb2f1c0 msgr2=0x55b9cbb36d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f32ffd640 0 client.27205 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 shutdown_connections 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 1 -- 172.21.15.96:0/28850259 wait complete. 2024-09-10T22:57:17.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.098+0000 7f8f5c81d740 20 client.27205 trim_cache size 0 max 16384 2024-09-10T22:57:17.106 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.119+0000 7f8f5c81d740 1 Processor -- start 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.119+0000 7f8f5c81d740 1 -- start start 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.119+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.119+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb301a0 0x55b9cbb33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a7b80 con 0x55b9cbb301a0 2024-09-10T22:57:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a96c0 con 0x55b9cbb393e0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a8250 con 0x55b9cbb34070 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49988/0 (socket says 172.21.15.96:49988) 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 -- 172.21.15.96:0/1435873381 learned_addr learned my addr 172.21.15.96:0/1435873381 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f537fe640 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 -- 172.21.15.96:0/1435873381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 msgr2=0x55b9cbb2fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb2fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 -- 172.21.15.96:0/1435873381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb301a0 msgr2=0x55b9cbb33590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb301a0 0x55b9cbb33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 -- 172.21.15.96:0/1435873381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9cbb3c790 con 0x55b9cbb393e0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.120+0000 7f8f527fc640 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005ab0 tx=0x7f8f48040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655b668891e52d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.121+0000 7f8f397fa640 1 -- 172.21.15.96:0/1435873381 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48043070 con 0x55b9cbb393e0 2024-09-10T22:57:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.121+0000 7f8f397fa640 1 -- 172.21.15.96:0/1435873381 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f4c011c90 con 0x55b9cbb393e0 2024-09-10T22:57:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.121+0000 7f8f397fa640 1 -- 172.21.15.96:0/1435873381 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48053870 con 0x55b9cbb393e0 2024-09-10T22:57:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- 172.21.15.96:0/1435873381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 msgr2=0x55b9cbb4ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005ab0 tx=0x7f8f48040000 comp rx=0 tx=0).stop 2024-09-10T22:57:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- 172.21.15.96:0/1435873381 shutdown_connections 2024-09-10T22:57:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb393e0 0x55b9cbb4ad40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb301a0 0x55b9cbb33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/1435873381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb34070 0x55b9cbb2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- 172.21.15.96:0/1435873381 >> 172.21.15.96:0/1435873381 conn(0x55b9cbb2f1c0 msgr2=0x55b9cbb3ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- 172.21.15.96:0/1435873381 shutdown_connections 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- 172.21.15.96:0/1435873381 wait complete. 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 -- start start 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 Processor -- start 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb301a0 0x55b9cbb45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.122+0000 7f8f5c81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a8ff0 con 0x55b9cbb393e0 2024-09-10T22:57:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a8920 con 0x55b9cbb34070 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f5c81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541a5b60 con 0x55b9cbb301a0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f5c81d740 10 client.0 _ll_register_callbacks cb 0x7f8f48005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50868/0 (socket says 172.21.15.96:50868) 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f537fe640 1 -- 172.21.15.96:0/3014007553 learned_addr learned my addr 172.21.15.96:0/3014007553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f537fe640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb301a0 0x55b9cbb45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 -- 172.21.15.96:0/3014007553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb301a0 msgr2=0x55b9cbb45250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb301a0 0x55b9cbb45250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 -- 172.21.15.96:0/3014007553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 msgr2=0x55b9cbb43ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 -- 172.21.15.96:0/3014007553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9cbb44820 con 0x55b9cbb34070 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f537fe640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f527fc640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb458f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b9cbb3e2d0 tx=0x7f8f48053450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762de68a263f035b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.123+0000 7f8f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.124+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48043030 con 0x55b9cbb34070 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.124+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f8f4802cdd0 con 0x55b9cbb34070 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.124+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c011c90 con 0x55b9cbb34070 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.125+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f4803f770 con 0x55b9cbb34070 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.125+0000 7f8f5c81d740 10 client.27146 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.125+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9cbb43640 con 0x55b9cbb34070 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f48030040 con 0x55b9cbb34070 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8f4805c880 con 0x55b9cbb34070 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 10 client.27146 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 20 client.27146 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbbb60b0 0x55b9cbbb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 4 client.27146 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.126+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8f4c011c90 con 0x55b9cbbb60b0 2024-09-10T22:57:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.127+0000 7f8f537fe640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbbb60b0 0x55b9cbbb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.127+0000 7f8f32ffd640 10 client.27146 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.127+0000 7f8f537fe640 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbbb60b0 0x55b9cbbb84a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32924f5a7ce6f6d5 server_cookie=2d3f015451e699f in_seq=0 out_seq=0 2024-09-10T22:57:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.144+0000 7f8f32ffd640 1 -- 172.21.15.96:0/3014007553 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8f4c011c90 con 0x55b9cbbb60b0 2024-09-10T22:57:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.144+0000 7f8f32ffd640 10 client.27146 handle_command_reply: tid=0 2024-09-10T22:57:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.144+0000 7f8f5c81d740 1 client.27146 shutdown 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 msgr2=0x55b9cbb458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb458f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b9cbb3e2d0 tx=0x7f8f48053450 comp rx=0 tx=0).stop 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 shutdown_connections 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b9cbbb60b0 0x55b9cbbb84a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9cbb34070 0x55b9cbb458f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9cbb393e0 0x55b9cbb43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 --2- 172.21.15.96:0/3014007553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b9cbb301a0 0x55b9cbb45250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 >> 172.21.15.96:0/3014007553 conn(0x55b9cbb2f1c0 msgr2=0x55b9cbb33160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f32ffd640 0 client.27146 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 shutdown_connections 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 1 -- 172.21.15.96:0/3014007553 wait complete. 2024-09-10T22:57:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.145+0000 7f8f5c81d740 20 client.27146 trim_cache size 0 max 16384 2024-09-10T22:57:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:57:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:57:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:57:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 msgr2=0x7f8f3c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 0x7f8f3c0b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f440eb060 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 msgr2=0x7f8f54196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 0x7f8f54196c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b3f0 tx=0x7f8f4400b8b0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 shutdown_connections 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f54103370 0x7f8f54197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f8f3c0b5360 0x7f8f3c0b7820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f541029b0 0x7f8f54197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 --2- 172.21.15.96:0/1044485317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f54101860 0x7f8f54196c10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 >> 172.21.15.96:0/1044485317 conn(0x7f8f540fd140 msgr2=0x7f8f54105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 shutdown_connections 2024-09-10T22:57:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:17.146+0000 7f8f59a4f640 1 -- 172.21.15.96:0/1044485317 wait complete. 2024-09-10T22:57:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:17 smithi096 ceph-mon[30869]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 721 op/s 2024-09-10T22:57:17.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.651+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/1934505608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410b340 msgr2=0x7fc4e4111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.651+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/1934505608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410b340 0x7fc4e4111b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009ca0 tx=0x7fc4dc02a8f0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/1934505608 shutdown_connections 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/1934505608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410a9c0 0x7fc4e410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/1934505608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410b340 0x7fc4e4111b40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/1934505608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 0x7fc4e4105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/1934505608 >> 172.21.15.96:0/1934505608 conn(0x7fc4e4100a20 msgr2=0x7fc4e4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/1934505608 shutdown_connections 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.652+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/1934505608 wait complete. 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.653+0000 7fc4ebbb1640 1 Processor -- start 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.653+0000 7fc4ebbb1640 1 -- start start 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.653+0000 7fc4ebbb1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 0x7fc4e406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.653+0000 7fc4ebbb1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410a9c0 0x7fc4e406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.653+0000 7fc4ebbb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ebbb1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4079dd0 con 0x7fc4e4104fb0 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ebbb1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4079c50 con 0x7fc4e410b340 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ebbb1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4079f50 con 0x7fc4e410a9c0 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ea14e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ea14e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50028/0 (socket says 172.21.15.96:50028) 2024-09-10T22:57:17.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4ea14e640 1 -- 172.21.15.96:0/2000813514 learned_addr learned my addr 172.21.15.96:0/2000813514 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4e914c640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410a9c0 0x7fc4e406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.654+0000 7fc4e994d640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 0x7fc4e406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 -- 172.21.15.96:0/2000813514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410a9c0 msgr2=0x7fc4e406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410a9c0 0x7fc4e406d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 -- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 msgr2=0x7fc4e406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 0x7fc4e406cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 -- 172.21.15.96:0/2000813514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e4070170 con 0x7fc4e410b340 2024-09-10T22:57:17.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.658+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0308a0 tx=0x7fc4dc0308d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.659+0000 7fc4daffd640 1 -- 172.21.15.96:0/2000813514 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc045430 con 0x7fc4e410b340 2024-09-10T22:57:17.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.659+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e4070400 con 0x7fc4e410b340 2024-09-10T22:57:17.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.659+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e4070880 con 0x7fc4e410b340 2024-09-10T22:57:17.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.659+0000 7fc4daffd640 1 -- 172.21.15.96:0/2000813514 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4dc02d070 con 0x7fc4e410b340 2024-09-10T22:57:17.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.659+0000 7fc4daffd640 1 -- 172.21.15.96:0/2000813514 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc03b640 con 0x7fc4e410b340 2024-09-10T22:57:17.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.670+0000 7fc4edf86740 1 Processor -- start 2024-09-10T22:57:17.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.671+0000 7fc4edf86740 1 -- start start 2024-09-10T22:57:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d89df0 0x5575e9d95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5575e9cffea0 con 0x5575e9d89df0 2024-09-10T22:57:17.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575e9cef2d0 con 0x5575e9d894d0 2024-09-10T22:57:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4edf86740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5575e9cf30e0 con 0x5575e9d95d80 2024-09-10T22:57:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4daffd640 1 -- 172.21.15.96:0/2000813514 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc4dc04d030 con 0x7fc4e410b340 2024-09-10T22:57:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4e914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4e914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50040/0 (socket says 172.21.15.96:50040) 2024-09-10T22:57:17.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4e914c640 1 -- 172.21.15.96:0/2695431313 learned_addr learned my addr 172.21.15.96:0/2695431313 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4e914c640 1 -- 172.21.15.96:0/2695431313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 msgr2=0x5575e9d98170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.672+0000 7fc4e914c640 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.673+0000 7fc4e914c640 1 -- 172.21.15.96:0/2695431313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d89df0 msgr2=0x5575e9d95840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.673+0000 7fc4e914c640 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d89df0 0x5575e9d95840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.673+0000 7fc4e914c640 1 -- 172.21.15.96:0/2695431313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e9a4d650 con 0x5575e9d894d0 2024-09-10T22:57:17.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.673+0000 7fc4e914c640 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d20 tx=0x7fc4d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9803512b2381208c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.674+0000 7fc4daffd640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 0x7fc4d00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.674+0000 7fc4daffd640 1 -- 172.21.15.96:0/2000813514 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc4dc106430 con 0x7fc4e410b340 2024-09-10T22:57:17.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.674+0000 7fc4cffff640 1 -- 172.21.15.96:0/2695431313 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004220 con 0x5575e9d894d0 2024-09-10T22:57:17.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.675+0000 7fc4cffff640 1 -- 172.21.15.96:0/2695431313 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d4009e10 con 0x5575e9d894d0 2024-09-10T22:57:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.676+0000 7fc4edf86740 1 -- 172.21.15.96:0/2695431313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 msgr2=0x5575e9d898b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.676+0000 7fc4edf86740 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d20 tx=0x7fc4d4011810 comp rx=0 tx=0).stop 2024-09-10T22:57:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4edf86740 1 -- 172.21.15.96:0/2695431313 shutdown_connections 2024-09-10T22:57:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4edf86740 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4edf86740 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d89df0 0x5575e9d95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4edf86740 1 --2- 172.21.15.96:0/2695431313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d898b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4edf86740 1 -- 172.21.15.96:0/2695431313 >> 172.21.15.96:0/2695431313 conn(0x5575e9d87300 msgr2=0x5575e9d884a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.679+0000 7fc4e994d640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 0x7fc4d00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.681+0000 7fc4edf86740 1 -- 172.21.15.96:0/2695431313 shutdown_connections 2024-09-10T22:57:17.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.681+0000 7fc4edf86740 1 -- 172.21.15.96:0/2695431313 wait complete. 2024-09-10T22:57:17.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.682+0000 7fc4edf86740 1 -- start start 2024-09-10T22:57:17.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.682+0000 7fc4edf86740 1 Processor -- start 2024-09-10T22:57:17.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.683+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.683+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d92a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4e994d640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 0x7fc4d00b7880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5575e9d881e0 tx=0x7fc4e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4e994d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4e994d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50886/0 (socket says 172.21.15.96:50886) 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4e994d640 1 -- 172.21.15.96:0/3488074589 learned_addr learned my addr 172.21.15.96:0/3488074589 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d92f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5575e9b50700 con 0x5575e9d894d0 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575e9c8a160 con 0x5575e9d89df0 2024-09-10T22:57:17.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4e914c640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d92a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5575e9d98a50 con 0x5575e9d95d80 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4edf86740 10 client.0 _ll_register_callbacks cb 0x5575e9ca39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.684+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d92f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.685+0000 7fc4ea14e640 1 -- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 msgr2=0x5575e9d92a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.685+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d92a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.685+0000 7fc4ea14e640 1 -- 172.21.15.96:0/3488074589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 msgr2=0x5575e9d955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.685+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.685+0000 7fc4ea14e640 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e9cb7c90 con 0x5575e9d95d80 2024-09-10T22:57:17.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.688+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d92f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc004af0 tx=0x7fc4dc0099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1dbc311de5c636d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.688+0000 7fc4cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:17.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.688+0000 7fc4e994d640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:17.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.689+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0b40b0 con 0x5575e9d95d80 2024-09-10T22:57:17.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.689+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc4dc0d11d0 con 0x5575e9d95d80 2024-09-10T22:57:17.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.690+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d4009e10 con 0x5575e9d95d80 2024-09-10T22:57:17.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.691+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4dc0cc700 con 0x5575e9d95d80 2024-09-10T22:57:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.693+0000 7fc4edf86740 10 client.27217 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.693+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5575e9da4380 con 0x5575e9d95d80 2024-09-10T22:57:17.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d4009e10 con 0x5575e9d95d80 2024-09-10T22:57:17.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc4dc0c8050 con 0x5575e9d95d80 2024-09-10T22:57:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4edf86740 10 client.27217 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4edf86740 10 client.27217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:57:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4edf86740 20 client.27217 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.694+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9da7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.695+0000 7fc4edf86740 4 client.27217 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.695+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e9d01df0 con 0x5575e9da5770 2024-09-10T22:57:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.697+0000 7fc4e994d640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9da7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.703+0000 7fc4e994d640 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9da7bb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69393c47346575b3 server_cookie=8054b6a3f1f4c4f0 in_seq=0 out_seq=0 2024-09-10T22:57:17.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.703+0000 7fc4cd7fa640 10 client.27217 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:57:17.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.707+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/3488074589 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575e9d01df0 con 0x5575e9da5770 2024-09-10T22:57:17.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.707+0000 7fc4cd7fa640 10 client.27217 handle_command_reply: tid=0 2024-09-10T22:57:17.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.707+0000 7fc4edf86740 1 client.27217 shutdown 2024-09-10T22:57:17.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.707+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 msgr2=0x5575e9d92f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.707+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d92f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc004af0 tx=0x7fc4dc0099f0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 shutdown_connections 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9da7bb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d92a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d92f40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4cd7fa640 0 client.27217 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 --2- 172.21.15.96:0/3488074589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 >> 172.21.15.96:0/3488074589 conn(0x5575e9d87300 msgr2=0x5575e9d8cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 shutdown_connections 2024-09-10T22:57:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 1 -- 172.21.15.96:0/3488074589 wait complete. 2024-09-10T22:57:17.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.708+0000 7fc4edf86740 20 client.27217 trim_cache size 0 max 16384 2024-09-10T22:57:17.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.732+0000 7fc4edf86740 1 Processor -- start 2024-09-10T22:57:17.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.732+0000 7fc4edf86740 1 -- start start 2024-09-10T22:57:17.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.732+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.732+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d89df0 0x5575e9d94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 0x5575e9d97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4edf86740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800e480 con 0x5575e9d95d80 2024-09-10T22:57:17.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4edf86740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800ffc0 con 0x5575e9d894d0 2024-09-10T22:57:17.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4edf86740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800eb50 con 0x5575e9d89df0 2024-09-10T22:57:17.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4ea14e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 0x5575e9d97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50066/0 (socket says 172.21.15.96:50066) 2024-09-10T22:57:17.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 -- 172.21.15.96:0/1276441176 learned_addr learned my addr 172.21.15.96:0/1276441176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e914c640 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d89df0 0x5575e9d94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 -- 172.21.15.96:0/1276441176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d89df0 msgr2=0x5575e9d94740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d89df0 0x5575e9d94740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 -- 172.21.15.96:0/1276441176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 msgr2=0x5575e9d97170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 0x5575e9d97170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 -- 172.21.15.96:0/1276441176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e9d87d80 con 0x5575e9d894d0 2024-09-10T22:57:17.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 0x5575e9d97170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.733+0000 7fc4e994d640 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000d8f0 tx=0x7fc4e0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd0d81f72f96679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4cf7fe640 1 -- 172.21.15.96:0/1276441176 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00144f0 con 0x5575e9d894d0 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4cf7fe640 1 -- 172.21.15.96:0/1276441176 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e002d070 con 0x5575e9d894d0 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4cf7fe640 1 -- 172.21.15.96:0/1276441176 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0028a40 con 0x5575e9d894d0 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 -- 172.21.15.96:0/1276441176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 msgr2=0x5575e9d86c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000d8f0 tx=0x7fc4e0013000 comp rx=0 tx=0).stop 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 -- 172.21.15.96:0/1276441176 shutdown_connections 2024-09-10T22:57:17.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d95d80 0x5575e9d97170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d89df0 0x5575e9d94740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 --2- 172.21.15.96:0/1276441176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d894d0 0x5575e9d86c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.734+0000 7fc4edf86740 1 -- 172.21.15.96:0/1276441176 >> 172.21.15.96:0/1276441176 conn(0x5575e9d87300 msgr2=0x5575e9d94030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.735+0000 7fc4edf86740 1 -- 172.21.15.96:0/1276441176 shutdown_connections 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.735+0000 7fc4edf86740 1 -- 172.21.15.96:0/1276441176 wait complete. 2024-09-10T22:57:17.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.735+0000 7fc4edf86740 1 -- start start 2024-09-10T22:57:17.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.735+0000 7fc4edf86740 1 Processor -- start 2024-09-10T22:57:17.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.736+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d9e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.737+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.737+0000 7fc4edf86740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.737+0000 7fc4e914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.737+0000 7fc4e914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50082/0 (socket says 172.21.15.96:50082) 2024-09-10T22:57:17.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e914c640 1 -- 172.21.15.96:0/497395713 learned_addr learned my addr 172.21.15.96:0/497395713 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:17.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d9e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800f8f0 con 0x5575e9d894d0 2024-09-10T22:57:17.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800f220 con 0x5575e9d89df0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c800c460 con 0x5575e9d95d80 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4edf86740 10 client.0 _ll_register_callbacks cb 0x7fc4dc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4ea14e640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 -- 172.21.15.96:0/497395713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 msgr2=0x5575e9d98ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 -- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 msgr2=0x5575e9d9e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e994d640 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e9d99720 con 0x5575e9d894d0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.738+0000 7fc4e914c640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4e994d640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d9e1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0004700 tx=0x7fc4e0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b763e2839fcb8dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0028ad0 con 0x5575e9d894d0 2024-09-10T22:57:17.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc4e0028c70 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e002d070 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.739+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4e0040050 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 10 client.17568 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5575e9e13350 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e002d070 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc4e0013070 con 0x5575e9d894d0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 10 client.17568 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 10 client.17568 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 20 client.17568 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9e15ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 4 client.17568 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.740+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575e9cfd8a0 con 0x5575e9da5770 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4e914c640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9e15ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:17.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4cd7fa640 10 client.17568 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4e914c640 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9e15ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed8236bfc9ab979d server_cookie=c4dfb6ef634a8884 in_seq=0 out_seq=0 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4cd7fa640 1 -- 172.21.15.96:0/497395713 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575e9cfd8a0 con 0x5575e9da5770 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4cd7fa640 10 client.17568 handle_command_reply: tid=0 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.741+0000 7fc4edf86740 1 client.17568 shutdown 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 msgr2=0x5575e9d9e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d9e1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0004700 tx=0x7fc4e0005bf0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 shutdown_connections 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5575e9da5770 0x5575e9e15ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575e9d89df0 0x5575e9d9e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575e9d95d80 0x5575e9d98ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 --2- 172.21.15.96:0/497395713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5575e9d894d0 0x5575e9d9e1a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 >> 172.21.15.96:0/497395713 conn(0x5575e9d87300 msgr2=0x5575e9d94850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4cd7fa640 0 client.17568 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:57:17.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 shutdown_connections 2024-09-10T22:57:17.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 1 -- 172.21.15.96:0/497395713 wait complete. 2024-09-10T22:57:17.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.742+0000 7fc4edf86740 20 client.17568 trim_cache size 0 max 16384 2024-09-10T22:57:17.758 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 msgr2=0x7fc4d00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 0x7fc4d00b7880 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5575e9d881e0 tx=0x7fc4e0008040 comp rx=0 tx=0).stop 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 msgr2=0x7fc4e406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0308a0 tx=0x7fc4dc0308d0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 shutdown_connections 2024-09-10T22:57:17.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e410b340 0x7fc4e406fc30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc4d00b53c0 0x7fc4d00b7880 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4e410a9c0 0x7fc4e406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 --2- 172.21.15.96:0/2000813514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e4104fb0 0x7fc4e406cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 >> 172.21.15.96:0/2000813514 conn(0x7fc4e4100a20 msgr2=0x7fc4e407a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 shutdown_connections 2024-09-10T22:57:17.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:17.743+0000 7fc4ebbb1640 1 -- 172.21.15.96:0/2000813514 wait complete. 2024-09-10T22:57:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:17 smithi121 ceph-mon[35119]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 721 op/s 2024-09-10T22:57:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:17 smithi117 ceph-mon[35081]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 721 op/s 2024-09-10T22:57:18.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:57:18.922 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:19.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 -- 172.21.15.96:0/1230200440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 msgr2=0x7fdb8010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/1230200440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8010e860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800bc00 tx=0x7fdb78009260 comp rx=0 tx=0).stop 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 -- 172.21.15.96:0/1230200440 shutdown_connections 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/1230200440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8010e860 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/1230200440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 0x7fdb80104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/1230200440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 -- 172.21.15.96:0/1230200440 >> 172.21.15.96:0/1230200440 conn(0x7fdb80078140 msgr2=0x7fdb800ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 -- 172.21.15.96:0/1230200440 shutdown_connections 2024-09-10T22:57:19.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.480+0000 7fdb84c7c640 1 -- 172.21.15.96:0/1230200440 wait complete. 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 Processor -- start 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 -- start start 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 0x7fdb80197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110bf0 con 0x7fdb801044c0 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110a70 con 0x7fdb80102420 2024-09-10T22:57:19.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb84c7c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110d70 con 0x7fdb80104e80 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50092/0 (socket says 172.21.15.96:50092) 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 0x7fdb80197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.481+0000 7fdb7e59c640 1 -- 172.21.15.96:0/3213906438 learned_addr learned my addr 172.21.15.96:0/3213906438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 -- 172.21.15.96:0/3213906438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 msgr2=0x7fdb8019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 -- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 msgr2=0x7fdb80197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 0x7fdb80197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 -- 172.21.15.96:0/3213906438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8019d0b0 con 0x7fdb80102420 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b590 tx=0x7fdb7000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb6f7fe640 1 -- 172.21.15.96:0/3213906438 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70008500 con 0x7fdb80102420 2024-09-10T22:57:19.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb6f7fe640 1 -- 172.21.15.96:0/3213906438 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb70008b50 con 0x7fdb80102420 2024-09-10T22:57:19.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8019d3a0 con 0x7fdb80102420 2024-09-10T22:57:19.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb6f7fe640 1 -- 172.21.15.96:0/3213906438 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001eca0 con 0x7fdb80102420 2024-09-10T22:57:19.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.482+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a3a60 con 0x7fdb80102420 2024-09-10T22:57:19.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.485+0000 7fdb87051740 1 Processor -- start 2024-09-10T22:57:19.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 -- start start 2024-09-10T22:57:19.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd41df0 0x55f17cd4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f17ccb7ea0 con 0x55f17cd41df0 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f17cca72d0 con 0x55f17cd414d0 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb87051740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f17ccab0e0 con 0x55f17cd4dd80 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb6f7fe640 1 -- 172.21.15.96:0/3213906438 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdb700086a0 con 0x7fdb80102420 2024-09-10T22:57:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50100/0 (socket says 172.21.15.96:50100) 2024-09-10T22:57:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/2923028817 learned_addr learned my addr 172.21.15.96:0/2923028817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd41df0 0x55f17cd4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.486+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.487+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/2923028817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 msgr2=0x55f17cd50170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.487+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd50170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.487+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/2923028817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd41df0 msgr2=0x55f17cd4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.487+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd41df0 0x55f17cd4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.487+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/2923028817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17ca05650 con 0x55f17cd414d0 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.488+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb7800b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81e286f276b88c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.489+0000 7fdb6f7fe640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 0x7fdb680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.489+0000 7fdb6cff9640 1 -- 172.21.15.96:0/2923028817 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78041070 con 0x55f17cd414d0 2024-09-10T22:57:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.489+0000 7fdb6f7fe640 1 -- 172.21.15.96:0/3213906438 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb700ee010 con 0x7fdb80102420 2024-09-10T22:57:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.489+0000 7fdb6cff9640 1 -- 172.21.15.96:0/2923028817 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7802ab70 con 0x55f17cd414d0 2024-09-10T22:57:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.490+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 0x7fdb680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.490+0000 7fdb6cff9640 1 -- 172.21.15.96:0/2923028817 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78003bd0 con 0x55f17cd414d0 2024-09-10T22:57:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.490+0000 7fdb7e59c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 0x7fdb680b7820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb700b4760 tx=0x7fdb7000d350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.490+0000 7fdb87051740 1 -- 172.21.15.96:0/2923028817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 msgr2=0x55f17cd418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.490+0000 7fdb87051740 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb7800b310 comp rx=0 tx=0).stop 2024-09-10T22:57:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 -- 172.21.15.96:0/2923028817 shutdown_connections 2024-09-10T22:57:19.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd50170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd41df0 0x55f17cd4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 --2- 172.21.15.96:0/2923028817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd418b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 -- 172.21.15.96:0/2923028817 >> 172.21.15.96:0/2923028817 conn(0x55f17cd3f300 msgr2=0x55f17cd404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 -- 172.21.15.96:0/2923028817 shutdown_connections 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 -- 172.21.15.96:0/2923028817 wait complete. 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 -- start start 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.491+0000 7fdb87051740 1 Processor -- start 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd4dd80 0x55f17cd4b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f17cb08700 con 0x55f17cd414d0 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f17cc42160 con 0x55f17cd4dd80 2024-09-10T22:57:19.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f17cd509f0 con 0x55f17cd41df0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb87051740 10 client.0 _ll_register_callbacks cb 0x55f17cc5b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44866/0 (socket says 172.21.15.96:44866) 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/3218215984 learned_addr learned my addr 172.21.15.96:0/3218215984 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd4dd80 0x55f17cd4b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/3218215984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd4dd80 msgr2=0x55f17cd4b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd4dd80 0x55f17cd4b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/3218215984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 msgr2=0x55f17cd4d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd4d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.492+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/3218215984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17cc6fc90 con 0x55f17cd41df0 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:19.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b9c0 tx=0x7fdb7400be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4307cccdde0fa741 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74011640 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb740117e0 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb7802ab70 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb740119e0 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb87051740 10 client.27235 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f17cd5c3d0 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.493+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7802ab70 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb74010be0 con 0x55f17cd41df0 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 10 client.27235 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 10 client.27235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 20 client.27235 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:19.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cd5d890 0x55f17cd5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 4 client.27235 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17ccb9df0 con 0x55f17cd5d890 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cd5d890 0x55f17cd5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb65ffb640 10 client.27235 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.494+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cd5d890 0x55f17cd5fcc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a6d35fd06326ee4 server_cookie=a21c2af946248a73 in_seq=0 out_seq=0 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb65ffb640 1 -- 172.21.15.96:0/3218215984 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f17ccb9df0 con 0x55f17cd5d890 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb65ffb640 10 client.27235 handle_command_reply: tid=0 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 client.27235 shutdown 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 msgr2=0x55f17cd4aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b9c0 tx=0x7fdb7400be80 comp rx=0 tx=0).stop 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 shutdown_connections 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cd5d890 0x55f17cd5fcc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd4dd80 0x55f17cd4b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb65ffb640 0 client.27235 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd41df0 0x55f17cd4aa90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 --2- 172.21.15.96:0/3218215984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 >> 172.21.15.96:0/3218215984 conn(0x55f17cd3f300 msgr2=0x55f17cd44ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.495+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 shutdown_connections 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.496+0000 7fdb87051740 1 -- 172.21.15.96:0/3218215984 wait complete. 2024-09-10T22:57:19.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.496+0000 7fdb87051740 20 client.27235 trim_cache size 0 max 16384 2024-09-10T22:57:19.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 Processor -- start 2024-09-10T22:57:19.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 -- start start 2024-09-10T22:57:19.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ac0d0 con 0x55f17cd414d0 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801adc10 con 0x55f17cd3ef20 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.513+0000 7fdb87051740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ac7a0 con 0x55f17cd4dd80 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50128/0 (socket says 172.21.15.96:50128) 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/3139610627 learned_addr learned my addr 172.21.15.96:0/3139610627 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/3139610627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 msgr2=0x55f17cd3a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd3a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/3139610627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 msgr2=0x55f17cd59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/3139610627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17cd41c20 con 0x55f17cd3ef20 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b580 tx=0x7fdb78032800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc520b6deb946cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb67fff640 1 -- 172.21.15.96:0/3139610627 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78054040 con 0x55f17cd3ef20 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb67fff640 1 -- 172.21.15.96:0/3139610627 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78003e20 con 0x55f17cd3ef20 2024-09-10T22:57:19.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.514+0000 7fdb67fff640 1 -- 172.21.15.96:0/3139610627 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78041040 con 0x55f17cd3ef20 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 -- 172.21.15.96:0/3139610627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 msgr2=0x55f17cd403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b580 tx=0x7fdb78032800 comp rx=0 tx=0).stop 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 -- 172.21.15.96:0/3139610627 shutdown_connections 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd3ef20 0x55f17cd403c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd4dd80 0x55f17cd3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 --2- 172.21.15.96:0/3139610627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd414d0 0x55f17cd59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 -- 172.21.15.96:0/3139610627 >> 172.21.15.96:0/3139610627 conn(0x55f17cd3f300 msgr2=0x55f17cd49310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 -- 172.21.15.96:0/3139610627 shutdown_connections 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.515+0000 7fdb87051740 1 -- 172.21.15.96:0/3139610627 wait complete. 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 -- start start 2024-09-10T22:57:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 Processor -- start 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd3ef20 0x55f17cd55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 0x55f17cd50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ad540 con 0x55f17cd4dd80 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801ace70 con 0x55f17cd414d0 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801aa0b0 con 0x55f17cd3ef20 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb87051740 10 client.0 _ll_register_callbacks cb 0x7fdb74004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 0x55f17cd50ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 0x55f17cd50ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50958/0 (socket says 172.21.15.96:50958) 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50134/0 (socket says 172.21.15.96:50134) 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7ed9d640 1 -- 172.21.15.96:0/2248856590 learned_addr learned my addr 172.21.15.96:0/2248856590 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd3ef20 0x55f17cd55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd3ef20 msgr2=0x55f17cd55fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.516+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd3ef20 0x55f17cd55fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/2248856590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 msgr2=0x55f17cd50ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 0x55f17cd50ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb7dd9b640 1 -- 172.21.15.96:0/2248856590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17cd51510 con 0x55f17cd414d0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb7dd9b640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009000 tx=0x7fdb74011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8523c9f4ccb2d249 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74034060 con 0x55f17cd414d0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb7401b970 con 0x55f17cd414d0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb78003e20 con 0x55f17cd414d0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb740039c0 con 0x55f17cd414d0 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb87051740 10 client.27182 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f17cdcb310 con 0x55f17cd414d0 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.517+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78003e20 con 0x55f17cd414d0 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb74039070 con 0x55f17cd414d0 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 10 client.27182 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 10 client.27182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 20 client.27182 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cdcba00 0x55f17cdcde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 4 client.27182 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f17ccb58a0 con 0x55f17cdcba00 2024-09-10T22:57:19.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.518+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cdcba00 0x55f17cdcde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.527+0000 7fdb7ed9d640 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cdcba00 0x55f17cdcde20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f09c3451fbf0dea5 server_cookie=6c1b2bdea74f40f3 in_seq=0 out_seq=0 2024-09-10T22:57:19.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb65ffb640 10 client.27182 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:57:19.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb65ffb640 1 -- 172.21.15.96:0/2248856590 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f17ccb58a0 con 0x55f17cdcba00 2024-09-10T22:57:19.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb65ffb640 10 client.27182 handle_command_reply: tid=0 2024-09-10T22:57:19.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb87051740 1 client.27182 shutdown 2024-09-10T22:57:19.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 msgr2=0x55f17cd56500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.530+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009000 tx=0x7fdb74011e30 comp rx=0 tx=0).stop 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 shutdown_connections 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55f17cdcba00 0x55f17cdcde20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb65ffb640 0 client.27182 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17cd414d0 0x55f17cd56500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f17cd4dd80 0x55f17cd50ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 --2- 172.21.15.96:0/2248856590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f17cd3ef20 0x55f17cd55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 >> 172.21.15.96:0/2248856590 conn(0x55f17cd3f300 msgr2=0x55f17cd40380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.532+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 shutdown_connections 2024-09-10T22:57:19.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.533+0000 7fdb87051740 1 -- 172.21.15.96:0/2248856590 wait complete. 2024-09-10T22:57:19.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.534+0000 7fdb87051740 20 client.27182 trim_cache size 0 max 16384 2024-09-10T22:57:19.538 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:57:19.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.538+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 msgr2=0x7fdb680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.538+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 0x7fdb680b7820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb700b4760 tx=0x7fdb7000d350 comp rx=0 tx=0).stop 2024-09-10T22:57:19.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.538+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 msgr2=0x7fdb80197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:19.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.538+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b590 tx=0x7fdb7000ba50 comp rx=0 tx=0).stop 2024-09-10T22:57:19.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 shutdown_connections 2024-09-10T22:57:19.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb680b5360 0x7fdb680b7820 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb80104e80 0x7fdb8019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb801044c0 0x7fdb80197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 --2- 172.21.15.96:0/3213906438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb80102420 0x7fdb80197000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:19.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.541+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 >> 172.21.15.96:0/3213906438 conn(0x7fdb80078140 msgr2=0x7fdb80105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:19.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.544+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 shutdown_connections 2024-09-10T22:57:19.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:19.544+0000 7fdb84c7c640 1 -- 172.21.15.96:0/3213906438 wait complete. 2024-09-10T22:57:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:19 smithi096 ceph-mon[30869]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 601 op/s 2024-09-10T22:57:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:19 smithi121 ceph-mon[35119]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 601 op/s 2024-09-10T22:57:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:19 smithi117 ceph-mon[35081]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 601 op/s 2024-09-10T22:57:20.290 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:57:20.681 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 585.0 s, 1742.4 tps, lat 57.391 ms stddev 58.940 2024-09-10T22:57:21.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.282+0000 7fa8588ae640 1 -- 172.21.15.96:0/126100528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854102de0 msgr2=0x7fa8541031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.282+0000 7fa8588ae640 1 --2- 172.21.15.96:0/126100528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854102de0 0x7fa8541031c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa844009900 tx=0x7fa84402a410 comp rx=0 tx=0).stop 2024-09-10T22:57:21.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.283+0000 7fa8588ae640 1 -- 172.21.15.96:0/126100528 shutdown_connections 2024-09-10T22:57:21.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.283+0000 7fa8588ae640 1 --2- 172.21.15.96:0/126100528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854109170 0x7fa85410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.283+0000 7fa8588ae640 1 --2- 172.21.15.96:0/126100528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8541087f0 0x7fa854108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.283+0000 7fa8588ae640 1 --2- 172.21.15.96:0/126100528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854102de0 0x7fa8541031c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.283+0000 7fa8588ae640 1 -- 172.21.15.96:0/126100528 >> 172.21.15.96:0/126100528 conn(0x7fa8540fe830 msgr2=0x7fa854100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.284+0000 7fa8588ae640 1 -- 172.21.15.96:0/126100528 shutdown_connections 2024-09-10T22:57:21.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.284+0000 7fa8588ae640 1 -- 172.21.15.96:0/126100528 wait complete. 2024-09-10T22:57:21.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.285+0000 7fa8588ae640 1 Processor -- start 2024-09-10T22:57:21.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.285+0000 7fa8588ae640 1 -- start start 2024-09-10T22:57:21.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 0x7fa8541a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa854112e80 con 0x7fa854109170 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa854112d00 con 0x7fa8541087f0 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa8588ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa854113000 con 0x7fa854102de0 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa85259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50140/0 (socket says 172.21.15.96:50140) 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa85259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44894/0 (socket says 172.21.15.96:44894) 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 -- 172.21.15.96:0/570231268 learned_addr learned my addr 172.21.15.96:0/570231268 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:21.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa852d9d640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 0x7fa8541a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 -- 172.21.15.96:0/570231268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 msgr2=0x7fa85410eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 -- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 msgr2=0x7fa8541a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 0x7fa8541a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa851d9b640 1 -- 172.21.15.96:0/570231268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8541a5810 con 0x7fa8541087f0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa85259c640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.286+0000 7fa852d9d640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 0x7fa8541a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa851d9b640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d770 tx=0x7fa84800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa8437fe640 1 -- 172.21.15.96:0/570231268 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848017070 con 0x7fa8541087f0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa8437fe640 1 -- 172.21.15.96:0/570231268 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8480054d0 con 0x7fa8541087f0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8541a5b00 con 0x7fa8541087f0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa8437fe640 1 -- 172.21.15.96:0/570231268 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004c30 con 0x7fa8541087f0 2024-09-10T22:57:21.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.287+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541ac1c0 con 0x7fa8541087f0 2024-09-10T22:57:21.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.290+0000 7fa85ac83740 1 Processor -- start 2024-09-10T22:57:21.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 -- start start 2024-09-10T22:57:21.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa8437fe640 1 -- 172.21.15.96:0/570231268 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa848004e10 con 0x7fa8541087f0 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea02460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ddea029a0 0x558ddea04d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558dde96cea0 con 0x558dde9f6a10 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dde95c2d0 con 0x558ddea029a0 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85ac83740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558dde9600e0 con 0x558dde9f60f0 2024-09-10T22:57:21.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44902/0 (socket says 172.21.15.96:44902) 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 -- 172.21.15.96:0/2435928435 learned_addr learned my addr 172.21.15.96:0/2435928435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa85259c640 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea02460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 -- 172.21.15.96:0/2435928435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ddea029a0 msgr2=0x558ddea04d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ddea029a0 0x558ddea04d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 -- 172.21.15.96:0/2435928435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 msgr2=0x558ddea02460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea02460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.291+0000 7fa852d9d640 1 -- 172.21.15.96:0/2435928435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dde6ba650 con 0x558dde9f60f0 2024-09-10T22:57:21.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.292+0000 7fa852d9d640 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011790 tx=0x7fa84c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b59f0682e05b7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.292+0000 7fa840ff9640 1 -- 172.21.15.96:0/2435928435 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c003c30 con 0x558dde9f60f0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.292+0000 7fa840ff9640 1 -- 172.21.15.96:0/2435928435 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84c00fe50 con 0x558dde9f60f0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.292+0000 7fa840ff9640 1 -- 172.21.15.96:0/2435928435 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0193e0 con 0x558dde9f60f0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.292+0000 7fa8437fe640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 0x7fa83c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85259c640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 0x7fa83c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa8437fe640 1 -- 172.21.15.96:0/570231268 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa8480e5500 con 0x7fa8541087f0 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 -- 172.21.15.96:0/2435928435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 msgr2=0x558dde9f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011790 tx=0x7fa84c011c50 comp rx=0 tx=0).stop 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 -- 172.21.15.96:0/2435928435 shutdown_connections 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ddea029a0 0x558ddea04d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea02460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2435928435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558dde9f60f0 0x558dde9f64d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 -- 172.21.15.96:0/2435928435 >> 172.21.15.96:0/2435928435 conn(0x558dde9f3f20 msgr2=0x558dde9f50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 -- 172.21.15.96:0/2435928435 shutdown_connections 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.293+0000 7fa85ac83740 1 -- 172.21.15.96:0/2435928435 wait complete. 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.294+0000 7fa85259c640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 0x7fa83c0b7820 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x558dde9f4e00 tx=0x7fa84400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.294+0000 7fa85ac83740 1 -- start start 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.294+0000 7fa85ac83740 1 Processor -- start 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.294+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea02300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.294+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9ff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85ac83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558dde7bd700 con 0x558dde9f6a10 2024-09-10T22:57:21.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85ac83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dde8f7160 con 0x558dde9f60f0 2024-09-10T22:57:21.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44916/0 (socket says 172.21.15.96:44916) 2024-09-10T22:57:21.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa852d9d640 1 -- 172.21.15.96:0/4269754142 learned_addr learned my addr 172.21.15.96:0/4269754142 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:21.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea02300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558ddea05620 con 0x558ddea029a0 2024-09-10T22:57:21.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85ac83740 10 client.0 _ll_register_callbacks cb 0x558dde9109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:21.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 -- 172.21.15.96:0/4269754142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 msgr2=0x558dde9ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 -- 172.21.15.96:0/4269754142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 msgr2=0x558dde9ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.295+0000 7fa85259c640 1 -- 172.21.15.96:0/4269754142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dde924c90 con 0x558dde9f60f0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa85259c640 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea02300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa84402a930 tx=0x7fa844031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3d8717f1eba7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844035040 con 0x558dde9f60f0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa844005220 con 0x558dde9f60f0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c00fe50 con 0x558dde9f60f0 2024-09-10T22:57:21.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.298+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa844005420 con 0x558dde9f60f0 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.299+0000 7fa85ac83740 10 client.27191 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.299+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ddea11000 con 0x558dde9f60f0 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.299+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa84c00fe50 con 0x558dde9f60f0 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa84404f070 con 0x558dde9f60f0 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 10 client.27191 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 10 client.27191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 20 client.27191 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 4 client.27191 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dde96edf0 con 0x558ddea12460 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa851d9b640 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa839ffb640 10 client.27191 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:21.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.300+0000 7fa851d9b640 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea148e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20fa195f03479796 server_cookie=1bde7a2566d0dd94 in_seq=0 out_seq=0 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa839ffb640 1 -- 172.21.15.96:0/4269754142 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558dde96edf0 con 0x558ddea12460 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa839ffb640 10 client.27191 handle_command_reply: tid=0 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 client.27191 shutdown 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 msgr2=0x558ddea02300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea02300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa84402a930 tx=0x7fa844031400 comp rx=0 tx=0).stop 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 shutdown_connections 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea148e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa839ffb640 0 client.27191 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4269754142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea02300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 >> 172.21.15.96:0/4269754142 conn(0x558dde9f3f20 msgr2=0x558dde9f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 shutdown_connections 2024-09-10T22:57:21.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.301+0000 7fa85ac83740 1 -- 172.21.15.96:0/4269754142 wait complete. 2024-09-10T22:57:21.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.302+0000 7fa85ac83740 20 client.27191 trim_cache size 0 max 16384 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 Processor -- start 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 -- start start 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b49b0 con 0x558dde9f6a10 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b64f0 con 0x558dde9f60f0 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85ac83740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b5080 con 0x558ddea029a0 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.320+0000 7fa85259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50164/0 (socket says 172.21.15.96:50164) 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa85259c640 1 -- 172.21.15.96:0/4016332161 learned_addr learned my addr 172.21.15.96:0/4016332161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa852d9d640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 -- 172.21.15.96:0/4016332161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 msgr2=0x558dde9f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 -- 172.21.15.96:0/4016332161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 msgr2=0x558dde9fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 -- 172.21.15.96:0/4016332161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ddea01f30 con 0x558dde9f6a10 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa85259c640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa852d9d640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa851d9b640 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9eee70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e30f0 tx=0x7fa8480a7cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4c984ec015870d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.321+0000 7fa83bfff640 1 -- 172.21.15.96:0/4016332161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848020040 con 0x558dde9f6a10 2024-09-10T22:57:21.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.322+0000 7fa83bfff640 1 -- 172.21.15.96:0/4016332161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa848005680 con 0x558dde9f6a10 2024-09-10T22:57:21.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.322+0000 7fa83bfff640 1 -- 172.21.15.96:0/4016332161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480a8080 con 0x558dde9f6a10 2024-09-10T22:57:21.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.323+0000 7fa85ac83740 1 -- 172.21.15.96:0/4016332161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 msgr2=0x558dde9eee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.323+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9eee70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e30f0 tx=0x7fa8480a7cb0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 -- 172.21.15.96:0/4016332161 shutdown_connections 2024-09-10T22:57:21.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558dde9f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558dde9eee70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 --2- 172.21.15.96:0/4016332161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558dde9fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 -- 172.21.15.96:0/4016332161 >> 172.21.15.96:0/4016332161 conn(0x558dde9f3f20 msgr2=0x558ddea015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.324+0000 7fa85ac83740 1 -- 172.21.15.96:0/4016332161 shutdown_connections 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.325+0000 7fa85ac83740 1 -- 172.21.15.96:0/4016332161 wait complete. 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.325+0000 7fa85ac83740 1 -- start start 2024-09-10T22:57:21.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.325+0000 7fa85ac83740 1 Processor -- start 2024-09-10T22:57:21.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.325+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.325+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea0b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85ac83740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558ddea058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85ac83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b5e20 con 0x558dde9f6a10 2024-09-10T22:57:21.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85ac83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b5750 con 0x558dde9f60f0 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea0b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea0b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51004/0 (socket says 172.21.15.96:51004) 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa851d9b640 1 -- 172.21.15.96:0/2621216904 learned_addr learned my addr 172.21.15.96:0/2621216904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85ac83740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8541b2990 con 0x558ddea029a0 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85ac83740 10 client.0 _ll_register_callbacks cb 0x7fa84400b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa852d9d640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558ddea058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 -- 172.21.15.96:0/2621216904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 msgr2=0x558ddea058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558ddea058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 -- 172.21.15.96:0/2621216904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 msgr2=0x558ddea0b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea0b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 -- 172.21.15.96:0/2621216904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ddea060d0 con 0x558dde9f60f0 2024-09-10T22:57:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa85259c640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea0aad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa844032900 tx=0x7fa844032860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60484c6cd42ac5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.326+0000 7fa839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844004fd0 con 0x558dde9f60f0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa844005170 con 0x558dde9f60f0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa848005680 con 0x558dde9f60f0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa84406a050 con 0x558dde9f60f0 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 10 client.27197 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ddea7fe30 con 0x558dde9f60f0 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa848005680 con 0x558dde9f60f0 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa84404f040 con 0x558dde9f60f0 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 10 client.27197 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 10 client.27197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 20 client.27197 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea82670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 4 client.27197 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:57:21.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.327+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dde96a8a0 con 0x558ddea12460 2024-09-10T22:57:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa851d9b640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea82670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa851d9b640 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea82670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c623633061ff979a server_cookie=bfdfeb818c406b96 in_seq=0 out_seq=0 2024-09-10T22:57:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa839ffb640 10 client.27197 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa839ffb640 1 -- 172.21.15.96:0/2621216904 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dde96a8a0 con 0x558ddea12460 2024-09-10T22:57:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa839ffb640 10 client.27197 handle_command_reply: tid=0 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 client.27197 shutdown 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 msgr2=0x558ddea0aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea0aad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa844032900 tx=0x7fa844032860 comp rx=0 tx=0).stop 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 shutdown_connections 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ddea12460 0x558ddea82670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ddea029a0 0x558ddea058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa839ffb640 0 client.27197 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dde9f6a10 0x558ddea0b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 --2- 172.21.15.96:0/2621216904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dde9f60f0 0x558ddea0aad0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.328+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 >> 172.21.15.96:0/2621216904 conn(0x558dde9f3f20 msgr2=0x558dde9f9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 shutdown_connections 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa85ac83740 1 -- 172.21.15.96:0/2621216904 wait complete. 2024-09-10T22:57:21.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa85ac83740 20 client.27197 trim_cache size 0 max 16384 2024-09-10T22:57:21.339 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:57:21.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 msgr2=0x7fa83c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 0x7fa83c0b7820 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x558dde9f4e00 tx=0x7fa84400b040 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 msgr2=0x7fa85410f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d770 tx=0x7fa84800dc30 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 shutdown_connections 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8541087f0 0x7fa85410f050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa83c0b5360 0x7fa83c0b7820 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa854109170 0x7fa8541a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 --2- 172.21.15.96:0/570231268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa854102de0 0x7fa85410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.329+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 >> 172.21.15.96:0/570231268 conn(0x7fa8540fe830 msgr2=0x7fa854101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:21.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.330+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 shutdown_connections 2024-09-10T22:57:21.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:21.330+0000 7fa8588ae640 1 -- 172.21.15.96:0/570231268 wait complete. 2024-09-10T22:57:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:21 smithi096 ceph-mon[30869]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 717 op/s 2024-09-10T22:57:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:21 smithi121 ceph-mon[35119]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 717 op/s 2024-09-10T22:57:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:21 smithi117 ceph-mon[35081]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 MiB/s wr, 717 op/s 2024-09-10T22:57:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:23 smithi096 ceph-mon[30869]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-09-10T22:57:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:23 smithi121 ceph-mon[35119]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-09-10T22:57:23.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:57:23.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:57:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:23 smithi117 ceph-mon[35081]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 659 op/s 2024-09-10T22:57:24.109 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:25.460 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:57:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:57:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "d789f646-bd3a-43aa-8c3e-87c97bf5fa06", 2024-09-10T22:57:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:57:25.460 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 -- 172.21.15.96:0/4236778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c101290 msgr2=0x7fe46c101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 --2- 172.21.15.96:0/4236778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c101290 0x7fe46c101670 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe468009ca0 tx=0x7fe46802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 -- 172.21.15.96:0/4236778942 shutdown_connections 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 --2- 172.21.15.96:0/4236778942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c1055b0 0x7fe46c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 --2- 172.21.15.96:0/4236778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c105f70 0x7fe46c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 --2- 172.21.15.96:0/4236778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c101290 0x7fe46c101670 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 -- 172.21.15.96:0/4236778942 >> 172.21.15.96:0/4236778942 conn(0x7fe46c078140 msgr2=0x7fe46c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 -- 172.21.15.96:0/4236778942 shutdown_connections 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.678+0000 7fe4746f6640 1 -- 172.21.15.96:0/4236778942 wait complete. 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 Processor -- start 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 -- start start 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c101290 0x7fe46c19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c105f70 0x7fe46c1a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c112d80 con 0x7fe46c1055b0 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c112c00 con 0x7fe46c105f70 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe4746f6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c112f00 con 0x7fe46c101290 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe471c91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe471c91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51010/0 (socket says 172.21.15.96:51010) 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe471c91640 1 -- 172.21.15.96:0/3208768814 learned_addr learned my addr 172.21.15.96:0/3208768814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe472c93640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c105f70 0x7fe46c1a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.679+0000 7fe472492640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c101290 0x7fe46c19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 -- 172.21.15.96:0/3208768814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c101290 msgr2=0x7fe46c19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c101290 0x7fe46c19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 -- 172.21.15.96:0/3208768814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c105f70 msgr2=0x7fe46c1a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c105f70 0x7fe46c1a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 -- 172.21.15.96:0/3208768814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c1a1660 con 0x7fe46c1055b0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe471c91640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe4637fe640 1 -- 172.21.15.96:0/3208768814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00cd80 con 0x7fe46c1055b0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a1950 con 0x7fe46c1055b0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a81e0 con 0x7fe46c1055b0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe4637fe640 1 -- 172.21.15.96:0/3208768814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe45c0108e0 con 0x7fe46c1055b0 2024-09-10T22:57:25.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.680+0000 7fe4637fe640 1 -- 172.21.15.96:0/3208768814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c026060 con 0x7fe46c1055b0 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 Processor -- start 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 -- start start 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b0622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b062800 0x55900b06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55900afd8e60 con 0x55900b062800 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55900afc8290 con 0x55900b061ee0 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.683+0000 7fe476acb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55900afcc0a0 con 0x55900b06e790 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.684+0000 7fe4637fe640 1 -- 172.21.15.96:0/3208768814 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe45c0188e0 con 0x7fe46c1055b0 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.684+0000 7fe471c91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.684+0000 7fe471c91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:44952/0 (socket says 172.21.15.96:44952) 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.684+0000 7fe471c91640 1 -- 172.21.15.96:0/450732304 learned_addr learned my addr 172.21.15.96:0/450732304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:25.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.685+0000 7fe472492640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b062800 0x55900b06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b0622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 -- 172.21.15.96:0/450732304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 msgr2=0x55900b070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 -- 172.21.15.96:0/450732304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b062800 msgr2=0x55900b06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b062800 0x55900b06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.686+0000 7fe472c93640 1 -- 172.21.15.96:0/450732304 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900ad26930 con 0x55900b061ee0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe471c91640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe472c93640 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b0622c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe464004530 tx=0x7fe46400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6599f2c925f9731 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe4637fe640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 0x7fe4580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe4637fe640 1 -- 172.21.15.96:0/3208768814 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe45c0e5d20 con 0x7fe46c1055b0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe460ff9640 1 -- 172.21.15.96:0/450732304 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464003ad0 con 0x55900b061ee0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe460ff9640 1 -- 172.21.15.96:0/450732304 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464015070 con 0x55900b061ee0 2024-09-10T22:57:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe460ff9640 1 -- 172.21.15.96:0/450732304 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464011440 con 0x55900b061ee0 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.687+0000 7fe472492640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 0x7fe4580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- 172.21.15.96:0/450732304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 msgr2=0x55900b0622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b0622c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe464004530 tx=0x7fe46400ba40 comp rx=0 tx=0).stop 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe472492640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 0x7fe4580b7820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55900b065060 tx=0x7fe468009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- 172.21.15.96:0/450732304 shutdown_connections 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b062800 0x55900b06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 --2- 172.21.15.96:0/450732304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b0622c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- 172.21.15.96:0/450732304 >> 172.21.15.96:0/450732304 conn(0x55900b05fd10 msgr2=0x55900b060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- 172.21.15.96:0/450732304 shutdown_connections 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- 172.21.15.96:0/450732304 wait complete. 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 -- start start 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.688+0000 7fe476acb740 1 Processor -- start 2024-09-10T22:57:25.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b062800 0x55900b06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55900ae29890 con 0x55900b06e790 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55900af62d40 con 0x55900b061ee0 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55900b079c50 con 0x55900b062800 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe476acb740 10 client.0 _ll_register_callbacks cb 0x55900afc6c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:50204/0 (socket says 172.21.15.96:50204) 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 -- 172.21.15.96:0/2741088973 learned_addr learned my addr 172.21.15.96:0/2741088973 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472c93640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe471c91640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b062800 0x55900b06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 -- 172.21.15.96:0/2741088973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b062800 msgr2=0x55900b06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b062800 0x55900b06bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 -- 172.21.15.96:0/2741088973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 msgr2=0x55900b06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b06c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.689+0000 7fe472492640 1 -- 172.21.15.96:0/2741088973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900b083140 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe472492640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe468009ca0 tx=0x7fe46802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2b132ccd46634a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468003b80 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe464015070 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe468003d20 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe468004040 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 10 client.27209 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55900b085af0 con 0x55900b061ee0 2024-09-10T22:57:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464015070 con 0x55900b061ee0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe468051070 con 0x55900b061ee0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 10 client.27209 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 10 client.27209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 20 client.27209 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0873b0 0x55900b0897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 4 client.27209 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.690+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900afdadb0 con 0x55900b0873b0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.691+0000 7fe471c91640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0873b0 0x55900b0897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.691+0000 7fe455ffb640 10 client.27209 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.691+0000 7fe471c91640 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0873b0 0x55900b0897d0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d6671e6ec02f89 server_cookie=e037e0f2e846d5b9 in_seq=0 out_seq=0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe455ffb640 1 -- 172.21.15.96:0/2741088973 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55900afdadb0 con 0x55900b0873b0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe455ffb640 10 client.27209 handle_command_reply: tid=0 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 client.27209 shutdown 2024-09-10T22:57:25.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 msgr2=0x55900b06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe468009ca0 tx=0x7fe46802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 shutdown_connections 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0873b0 0x55900b0897d0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b062800 0x55900b06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 --2- 172.21.15.96:0/2741088973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b061ee0 0x55900b06b7d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 >> 172.21.15.96:0/2741088973 conn(0x55900b05fd10 msgr2=0x55900b05ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe455ffb640 0 client.27209 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 shutdown_connections 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 1 -- 172.21.15.96:0/2741088973 wait complete. 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.692+0000 7fe476acb740 20 client.27209 trim_cache size 0 max 16384 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 Processor -- start 2024-09-10T22:57:25.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 -- start start 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 0x55900b066ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000e350 con 0x55900b06e790 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000fe90 con 0x55900b062800 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe476acb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000ea20 con 0x55900b061ee0 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe472c93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe471c91640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe472c93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51032/0 (socket says 172.21.15.96:51032) 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe472c93640 1 -- 172.21.15.96:0/2357308031 learned_addr learned my addr 172.21.15.96:0/2357308031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.718+0000 7fe472492640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 0x55900b066ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 -- 172.21.15.96:0/2357308031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 msgr2=0x55900b066ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 0x55900b066ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 -- 172.21.15.96:0/2357308031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 msgr2=0x55900b068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 -- 172.21.15.96:0/2357308031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900b06caa0 con 0x55900b06e790 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe471c91640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472492640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 0x55900b066ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe472c93640 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe46400bf40 tx=0x7fe46401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7d181f4984edc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe457fff640 1 -- 172.21.15.96:0/2357308031 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464005230 con 0x55900b06e790 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe457fff640 1 -- 172.21.15.96:0/2357308031 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464015040 con 0x55900b06e790 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.719+0000 7fe457fff640 1 -- 172.21.15.96:0/2357308031 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464003f40 con 0x55900b06e790 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 -- 172.21.15.96:0/2357308031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 msgr2=0x55900b069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe46400bf40 tx=0x7fe46401ac00 comp rx=0 tx=0).stop 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 -- 172.21.15.96:0/2357308031 shutdown_connections 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b06e790 0x55900b069130 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 --2- 172.21.15.96:0/2357308031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b061ee0 0x55900b066ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 -- 172.21.15.96:0/2357308031 >> 172.21.15.96:0/2357308031 conn(0x55900b05fd10 msgr2=0x55900b06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 -- 172.21.15.96:0/2357308031 shutdown_connections 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.720+0000 7fe476acb740 1 -- 172.21.15.96:0/2357308031 wait complete. 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 -- start start 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 Processor -- start 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b08fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b08b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000f7c0 con 0x55900b061ee0 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000f0f0 con 0x55900b062800 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe45000c330 con 0x55900b06e790 2024-09-10T22:57:25.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51036/0 (socket says 172.21.15.96:51036) 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 -- 172.21.15.96:0/2595849760 learned_addr learned my addr 172.21.15.96:0/2595849760 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe476acb740 10 client.0 _ll_register_callbacks cb 0x7fe468032f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe471c91640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b08fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472c93640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b08b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 -- 172.21.15.96:0/2595849760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 msgr2=0x55900b08b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b08b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 -- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 msgr2=0x55900b08fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b08fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.721+0000 7fe472492640 1 -- 172.21.15.96:0/2595849760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900b08bb20 con 0x55900b061ee0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe472492640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe468009ca0 tx=0x7fe46802fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48f9dab2f351c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468003d90 con 0x55900b061ee0 2024-09-10T22:57:25.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe464015040 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe46802f660 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe46806d050 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 10 client.17619 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55900b0f3fa0 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe464015040 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe468051050 con 0x55900b061ee0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 10 client.17619 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.722+0000 7fe476acb740 20 client.17619 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.723+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0f4750 0x55900b0f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.723+0000 7fe476acb740 4 client.17619 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.723+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55900afd6860 con 0x55900b0f4750 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.723+0000 7fe471c91640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0f4750 0x55900b0f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.724+0000 7fe455ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.724+0000 7fe471c91640 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0f4750 0x55900b0f6b70 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2293ef7db3a773 server_cookie=d537f99746ea177e in_seq=0 out_seq=0 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe455ffb640 1 -- 172.21.15.96:0/2595849760 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55900afd6860 con 0x55900b0f4750 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe455ffb640 10 client.17619 handle_command_reply: tid=0 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 client.17619 shutdown 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 msgr2=0x55900b08f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe468009ca0 tx=0x7fe46802fee0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 shutdown_connections 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55900b0f4750 0x55900b0f6b70 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55900b062800 0x55900b08fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55900b06e790 0x55900b08b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 --2- 172.21.15.96:0/2595849760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55900b061ee0 0x55900b08f4d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 >> 172.21.15.96:0/2595849760 conn(0x55900b05fd10 msgr2=0x55900b068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.726+0000 7fe455ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.727+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 shutdown_connections 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.727+0000 7fe476acb740 1 -- 172.21.15.96:0/2595849760 wait complete. 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.727+0000 7fe476acb740 20 client.17619 trim_cache size 0 max 16384 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 msgr2=0x7fe4580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 0x7fe4580b7820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55900b065060 tx=0x7fe468009360 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 msgr2=0x7fe46c19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 shutdown_connections 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe46c105f70 0x7fe46c1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe4580b5360 0x7fe4580b7820 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe46c1055b0 0x7fe46c19baf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 --2- 172.21.15.96:0/3208768814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46c101290 0x7fe46c19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:25.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 >> 172.21.15.96:0/3208768814 conn(0x7fe46c078140 msgr2=0x7fe46c108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:25.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 shutdown_connections 2024-09-10T22:57:25.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:24.728+0000 7fe4746f6640 1 -- 172.21.15.96:0/3208768814 wait complete. 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 709 op/s 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:57:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:25 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 709 op/s 2024-09-10T22:57:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:57:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:57:25.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:57:25.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:25 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 709 op/s 2024-09-10T22:57:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:57:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:57:25.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:57:25.777 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:25 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 590.0 s, 1700.8 tps, lat 58.281 ms stddev 62.435 2024-09-10T22:57:26.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:26.687 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:26.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:26 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:26.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:26 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:26.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:26 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 -- 172.21.15.96:0/3095480351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 msgr2=0x7fa66c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 --2- 172.21.15.96:0/3095480351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c10e850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa664009ca0 tx=0x7fa66402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 -- 172.21.15.96:0/3095480351 shutdown_connections 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 --2- 172.21.15.96:0/3095480351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c10e850 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 --2- 172.21.15.96:0/3095480351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c1076d0 0x7fa66c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 --2- 172.21.15.96:0/3095480351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c101cc0 0x7fa66c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 -- 172.21.15.96:0/3095480351 >> 172.21.15.96:0/3095480351 conn(0x7fa66c0fd770 msgr2=0x7fa66c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 -- 172.21.15.96:0/3095480351 shutdown_connections 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.255+0000 7fa674143640 1 -- 172.21.15.96:0/3095480351 wait complete. 2024-09-10T22:57:27.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 Processor -- start 2024-09-10T22:57:27.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 -- start start 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c1076d0 0x7fa66c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c111d10 con 0x7fa66c101cc0 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c111b90 con 0x7fa66c1076d0 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa674143640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c111e90 con 0x7fa66c108050 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa6726e0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42018/0 (socket says 172.21.15.96:42018) 2024-09-10T22:57:27.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 -- 172.21.15.96:0/1881333953 learned_addr learned my addr 172.21.15.96:0/1881333953 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa6716de640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c1076d0 0x7fa66c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 -- 172.21.15.96:0/1881333953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 msgr2=0x7fa66c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 -- 172.21.15.96:0/1881333953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c1076d0 msgr2=0x7fa66c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c1076d0 0x7fa66c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.256+0000 7fa671edf640 1 -- 172.21.15.96:0/1881333953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c1a5400 con 0x7fa66c101cc0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa671edf640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d950 tx=0x7fa66800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa662ffd640 1 -- 172.21.15.96:0/1881333953 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668017070 con 0x7fa66c101cc0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa662ffd640 1 -- 172.21.15.96:0/1881333953 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa668009c30 con 0x7fa66c101cc0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c1a56f0 con 0x7fa66c101cc0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa662ffd640 1 -- 172.21.15.96:0/1881333953 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800f620 con 0x7fa66c101cc0 2024-09-10T22:57:27.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.257+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a5c80 con 0x7fa66c101cc0 2024-09-10T22:57:27.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.261+0000 7fa676518740 1 Processor -- start 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.261+0000 7fa676518740 1 -- start start 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.261+0000 7fa676518740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.261+0000 7fa676518740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa676518740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa676518740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb544e50 con 0x55f2fb5ce480 2024-09-10T22:57:27.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa676518740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb534280 con 0x55f2fb5d87e0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa676518740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb538090 con 0x55f2fb5e33e0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39504/0 (socket says 172.21.15.96:39504) 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 -- 172.21.15.96:0/2474457791 learned_addr learned my addr 172.21.15.96:0/2474457791 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6726e0640 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 -- 172.21.15.96:0/2474457791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 msgr2=0x55f2fb5ceda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ceda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 -- 172.21.15.96:0/2474457791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 msgr2=0x55f2fb5ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 -- 172.21.15.96:0/2474457791 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fb292650 con 0x55f2fb5e33e0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa662ffd640 1 -- 172.21.15.96:0/1881333953 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa668010460 con 0x7fa66c101cc0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.262+0000 7fa6716de640 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009c90 tx=0x7fa65c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=816b6427dd4d7a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa657fff640 1 -- 172.21.15.96:0/2474457791 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c01e070 con 0x55f2fb5e33e0 2024-09-10T22:57:27.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa657fff640 1 -- 172.21.15.96:0/2474457791 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa65c019ae0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa657fff640 1 -- 172.21.15.96:0/2474457791 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00ea40 con 0x55f2fb5e33e0 2024-09-10T22:57:27.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa662ffd640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 0x7fa6580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 0x7fa6580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa676518740 1 -- 172.21.15.96:0/2474457791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 msgr2=0x55f2fb5e57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa676518740 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009c90 tx=0x7fa65c011810 comp rx=0 tx=0).stop 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.263+0000 7fa662ffd640 1 -- 172.21.15.96:0/1881333953 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa6680e56c0 con 0x7fa66c101cc0 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 -- 172.21.15.96:0/2474457791 shutdown_connections 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5e57d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 --2- 172.21.15.96:0/2474457791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 -- 172.21.15.96:0/2474457791 >> 172.21.15.96:0/2474457791 conn(0x55f2fb5cc2b0 msgr2=0x55f2fb5cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 -- 172.21.15.96:0/2474457791 shutdown_connections 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 -- 172.21.15.96:0/2474457791 wait complete. 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 0x7fa6580b7820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa664006010 tx=0x7fa664005f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 -- start start 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.264+0000 7fa676518740 1 Processor -- start 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.265+0000 7fa676518740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 0x55f2fb5d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.265+0000 7fa676518740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa676518740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa676518740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb395700 con 0x55f2fb5d87e0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6716de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa671edf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 0x55f2fb5d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6716de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5d5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42038/0 (socket says 172.21.15.96:42038) 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6716de640 1 -- 172.21.15.96:0/1965708922 learned_addr learned my addr 172.21.15.96:0/1965708922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa676518740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb4cf110 con 0x55f2fb5ce480 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb5e61d0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa676518740 10 client.0 _ll_register_callbacks cb 0x55f2fb4e8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 -- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 msgr2=0x55f2fb5d8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 0x55f2fb5d8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 -- 172.21.15.96:0/1965708922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 msgr2=0x55f2fb5d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa6726e0640 1 -- 172.21.15.96:0/1965708922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fb4fcc40 con 0x55f2fb5e33e0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.266+0000 7fa671edf640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 0x55f2fb5d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6726e0640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5d5c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa66c19b000 tx=0x7fa66402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3455a40ff3b6559b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664003b80 con 0x55f2fb5e33e0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa664003d20 con 0x55f2fb5e33e0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65c019ae0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa664004040 con 0x55f2fb5e33e0 2024-09-10T22:57:27.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 10 client.27289 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2fb5f1b90 con 0x55f2fb5e33e0 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa664036050 con 0x55f2fb5e33e0 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa664051070 con 0x55f2fb5e33e0 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 10 client.27289 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 20 client.27289 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.267+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb5f3040 0x55f2fb5f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.268+0000 7fa676518740 4 client.27289 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.268+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa65c019ae0 con 0x55f2fb5f3040 2024-09-10T22:57:27.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.268+0000 7fa671edf640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb5f3040 0x55f2fb5f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.268+0000 7fa6557fa640 10 client.27289 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.268+0000 7fa671edf640 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb5f3040 0x55f2fb5f5460 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad89f1a1c0883c61 server_cookie=d468aeaa77a72727 in_seq=0 out_seq=0 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa6557fa640 1 -- 172.21.15.96:0/1965708922 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa65c019ae0 con 0x55f2fb5f3040 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa6557fa640 10 client.27289 handle_command_reply: tid=0 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 client.27289 shutdown 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 msgr2=0x55f2fb5d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5d5c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa66c19b000 tx=0x7fa66402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 shutdown_connections 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb5f3040 0x55f2fb5f5460 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5d5c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 --2- 172.21.15.96:0/1965708922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5ce480 0x55f2fb5d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.269+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 >> 172.21.15.96:0/1965708922 conn(0x55f2fb5cc2b0 msgr2=0x55f2fb5cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.270+0000 7fa6557fa640 0 client.27289 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:27.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.270+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 shutdown_connections 2024-09-10T22:57:27.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.270+0000 7fa676518740 1 -- 172.21.15.96:0/1965708922 wait complete. 2024-09-10T22:57:27.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.270+0000 7fa676518740 20 client.27289 trim_cache size 0 max 16384 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 Processor -- start 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 -- start start 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ac0c0 con 0x55f2fb5ce480 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1adc00 con 0x55f2fb5d87e0 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa676518740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ac790 con 0x55f2fb5e33e0 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.293+0000 7fa671edf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42054/0 (socket says 172.21.15.96:42054) 2024-09-10T22:57:27.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 -- 172.21.15.96:0/713161114 learned_addr learned my addr 172.21.15.96:0/713161114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:27.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa6716de640 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa6726e0640 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 -- 172.21.15.96:0/713161114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 msgr2=0x55f2fb5cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 -- 172.21.15.96:0/713161114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 msgr2=0x55f2fb5ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 -- 172.21.15.96:0/713161114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fb5d1340 con 0x55f2fb5ce480 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa671edf640 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa6680abf40 tx=0x7fa668093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756c303a4df5a11a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa6577fe640 1 -- 172.21.15.96:0/713161114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668020040 con 0x55f2fb5ce480 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa6577fe640 1 -- 172.21.15.96:0/713161114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6680abfb0 con 0x55f2fb5ce480 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.294+0000 7fa6577fe640 1 -- 172.21.15.96:0/713161114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668010780 con 0x55f2fb5ce480 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 -- 172.21.15.96:0/713161114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 msgr2=0x55f2fb5ef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa6680abf40 tx=0x7fa668093040 comp rx=0 tx=0).stop 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 -- 172.21.15.96:0/713161114 shutdown_connections 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 --2- 172.21.15.96:0/713161114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ef430 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 -- 172.21.15.96:0/713161114 >> 172.21.15.96:0/713161114 conn(0x55f2fb5cc2b0 msgr2=0x55f2fb5d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 -- 172.21.15.96:0/713161114 shutdown_connections 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.295+0000 7fa676518740 1 -- 172.21.15.96:0/713161114 wait complete. 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 -- start start 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 Processor -- start 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ad530 con 0x55f2fb5e33e0 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1ace60 con 0x55f2fb5d87e0 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1aa0a0 con 0x55f2fb5ce480 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa676518740 10 client.0 _ll_register_callbacks cb 0x7fa664009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa671edf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa6726e0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa671edf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39532/0 (socket says 172.21.15.96:39532) 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.296+0000 7fa671edf640 1 -- 172.21.15.96:0/2681349732 learned_addr learned my addr 172.21.15.96:0/2681349732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6716de640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 -- 172.21.15.96:0/2681349732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 msgr2=0x55f2fb5ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 -- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 msgr2=0x55f2fb5faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 -- 172.21.15.96:0/2681349732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fb5e7920 con 0x55f2fb5e33e0 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa671edf640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6716de640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6726e0640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5fb330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa6640305e0 tx=0x7fa66402f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1390796895e77e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6557fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:27.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664032ea0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa664003d90 con 0x55f2fb5e33e0 2024-09-10T22:57:27.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6680abfb0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa6640041e0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa676518740 10 client.17637 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:27.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2fb6601a0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.297+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6680abfb0 con 0x55f2fb5e33e0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa664067020 con 0x55f2fb5e33e0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 10 client.17637 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 20 client.17637 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb6609a0 0x55f2fb662dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 4 client.17637 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2fb546da0 con 0x55f2fb6609a0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa671edf640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb6609a0 0x55f2fb662dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa671edf640 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb6609a0 0x55f2fb662dc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bd95f7628d1219 server_cookie=320dcc458785533b in_seq=0 out_seq=0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.298+0000 7fa6557fa640 10 client.17637 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.299+0000 7fa6557fa640 1 -- 172.21.15.96:0/2681349732 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55f2fb546da0 con 0x55f2fb6609a0 2024-09-10T22:57:27.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.299+0000 7fa6557fa640 10 client.17637 handle_command_reply: tid=0 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.299+0000 7fa676518740 1 client.17637 shutdown 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.299+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 msgr2=0x55f2fb5fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.299+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5fb330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa6640305e0 tx=0x7fa66402f890 comp rx=0 tx=0).stop 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 shutdown_connections 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f2fb6609a0 0x55f2fb662dc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2fb5d87e0 0x55f2fb5faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa6557fa640 0 client.17637 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f2fb5e33e0 0x55f2fb5fb330 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 --2- 172.21.15.96:0/2681349732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2fb5ce480 0x55f2fb5ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 >> 172.21.15.96:0/2681349732 conn(0x55f2fb5cc2b0 msgr2=0x55f2fb5ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.300+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 shutdown_connections 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.301+0000 7fa676518740 1 -- 172.21.15.96:0/2681349732 wait complete. 2024-09-10T22:57:27.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.301+0000 7fa676518740 20 client.17637 trim_cache size 0 max 16384 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "d789f646-bd3a-43aa-8c3e-87c97bf5fa06", 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:57:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:57:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:57:27.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 msgr2=0x7fa6580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 0x7fa6580b7820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa664006010 tx=0x7fa664005f10 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 msgr2=0x7fa66c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d950 tx=0x7fa66800de10 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 shutdown_connections 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c1076d0 0x7fa66c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa6580b5360 0x7fa6580b7820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa66c108050 0x7fa66c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 --2- 172.21.15.96:0/1881333953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa66c101cc0 0x7fa66c19a020 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.302+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 >> 172.21.15.96:0/1881333953 conn(0x7fa66c0fd770 msgr2=0x7fa66c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.303+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 shutdown_connections 2024-09-10T22:57:27.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:27.303+0000 7fa674143640 1 -- 172.21.15.96:0/1881333953 wait complete. 2024-09-10T22:57:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:27 smithi096 ceph-mon[30869]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 767 op/s 2024-09-10T22:57:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:27 smithi121 ceph-mon[35119]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 767 op/s 2024-09-10T22:57:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:27 smithi117 ceph-mon[35081]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.6 MiB/s wr, 767 op/s 2024-09-10T22:57:28.474 INFO:tasks.cephfs.filesystem:scrub status for tag:d789f646-bd3a-43aa-8c3e-87c97bf5fa06 - {'path': '/', 'tag': 'd789f646-bd3a-43aa-8c3e-87c97bf5fa06', 'options': 'recursive,force'} 2024-09-10T22:57:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:29 smithi096 ceph-mon[30869]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T22:57:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:29 smithi121 ceph-mon[35119]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T22:57:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:29 smithi117 ceph-mon[35081]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-09-10T22:57:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 -- 172.21.15.96:0/2041059733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 msgr2=0x7f24d0109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 --2- 172.21.15.96:0/2041059733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d0109cf0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009cd0 tx=0x7f24c802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 -- 172.21.15.96:0/2041059733 shutdown_connections 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 --2- 172.21.15.96:0/2041059733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 0x7f24d0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 --2- 172.21.15.96:0/2041059733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d0109cf0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 --2- 172.21.15.96:0/2041059733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 0x7f24d0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 -- 172.21.15.96:0/2041059733 >> 172.21.15.96:0/2041059733 conn(0x7f24d00fd140 msgr2=0x7f24d00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 -- 172.21.15.96:0/2041059733 shutdown_connections 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 -- 172.21.15.96:0/2041059733 wait complete. 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.257+0000 7f24d610e640 1 Processor -- start 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 -- start start 2024-09-10T22:57:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 0x7f24d006b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 0x7f24d019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24d010d4d0 con 0x7f24d0103370 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24d010d350 con 0x7f24d01029b0 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24d610e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24d010d650 con 0x7f24d0101860 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 0x7f24d019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42088/0 (socket says 172.21.15.96:42088) 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 -- 172.21.15.96:0/3466594944 learned_addr learned my addr 172.21.15.96:0/3466594944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 0x7f24d006b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 -- 172.21.15.96:0/3466594944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 msgr2=0x7f24d006b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 0x7f24d006b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 -- 172.21.15.96:0/3466594944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 msgr2=0x7f24d019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 0x7f24d019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.258+0000 7f24cffff640 1 -- 172.21.15.96:0/3466594944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d01a08e0 con 0x7f24d0103370 2024-09-10T22:57:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.259+0000 7f24cffff640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f24c8033040 tx=0x7f24c8003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.259+0000 7f24cd7fa640 1 -- 172.21.15.96:0/3466594944 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8004240 con 0x7f24d0103370 2024-09-10T22:57:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.259+0000 7f24cd7fa640 1 -- 172.21.15.96:0/3466594944 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c80305a0 con 0x7f24d0103370 2024-09-10T22:57:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.260+0000 7f24cd7fa640 1 -- 172.21.15.96:0/3466594944 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80466e0 con 0x7f24d0103370 2024-09-10T22:57:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.260+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d0197800 con 0x7f24d0103370 2024-09-10T22:57:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.260+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d0197d00 con 0x7f24d0103370 2024-09-10T22:57:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.263+0000 7f24d8edb740 1 Processor -- start 2024-09-10T22:57:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.263+0000 7f24cd7fa640 1 -- 172.21.15.96:0/3466594944 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f24c804e020 con 0x7f24d0103370 2024-09-10T22:57:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 -- start start 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24cd7fa640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 0x7f24b00bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a38860 0x55b063a3ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3dbd0 0x55b063a40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0639aa6f0 con 0x55b063a38860 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b063999b20 con 0x55b063a3dbd0 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24d8edb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b06399d930 con 0x55b063a3b190 2024-09-10T22:57:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.264+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 0x7f24b00bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24cd7fa640 1 -- 172.21.15.96:0/3466594944 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f24c802e030 con 0x7f24d0103370 2024-09-10T22:57:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39568/0 (socket says 172.21.15.96:39568) 2024-09-10T22:57:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 -- 172.21.15.96:0/2688833135 learned_addr learned my addr 172.21.15.96:0/2688833135 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24cffff640 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3dbd0 0x55b063a40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a38860 0x55b063a3ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 -- 172.21.15.96:0/2688833135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3dbd0 msgr2=0x55b063a40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3dbd0 0x55b063a40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 -- 172.21.15.96:0/2688833135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a38860 msgr2=0x55b063a3ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a38860 0x55b063a3ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 -- 172.21.15.96:0/2688833135 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06395f530 con 0x55b063a3b190 2024-09-10T22:57:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24c0005bc0 tx=0x7f24c001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c1056f67939a531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.265+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 0x7f24b00bbd00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24c000afd0 tx=0x7f24c000ab20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.266+0000 7f24c5ffb640 1 -- 172.21.15.96:0/2688833135 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0023070 con 0x55b063a3b190 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.266+0000 7f24c5ffb640 1 -- 172.21.15.96:0/2688833135 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c000fe50 con 0x55b063a3b190 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.266+0000 7f24c5ffb640 1 -- 172.21.15.96:0/2688833135 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c001fb20 con 0x55b063a3b190 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 -- 172.21.15.96:0/2688833135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 msgr2=0x55b063a3d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24c0005bc0 tx=0x7f24c001b000 comp rx=0 tx=0).stop 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 -- 172.21.15.96:0/2688833135 shutdown_connections 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3dbd0 0x55b063a40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a3d580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2688833135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a38860 0x55b063a3ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 -- 172.21.15.96:0/2688833135 >> 172.21.15.96:0/2688833135 conn(0x55b063a339b0 msgr2=0x55b063a34740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 -- 172.21.15.96:0/2688833135 shutdown_connections 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.267+0000 7f24d8edb740 1 -- 172.21.15.96:0/2688833135 wait complete. 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24d8edb740 1 -- start start 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24d8edb740 1 Processor -- start 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42110/0 (socket says 172.21.15.96:42110) 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39578/0 (socket says 172.21.15.96:39578) 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.268+0000 7f24cffff640 1 -- 172.21.15.96:0/2171361572 learned_addr learned my addr 172.21.15.96:0/2171361572 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0638d1ab0 con 0x55b063a3dbd0 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b06393f730 con 0x55b063a38860 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b063a40b50 con 0x55b063a3b190 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 -- 172.21.15.96:0/2171361572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 msgr2=0x55b063a425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 -- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 msgr2=0x55b063a41f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a41f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0636fb790 con 0x55b063a3dbd0 2024-09-10T22:57:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a41f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24cffff640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c807a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a8248e3752f9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24beffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24beffd640 1 -- 172.21.15.96:0/2171361572 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80eedc0 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24beffd640 1 -- 172.21.15.96:0/2171361572 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f24c80cc440 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24d8edb740 10 client.0 _ll_register_callbacks cb 0x55b0639936b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.269+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c000fe50 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24beffd640 1 -- 172.21.15.96:0/2171361572 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24c80d60b0 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 10 client.17649 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b063774620 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24beffd640 1 -- 172.21.15.96:0/2171361572 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f24c8030070 con 0x55b063a3dbd0 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 20 client.17649 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063a4f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 4 client.17649 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24c000fe50 con 0x55b063a4d470 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.270+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063a4f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.271+0000 7f24beffd640 10 client.17649 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.271+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063a4f930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eadec955e5744d6 server_cookie=cf29324276a3f3b5 in_seq=0 out_seq=0 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24beffd640 1 -- 172.21.15.96:0/2171361572 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f24c000fe50 con 0x55b063a4d470 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24beffd640 10 client.17649 handle_command_reply: tid=0 2024-09-10T22:57:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 client.17649 shutdown 2024-09-10T22:57:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 msgr2=0x55b063a43170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c807a4f0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 shutdown_connections 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063a4f930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a43170 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24beffd640 0 client.17649 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.272+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.273+0000 7f24d8edb740 1 --2- 172.21.15.96:0/2171361572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a41f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.273+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 >> 172.21.15.96:0/2171361572 conn(0x55b063a339b0 msgr2=0x55b063a32bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.273+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 shutdown_connections 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.273+0000 7f24d8edb740 1 -- 172.21.15.96:0/2171361572 wait complete. 2024-09-10T22:57:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.273+0000 7f24d8edb740 20 client.17649 trim_cache size 0 max 16384 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.292+0000 7f24d8edb740 1 Processor -- start 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.292+0000 7f24d8edb740 1 -- start start 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.292+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.292+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3b190 0x55b063a34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.292+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a34750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24d8edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400e1f0 con 0x55b063a3dbd0 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24d8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400fd30 con 0x55b063a3b190 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24d8edb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400e8c0 con 0x55b063a38860 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a34750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39582/0 (socket says 172.21.15.96:39582) 2024-09-10T22:57:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 -- 172.21.15.96:0/1887410515 learned_addr learned my addr 172.21.15.96:0/1887410515 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a34750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42116/0 (socket says 172.21.15.96:42116) 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3b190 0x55b063a34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 -- 172.21.15.96:0/1887410515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3b190 msgr2=0x55b063a34210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3b190 0x55b063a34210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 -- 172.21.15.96:0/1887410515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 msgr2=0x55b063a34750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a34750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 -- 172.21.15.96:0/1887410515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b063a3afc0 con 0x55b063a38860 2024-09-10T22:57:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009b40 tx=0x7f24b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9441854a639744 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.293+0000 7f24c57fa640 1 -- 172.21.15.96:0/1887410515 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8020070 con 0x55b063a38860 2024-09-10T22:57:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.294+0000 7f24c57fa640 1 -- 172.21.15.96:0/1887410515 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c000fe50 con 0x55b063a38860 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.294+0000 7f24c57fa640 1 -- 172.21.15.96:0/1887410515 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8010690 con 0x55b063a38860 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.295+0000 7f24d8edb740 1 -- 172.21.15.96:0/1887410515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 msgr2=0x55b063a4f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.295+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009b40 tx=0x7f24b80046c0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 -- 172.21.15.96:0/1887410515 shutdown_connections 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a3b190 0x55b063a34210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a34750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1887410515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a38860 0x55b063a4f890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 -- 172.21.15.96:0/1887410515 >> 172.21.15.96:0/1887410515 conn(0x55b063a339b0 msgr2=0x55b063a3dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 -- 172.21.15.96:0/1887410515 shutdown_connections 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 -- 172.21.15.96:0/1887410515 wait complete. 2024-09-10T22:57:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.296+0000 7f24d8edb740 1 -- start start 2024-09-10T22:57:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 Processor -- start 2024-09-10T22:57:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a49710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400f660 con 0x55b063a3dbd0 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400ef90 con 0x55b063a38860 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24b400c1d0 con 0x55b063a3b190 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24cf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a49710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39584/0 (socket says 172.21.15.96:39584) 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 -- 172.21.15.96:0/1599478840 learned_addr learned my addr 172.21.15.96:0/1599478840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24cffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a48950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42126/0 (socket says 172.21.15.96:42126) 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24d8edb740 10 client.0 _ll_register_callbacks cb 0x7f24c8009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 -- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 msgr2=0x55b063a49710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a49710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 -- 172.21.15.96:0/1599478840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 msgr2=0x55b063a48950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a48950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.297+0000 7f24c6ffd640 1 -- 172.21.15.96:0/1599478840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b063a48f60 con 0x55b063a3b190 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24c6ffd640 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c000a7f0 tx=0x7f24c001ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e85cbd349fb8d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24beffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0040060 con 0x55b063a3b190 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c000fe50 con 0x55b063a3b190 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f24c001f890 con 0x55b063a3b190 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24c0023040 con 0x55b063a3b190 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24d8edb740 10 client.27313 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.298+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b063aba440 con 0x55b063a3b190 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c000fe50 con 0x55b063a3b190 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f24c0045070 con 0x55b063a3b190 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 10 client.27313 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 20 client.27313 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063abcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 4 client.27313 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0639ac640 con 0x55b063a4d470 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.299+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063abcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.300+0000 7f24cf7fe640 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063abcbb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ca2abc344fde94 server_cookie=31ea7e48ec68929a in_seq=0 out_seq=0 2024-09-10T22:57:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.300+0000 7f24beffd640 10 client.27313 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.349+0000 7f24beffd640 1 -- 172.21.15.96:0/1599478840 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0639ac640 con 0x55b063a4d470 2024-09-10T22:57:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.349+0000 7f24beffd640 10 client.27313 handle_command_reply: tid=0 2024-09-10T22:57:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.352+0000 7f24d8edb740 1 client.27313 shutdown 2024-09-10T22:57:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.359+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 msgr2=0x55b063a48410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.359+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c000a7f0 tx=0x7f24c001ec30 comp rx=0 tx=0).stop 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 shutdown_connections 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b063a4d470 0x55b063abcbb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b063a3dbd0 0x55b063a48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b063a3b190 0x55b063a48410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 --2- 172.21.15.96:0/1599478840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b063a38860 0x55b063a49710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 >> 172.21.15.96:0/1599478840 conn(0x55b063a339b0 msgr2=0x55b063a37920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.365+0000 7f24beffd640 0 client.27313 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.373+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 shutdown_connections 2024-09-10T22:57:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.376+0000 7f24d8edb740 1 -- 172.21.15.96:0/1599478840 wait complete. 2024-09-10T22:57:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.377+0000 7f24d8edb740 20 client.27313 trim_cache size 0 max 16384 2024-09-10T22:57:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:57:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:57:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:57:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 msgr2=0x7f24b00bbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 0x7f24b00bbd00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24c000afd0 tx=0x7f24c000ab20 comp rx=0 tx=0).stop 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 msgr2=0x7f24d01a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f24c8033040 tx=0x7f24c8003b80 comp rx=0 tx=0).stop 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 shutdown_connections 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.384+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24d01029b0 0x7f24d019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24b00b9840 0x7f24b00bbd00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d0103370 0x7f24d01a0020 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 --2- 172.21.15.96:0/3466594944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24d0101860 0x7f24d006b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 >> 172.21.15.96:0/3466594944 conn(0x7f24d00fd140 msgr2=0x7f24d0103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 shutdown_connections 2024-09-10T22:57:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:30.385+0000 7f24d610e640 1 -- 172.21.15.96:0/3466594944 wait complete. 2024-09-10T22:57:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 595.0 s, 1961.8 tps, lat 51.065 ms stddev 51.521 2024-09-10T22:57:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:31 smithi096 ceph-mon[30869]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 825 op/s 2024-09-10T22:57:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:31 smithi121 ceph-mon[35119]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 825 op/s 2024-09-10T22:57:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:31 smithi117 ceph-mon[35081]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 825 op/s 2024-09-10T22:57:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:33 smithi096 ceph-mon[30869]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 749 op/s 2024-09-10T22:57:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:33 smithi121 ceph-mon[35119]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 749 op/s 2024-09-10T22:57:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:33 smithi117 ceph-mon[35081]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 749 op/s 2024-09-10T22:57:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:34 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:57:34.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:34 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:57:34.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:34 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:57:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:35 smithi096 ceph-mon[30869]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.8 MiB/s wr, 762 op/s 2024-09-10T22:57:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:35 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:57:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:35 smithi121 ceph-mon[35119]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.8 MiB/s wr, 762 op/s 2024-09-10T22:57:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:35 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:57:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:35 smithi117 ceph-mon[35081]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.8 MiB/s wr, 762 op/s 2024-09-10T22:57:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:35 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:57:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 600.0 s, 2421.8 tps, lat 41.566 ms stddev 49.152 2024-09-10T22:57:37.643 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:37 smithi117 ceph-mon[35081]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.4 MiB/s wr, 741 op/s 2024-09-10T22:57:37.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:37 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:37 smithi096 ceph-mon[30869]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.4 MiB/s wr, 741 op/s 2024-09-10T22:57:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:37 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:37 smithi121 ceph-mon[35119]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.4 MiB/s wr, 741 op/s 2024-09-10T22:57:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:37 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.9 MiB/s wr, 692 op/s 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.9 MiB/s wr, 692 op/s 2024-09-10T22:57:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.9 MiB/s wr, 692 op/s 2024-09-10T22:57:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:57:39.527 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:57:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 605.0 s, 2240.2 tps, lat 44.327 ms stddev 56.709 2024-09-10T22:57:41.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:41 smithi096 ceph-mon[30869]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.2 MiB/s wr, 791 op/s 2024-09-10T22:57:41.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:41 smithi121 ceph-mon[35119]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.2 MiB/s wr, 791 op/s 2024-09-10T22:57:41.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:41 smithi117 ceph-mon[35081]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.2 MiB/s wr, 791 op/s 2024-09-10T22:57:43.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:43 smithi096 ceph-mon[30869]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 627 op/s 2024-09-10T22:57:43.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:43 smithi121 ceph-mon[35119]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 627 op/s 2024-09-10T22:57:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 -- 172.21.15.96:0/2673367134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c101870 msgr2=0x7fbc3c101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/2673367134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c101870 0x7fbc3c101c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801a940 tx=0x7fbc3803b450 comp rx=0 tx=0).stop 2024-09-10T22:57:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 -- 172.21.15.96:0/2673367134 shutdown_connections 2024-09-10T22:57:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/2673367134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/2673367134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c103380 0x7fbc3c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/2673367134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c101870 0x7fbc3c101c70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 -- 172.21.15.96:0/2673367134 >> 172.21.15.96:0/2673367134 conn(0x7fbc3c0fd130 msgr2=0x7fbc3c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.519+0000 7fbc43ebd640 1 -- 172.21.15.96:0/2673367134 shutdown_connections 2024-09-10T22:57:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 -- 172.21.15.96:0/2673367134 wait complete. 2024-09-10T22:57:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 Processor -- start 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 -- start start 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c101870 0x7fbc3c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c070760 con 0x7fbc3c101870 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0705e0 con 0x7fbc3c1029c0 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.520+0000 7fbc43ebd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0708e0 con 0x7fbc3c103380 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59776/0 (socket says 172.21.15.96:59776) 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 -- 172.21.15.96:0/519682530 learned_addr learned my addr 172.21.15.96:0/519682530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc41458640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc41c59640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c101870 0x7fbc3c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 -- 172.21.15.96:0/519682530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 msgr2=0x7fbc3c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 -- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c101870 msgr2=0x7fbc3c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c101870 0x7fbc3c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 -- 172.21.15.96:0/519682530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c06e5a0 con 0x7fbc3c103380 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc4245a640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eb10 tx=0x7fbc3400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc32ffd640 1 -- 172.21.15.96:0/519682530 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400ce20 con 0x7fbc3c103380 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc32ffd640 1 -- 172.21.15.96:0/519682530 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34005020 con 0x7fbc3c103380 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc32ffd640 1 -- 172.21.15.96:0/519682530 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34010690 con 0x7fbc3c103380 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.521+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c06e890 con 0x7fbc3c103380 2024-09-10T22:57:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.522+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c19ef30 con 0x7fbc3c103380 2024-09-10T22:57:43.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:43 smithi117 ceph-mon[35081]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 627 op/s 2024-09-10T22:57:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.525+0000 7fbc46c8c740 1 Processor -- start 2024-09-10T22:57:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.525+0000 7fbc32ffd640 1 -- 172.21.15.96:0/519682530 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbc34005510 con 0x7fbc3c103380 2024-09-10T22:57:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.525+0000 7fbc46c8c740 1 -- start start 2024-09-10T22:57:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.525+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b04fff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b05002090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41458640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b04fff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41458640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39060/0 (socket says 172.21.15.96:39060) 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41458640 1 -- 172.21.15.96:0/112366854 learned_addr learned my addr 172.21.15.96:0/112366854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556b04f6c6f0 con 0x556b04ffa860 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b04f5bb20 con 0x556b04fffbd0 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556b04f5f930 con 0x556b04ffd190 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41458640 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b05002090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 -- 172.21.15.96:0/112366854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 msgr2=0x556b05002090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b05002090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 -- 172.21.15.96:0/112366854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 msgr2=0x556b04ffcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc41c59640 1 -- 172.21.15.96:0/112366854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b04f21530 con 0x556b04ffd190 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.526+0000 7fbc32ffd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 0x7fbc280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc41c59640 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b04fff580 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801a910 tx=0x7fbc38004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149d371668a2c6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc32ffd640 1 -- 172.21.15.96:0/519682530 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbc340e6050 con 0x7fbc3c103380 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc41458640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 0x7fbc280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc27fff640 1 -- 172.21.15.96:0/112366854 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38046030 con 0x556b04ffd190 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc27fff640 1 -- 172.21.15.96:0/112366854 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380053d0 con 0x556b04ffd190 2024-09-10T22:57:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.527+0000 7fbc27fff640 1 -- 172.21.15.96:0/112366854 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38040a70 con 0x556b04ffd190 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 msgr2=0x556b04fff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b04fff580 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801a910 tx=0x7fbc38004740 comp rx=0 tx=0).stop 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc41458640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 0x7fbc280b7820 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00a5c0 tx=0x7fbc2c014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 shutdown_connections 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b05002090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b04fff580 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/112366854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 >> 172.21.15.96:0/112366854 conn(0x556b04ff59b0 msgr2=0x556b04ff6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 shutdown_connections 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- 172.21.15.96:0/112366854 wait complete. 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 -- start start 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.528+0000 7fbc46c8c740 1 Processor -- start 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b050044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05005070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556b04e93ab0 con 0x556b04ffd190 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b04f01730 con 0x556b04ffa860 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556b05002b50 con 0x556b04fffbd0 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc46c8c740 10 client.0 _ll_register_callbacks cb 0x556b04f556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60890/0 (socket says 172.21.15.96:60890) 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 -- 172.21.15.96:0/879398020 learned_addr learned my addr 172.21.15.96:0/879398020 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41458640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b050044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc4245a640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05005070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 -- 172.21.15.96:0/879398020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 msgr2=0x556b05005070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05005070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 -- 172.21.15.96:0/879398020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 msgr2=0x556b050044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b050044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.529+0000 7fbc41c59640 1 -- 172.21.15.96:0/879398020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b04cbd790 con 0x556b04ffa860 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc41c59640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbc380057c0 tx=0x7fbc38005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca1e9f341887c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc257fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38046040 con 0x556b04ffa860 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbc38004f20 con 0x556b04ffa860 2024-09-10T22:57:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc380053d0 con 0x556b04ffa860 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbc38005120 con 0x556b04ffa860 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc46c8c740 10 client.27254 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.530+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b04d36620 con 0x556b04ffa860 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38040850 con 0x556b04ffa860 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbc380035f0 con 0x556b04ffa860 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 10 client.27254 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 10 client.27254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 20 client.27254 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0500f2a0 0x556b05011690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 4 client.27254 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbc380053d0 con 0x556b0500f2a0 2024-09-10T22:57:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.531+0000 7fbc41458640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0500f2a0 0x556b05011690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.532+0000 7fbc257fa640 10 client.27254 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.532+0000 7fbc41458640 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0500f2a0 0x556b05011690 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0bf9f82860c7b4 server_cookie=493c441782450703 in_seq=0 out_seq=0 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc257fa640 1 -- 172.21.15.96:0/879398020 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbc380053d0 con 0x556b0500f2a0 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc257fa640 10 client.27254 handle_command_reply: tid=0 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc46c8c740 1 client.27254 shutdown 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 msgr2=0x556b05003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbc380057c0 tx=0x7fbc38005610 comp rx=0 tx=0).stop 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 shutdown_connections 2024-09-10T22:57:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.533+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0500f2a0 0x556b05011690 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05005070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b050044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/879398020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05003e00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 >> 172.21.15.96:0/879398020 conn(0x556b04ff59b0 msgr2=0x556b04ff4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc257fa640 0 client.27254 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 shutdown_connections 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 1 -- 172.21.15.96:0/879398020 wait complete. 2024-09-10T22:57:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.534+0000 7fbc46c8c740 20 client.27254 trim_cache size 0 max 16384 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 Processor -- start 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 -- start start 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b050004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b04ff43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a7c70 con 0x556b04ffa860 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a97b0 con 0x556b04fffbd0 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a8340 con 0x556b04ffd190 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc41c59640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc41c59640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39082/0 (socket says 172.21.15.96:39082) 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc41c59640 1 -- 172.21.15.96:0/3187228563 learned_addr learned my addr 172.21.15.96:0/3187228563 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.552+0000 7fbc41458640 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b050004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.553+0000 7fbc41c59640 1 -- 172.21.15.96:0/3187228563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 msgr2=0x556b050004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.553+0000 7fbc41c59640 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b050004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.553+0000 7fbc41c59640 1 -- 172.21.15.96:0/3187228563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 msgr2=0x556b04ff43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.553+0000 7fbc41c59640 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b04ff43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.553+0000 7fbc41c59640 1 -- 172.21.15.96:0/3187228563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b04ff6c10 con 0x556b04ffa860 2024-09-10T22:57:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.554+0000 7fbc41c59640 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801aa60 tx=0x7fbc38004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f5c10ef8d4d697 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.554+0000 7fbc277fe640 1 -- 172.21.15.96:0/3187228563 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38046070 con 0x556b04ffa860 2024-09-10T22:57:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.554+0000 7fbc277fe640 1 -- 172.21.15.96:0/3187228563 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380044b0 con 0x556b04ffa860 2024-09-10T22:57:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.554+0000 7fbc277fe640 1 -- 172.21.15.96:0/3187228563 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38040580 con 0x556b04ffa860 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- 172.21.15.96:0/3187228563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 msgr2=0x556b04ffffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801aa60 tx=0x7fbc38004e20 comp rx=0 tx=0).stop 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- 172.21.15.96:0/3187228563 shutdown_connections 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04fffbd0 0x556b04ff43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04ffd190 0x556b050004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/3187228563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffa860 0x556b04ffffb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- 172.21.15.96:0/3187228563 >> 172.21.15.96:0/3187228563 conn(0x556b04ff59b0 msgr2=0x556b04ff9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- 172.21.15.96:0/3187228563 shutdown_connections 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- 172.21.15.96:0/3187228563 wait complete. 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.556+0000 7fbc46c8c740 1 -- start start 2024-09-10T22:57:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 Processor -- start 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05008cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b05009350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05009e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a90e0 con 0x556b04ffd190 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a8a10 con 0x556b04ffa860 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c1a5ca0 con 0x556b04fffbd0 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc41458640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b05009350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc41458640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b05009350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39086/0 (socket says 172.21.15.96:39086) 2024-09-10T22:57:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc41458640 1 -- 172.21.15.96:0/1500606897 learned_addr learned my addr 172.21.15.96:0/1500606897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc46c8c740 10 client.0 _ll_register_callbacks cb 0x7fbc2c00ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc41c59640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05008cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.557+0000 7fbc4245a640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05009e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 -- 172.21.15.96:0/1500606897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 msgr2=0x556b05009e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05009e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 -- 172.21.15.96:0/1500606897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 msgr2=0x556b05009350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b05009350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 -- 172.21.15.96:0/1500606897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b04ffcfc0 con 0x556b04ffa860 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc41c59640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05008cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc38050040 tx=0x7fbc38000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206e6034db23513d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc257fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3806f020 con 0x556b04ffa860 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbc38004000 con 0x556b04ffa860 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc380044b0 con 0x556b04ffa860 2024-09-10T22:57:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbc38046040 con 0x556b04ffa860 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc46c8c740 10 client.27257 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.558+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b0500b8a0 con 0x556b04ffa860 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380044b0 con 0x556b04ffa860 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbc38074070 con 0x556b04ffa860 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 10 client.27257 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 20 client.27257 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0507c0e0 0x556b0507e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 4 client.27257 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.559+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b04f6e640 con 0x556b0507c0e0 2024-09-10T22:57:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.560+0000 7fbc41458640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0507c0e0 0x556b0507e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.560+0000 7fbc257fa640 10 client.27257 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.560+0000 7fbc41458640 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0507c0e0 0x556b0507e4d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4313ab76746e7f28 server_cookie=c767a5b92dc6ef81 in_seq=0 out_seq=0 2024-09-10T22:57:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.583+0000 7fbc257fa640 1 -- 172.21.15.96:0/1500606897 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b04f6e640 con 0x556b0507c0e0 2024-09-10T22:57:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.583+0000 7fbc257fa640 10 client.27257 handle_command_reply: tid=0 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.583+0000 7fbc46c8c740 1 client.27257 shutdown 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.583+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 msgr2=0x556b05008cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.583+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05008cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc38050040 tx=0x7fbc38000bc0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 shutdown_connections 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x556b0507c0e0 0x556b0507e4d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b04fffbd0 0x556b05009e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc257fa640 0 client.27257 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b04ffd190 0x556b05009350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 --2- 172.21.15.96:0/1500606897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b04ffa860 0x556b05008cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 >> 172.21.15.96:0/1500606897 conn(0x556b04ff59b0 msgr2=0x556b04ff43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 shutdown_connections 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 1 -- 172.21.15.96:0/1500606897 wait complete. 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.584+0000 7fbc46c8c740 20 client.27257 trim_cache size 0 max 16384 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:57:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:57:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.586+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 msgr2=0x7fbc280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.586+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 0x7fbc280b7820 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00a5c0 tx=0x7fbc2c014040 comp rx=0 tx=0).stop 2024-09-10T22:57:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.586+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 msgr2=0x7fbc3c06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.586+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eb10 tx=0x7fbc3400ef10 comp rx=0 tx=0).stop 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 shutdown_connections 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc3c1029c0 0x7fbc3c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbc280b5360 0x7fbc280b7820 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc3c103380 0x7fbc3c06e060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 --2- 172.21.15.96:0/519682530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc3c101870 0x7fbc3c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 >> 172.21.15.96:0/519682530 conn(0x7fbc3c0fd130 msgr2=0x7fbc3c0731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 shutdown_connections 2024-09-10T22:57:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:43.587+0000 7fbc43ebd640 1 -- 172.21.15.96:0/519682530 wait complete. 2024-09-10T22:57:45.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:45 smithi096 ceph-mon[30869]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T22:57:45.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:45 smithi121 ceph-mon[35119]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T22:57:45.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:45 smithi117 ceph-mon[35081]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-10T22:57:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 610.0 s, 2353.0 tps, lat 42.382 ms stddev 52.237 2024-09-10T22:57:47.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:47 smithi096 ceph-mon[30869]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-10T22:57:47.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:47 smithi121 ceph-mon[35119]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-10T22:57:47.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:47 smithi117 ceph-mon[35081]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 705 op/s 2024-09-10T22:57:49.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:49 smithi121 ceph-mon[35119]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 663 op/s 2024-09-10T22:57:49.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:49 smithi117 ceph-mon[35081]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 663 op/s 2024-09-10T22:57:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:49 smithi096 ceph-mon[30869]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 663 op/s 2024-09-10T22:57:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 615.0 s, 1606.2 tps, lat 62.697 ms stddev 76.001 2024-09-10T22:57:51.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:51 smithi121 ceph-mon[35119]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-09-10T22:57:51.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:51 smithi117 ceph-mon[35081]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-09-10T22:57:51.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:51 smithi096 ceph-mon[30869]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 693 op/s 2024-09-10T22:57:53.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:53 smithi121 ceph-mon[35119]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-09-10T22:57:53.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:53 smithi117 ceph-mon[35081]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-09-10T22:57:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:53 smithi096 ceph-mon[30869]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-09-10T22:57:55.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:55 smithi117 ceph-mon[35081]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 MiB/s wr, 714 op/s 2024-09-10T22:57:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:55 smithi096 ceph-mon[30869]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 MiB/s wr, 714 op/s 2024-09-10T22:57:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:55 smithi121 ceph-mon[35119]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 MiB/s wr, 714 op/s 2024-09-10T22:57:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 620.0 s, 2492.0 tps, lat 40.219 ms stddev 46.430 2024-09-10T22:57:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 -- 172.21.15.96:0/2295902201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41029e0 msgr2=0x7fdeb4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/2295902201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41029e0 0x7fdeb4102e60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009930 tx=0x7fde9c02a420 comp rx=0 tx=0).stop 2024-09-10T22:57:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 -- 172.21.15.96:0/2295902201 shutdown_connections 2024-09-10T22:57:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/2295902201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 0x7fdeb4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/2295902201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41029e0 0x7fdeb4102e60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/2295902201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4101890 0x7fdeb4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 -- 172.21.15.96:0/2295902201 >> 172.21.15.96:0/2295902201 conn(0x7fdeb40fd170 msgr2=0x7fdeb40ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 -- 172.21.15.96:0/2295902201 shutdown_connections 2024-09-10T22:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.714+0000 7fdeba3bb640 1 -- 172.21.15.96:0/2295902201 wait complete. 2024-09-10T22:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 Processor -- start 2024-09-10T22:57:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 -- start start 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 0x7fdeb419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 0x7fdeb41a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d4e0 con 0x7fdeb4101890 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d360 con 0x7fdeb41029e0 2024-09-10T22:57:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeba3bb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d660 con 0x7fdeb41033a0 2024-09-10T22:57:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeb3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41308/0 (socket says 172.21.15.96:41308) 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeb3fff640 1 -- 172.21.15.96:0/103240215 learned_addr learned my addr 172.21.15.96:0/103240215 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeb8958640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 0x7fdeb41a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.715+0000 7fdeabfff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 0x7fdeb419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 -- 172.21.15.96:0/103240215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 msgr2=0x7fdeb41a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 0x7fdeb41a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 -- 172.21.15.96:0/103240215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 msgr2=0x7fdeb419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 0x7fdeb419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 -- 172.21.15.96:0/103240215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb41a09c0 con 0x7fdeb4101890 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeabfff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 0x7fdeb419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9f0 tx=0x7fdea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb1ffb640 1 -- 172.21.15.96:0/103240215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea400cd80 con 0x7fdeb4101890 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb1ffb640 1 -- 172.21.15.96:0/103240215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40108e0 con 0x7fdeb4101890 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeb1ffb640 1 -- 172.21.15.96:0/103240215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4018980 con 0x7fdeb4101890 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb41a0c50 con 0x7fdeb4101890 2024-09-10T22:57:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.716+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41a11a0 con 0x7fdeb4101890 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.720+0000 7fdeb1ffb640 1 -- 172.21.15.96:0/103240215 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdea4010430 con 0x7fdeb4101890 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.720+0000 7fdebd265740 1 Processor -- start 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 -- start start 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ef116f0 con 0x55ab0efa21b0 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ef00b20 con 0x55ab0ef9f860 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdebd265740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ef04930 con 0x55ab0efa2bc0 2024-09-10T22:57:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37988/0 (socket says 172.21.15.96:37988) 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 -- 172.21.15.96:0/100206560 learned_addr learned my addr 172.21.15.96:0/100206560 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeabfff640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 -- 172.21.15.96:0/100206560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa2bc0 msgr2=0x55ab0efa3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 -- 172.21.15.96:0/100206560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 msgr2=0x55ab0efa2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb8958640 1 -- 172.21.15.96:0/100206560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0eec6530 con 0x55ab0ef9f860 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.721+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeb8958640 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb33c0b39899be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeb1ffb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 0x7fde940b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/100206560 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004d20 con 0x55ab0ef9f860 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/100206560 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac004050 con 0x55ab0ef9f860 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/100206560 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac005280 con 0x55ab0ef9f860 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 0x7fde940b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.722+0000 7fdeb1ffb640 1 -- 172.21.15.96:0/103240215 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdea40e5fd0 con 0x7fdeb4101890 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 -- 172.21.15.96:0/100206560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 msgr2=0x55ab0efa1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac0118d0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 -- 172.21.15.96:0/100206560 shutdown_connections 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 --2- 172.21.15.96:0/100206560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efa1c70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.723+0000 7fdebd265740 1 -- 172.21.15.96:0/100206560 >> 172.21.15.96:0/100206560 conn(0x55ab0ef9a9b0 msgr2=0x55ab0ef9b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.724+0000 7fdebd265740 1 -- 172.21.15.96:0/100206560 shutdown_connections 2024-09-10T22:57:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.724+0000 7fdeb3fff640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 0x7fde940b7820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9c0 tx=0x7fdea4018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.724+0000 7fdebd265740 1 -- 172.21.15.96:0/100206560 wait complete. 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.724+0000 7fdebd265740 1 -- start start 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.725+0000 7fdebd265740 1 Processor -- start 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.725+0000 7fdebd265740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39056/0 (socket says 172.21.15.96:39056) 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdeb8958640 1 -- 172.21.15.96:0/3170504958 learned_addr learned my addr 172.21.15.96:0/3170504958 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 0x55ab0efa9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efaa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdeabfff640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 0x55ab0efa9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab0ee38ab0 con 0x55ab0efa2bc0 2024-09-10T22:57:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab0eea6730 con 0x55ab0efa21b0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ab0efa3b90 con 0x55ab0ef9f860 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.726+0000 7fdebd265740 10 client.0 _ll_register_callbacks cb 0x55ab0eefa6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeb8958640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efaa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 -- 172.21.15.96:0/3170504958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 msgr2=0x55ab0efa8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 -- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 msgr2=0x55ab0efaa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efaa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0ec62790 con 0x55ab0efa21b0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeb8958640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efaa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac004050 con 0x55ab0efa21b0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdeabfff640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 0x55ab0efa9570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006010 tx=0x7fde9c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a03c1b7513c942 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdea37fe640 10 client.27266 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdea37fe640 1 -- 172.21.15.96:0/3170504958 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c03f070 con 0x55ab0efa21b0 2024-09-10T22:57:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.727+0000 7fdea37fe640 1 -- 172.21.15.96:0/3170504958 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fde9c004450 con 0x55ab0efa21b0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdea37fe640 1 -- 172.21.15.96:0/3170504958 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde9c035070 con 0x55ab0efa21b0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 10 client.27266 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ab0ecdb620 con 0x55ab0efa21b0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdea37fe640 1 -- 172.21.15.96:0/3170504958 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fde9c005030 con 0x55ab0efa21b0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 10 client.27266 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 10 client.27266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 20 client.27266 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0efc57c0 0x55ab0efc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 4 client.27266 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab0ef13640 con 0x55ab0efc57c0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.728+0000 7fdeb8958640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0efc57c0 0x55ab0efc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.729+0000 7fdeb8958640 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0efc57c0 0x55ab0efc7bb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c381d23b721e90bb server_cookie=f5b780106d974473 in_seq=0 out_seq=0 2024-09-10T22:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.729+0000 7fdea37fe640 10 client.27266 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdea37fe640 1 -- 172.21.15.96:0/3170504958 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab0ef13640 con 0x55ab0efc57c0 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdea37fe640 10 client.27266 handle_command_reply: tid=0 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 client.27266 shutdown 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 msgr2=0x55ab0efa9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 0x55ab0efa9570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006010 tx=0x7fde9c031850 comp rx=0 tx=0).stop 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 shutdown_connections 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0efc57c0 0x55ab0efc7bb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa21b0 0x55ab0efa9570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efaa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 --2- 172.21.15.96:0/3170504958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 >> 172.21.15.96:0/3170504958 conn(0x55ab0ef9a9b0 msgr2=0x55ab0ef99bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdea37fe640 0 client.27266 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 shutdown_connections 2024-09-10T22:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 1 -- 172.21.15.96:0/3170504958 wait complete. 2024-09-10T22:57:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.730+0000 7fdebd265740 20 client.27266 trim_cache size 0 max 16384 2024-09-10T22:57:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 Processor -- start 2024-09-10T22:57:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 -- start start 2024-09-10T22:57:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.752+0000 7fdebd265740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a7c60 con 0x55ab0efa21b0 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdebd265740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a97a0 con 0x55ab0efa2bc0 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41330/0 (socket says 172.21.15.96:41330) 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39068/0 (socket says 172.21.15.96:39068) 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 -- 172.21.15.96:0/1027876254 learned_addr learned my addr 172.21.15.96:0/1027876254 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a8330 con 0x55ab0ef9f860 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeb8958640 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 -- 172.21.15.96:0/1027876254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 msgr2=0x55ab0efa95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 -- 172.21.15.96:0/1027876254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa2bc0 msgr2=0x55ab0efa8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 -- 172.21.15.96:0/1027876254 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0efb02d0 con 0x55ab0efa21b0 2024-09-10T22:57:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeb8958640 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdeabfff640 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fde9c005ab0 tx=0x7fde9c035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9425306dda8aad4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdea9ffb640 1 -- 172.21.15.96:0/1027876254 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c005d50 con 0x55ab0efa21b0 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdea9ffb640 1 -- 172.21.15.96:0/1027876254 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac004050 con 0x55ab0efa21b0 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.753+0000 7fdea9ffb640 1 -- 172.21.15.96:0/1027876254 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0586b0 con 0x55ab0efa21b0 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.754+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 msgr2=0x55ab0efa8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.754+0000 7fdebd265740 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fde9c005ab0 tx=0x7fde9c035000 comp rx=0 tx=0).stop 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 shutdown_connections 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0efa2bc0 0x55ab0efa8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa21b0 0x55ab0efa8110 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 --2- 172.21.15.96:0/1027876254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0ef9f860 0x55ab0efa95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 >> 172.21.15.96:0/1027876254 conn(0x55ab0ef9a9b0 msgr2=0x55ab0efa8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 shutdown_connections 2024-09-10T22:57:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.755+0000 7fdebd265740 1 -- 172.21.15.96:0/1027876254 wait complete. 2024-09-10T22:57:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 -- start start 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 Processor -- start 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efd9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa21b0 0x55ab0efd87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efd8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a90d0 con 0x55ab0efa2bc0 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a8a00 con 0x55ab0ef9f860 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a5c40 con 0x55ab0efa21b0 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdebd265740 10 client.0 _ll_register_callbacks cb 0x7fdea4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.756+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efd8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efd8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41332/0 (socket says 172.21.15.96:41332) 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 -- 172.21.15.96:0/2533203241 learned_addr learned my addr 172.21.15.96:0/2533203241 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efd9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeabfff640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa21b0 0x55ab0efd87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 -- 172.21.15.96:0/2533203241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa21b0 msgr2=0x55ab0efd87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa21b0 0x55ab0efd87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 -- 172.21.15.96:0/2533203241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 msgr2=0x55ab0efd8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efd8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 -- 172.21.15.96:0/2533203241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0ef9c0b0 con 0x55ab0ef9f860 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdeb8958640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efd9a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdeac002230 tx=0x7fdeac000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52c9fb934a4df27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00f6e0 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac004050 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdeac00f880 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdeac004800 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdebd265740 10 client.27272 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.757+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab0f0324a0 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac004050 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdeac0289e0 con 0x55ab0ef9f860 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 10 client.27272 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 10 client.27272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 20 client.27272 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0f032c60 0x55ab0f035050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 4 client.27272 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:57:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab0ef0f0f0 con 0x55ab0f032c60 2024-09-10T22:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdeabfff640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0f032c60 0x55ab0f035050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdeabfff640 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0f032c60 0x55ab0f035050 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156791392f40bebb server_cookie=92ddb1b8e6096cab in_seq=0 out_seq=0 2024-09-10T22:57:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.758+0000 7fdea37fe640 10 client.27272 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdea37fe640 1 -- 172.21.15.96:0/2533203241 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab0ef0f0f0 con 0x55ab0f032c60 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdea37fe640 10 client.27272 handle_command_reply: tid=0 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 client.27272 shutdown 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 msgr2=0x55ab0efd9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efd9a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdeac002230 tx=0x7fdeac000f40 comp rx=0 tx=0).stop 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 shutdown_connections 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ab0f032c60 0x55ab0f035050 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab0efa2bc0 0x55ab0efd8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab0efa21b0 0x55ab0efd87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 --2- 172.21.15.96:0/2533203241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab0ef9f860 0x55ab0efd9a90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdea37fe640 0 client.27272 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 >> 172.21.15.96:0/2533203241 conn(0x55ab0ef9a9b0 msgr2=0x55ab0ef9e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 shutdown_connections 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 1 -- 172.21.15.96:0/2533203241 wait complete. 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.771+0000 7fdebd265740 20 client.27272 trim_cache size 0 max 16384 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:57:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.772+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 msgr2=0x7fde940b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.772+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 0x7fde940b7820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9c0 tx=0x7fdea4018420 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.772+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 msgr2=0x7fdeb406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.772+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e9f0 tx=0x7fdea400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 shutdown_connections 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb41029e0 0x7fdeb419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fde940b5360 0x7fde940b7820 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb41033a0 0x7fdeb41a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 --2- 172.21.15.96:0/103240215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4101890 0x7fdeb406bce0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 >> 172.21.15.96:0/103240215 conn(0x7fdeb40fd170 msgr2=0x7fdeb4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 shutdown_connections 2024-09-10T22:57:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:57:56.773+0000 7fdeba3bb640 1 -- 172.21.15.96:0/103240215 wait complete. 2024-09-10T22:57:57.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:57 smithi117 ceph-mon[35081]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 747 op/s 2024-09-10T22:57:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:57 smithi096 ceph-mon[30869]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 747 op/s 2024-09-10T22:57:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:57 smithi121 ceph-mon[35119]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 747 op/s 2024-09-10T22:57:58.476 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:57:58.845 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:57:59.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.457+0000 7fc8c5746640 1 -- 172.21.15.96:0/2699441819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c01135d0 msgr2=0x7fc8c0115a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.457+0000 7fc8c5746640 1 --2- 172.21.15.96:0/2699441819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c01135d0 0x7fc8c0115a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009ca0 tx=0x7fc8b802a8f0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 -- 172.21.15.96:0/2699441819 shutdown_connections 2024-09-10T22:57:59.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 --2- 172.21.15.96:0/2699441819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c01135d0 0x7fc8c0115a00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 --2- 172.21.15.96:0/2699441819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c010aa20 0x7fc8c0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 --2- 172.21.15.96:0/2699441819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010a020 0x7fc8c010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 -- 172.21.15.96:0/2699441819 >> 172.21.15.96:0/2699441819 conn(0x7fc8c00788e0 msgr2=0x7fc8c007ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 -- 172.21.15.96:0/2699441819 shutdown_connections 2024-09-10T22:57:59.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.458+0000 7fc8c5746640 1 -- 172.21.15.96:0/2699441819 wait complete. 2024-09-10T22:57:59.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.459+0000 7fc8c5746640 1 Processor -- start 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 -- start start 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c010a020 0x7fc8c0112900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c01135d0 0x7fc8c010cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c006d470 con 0x7fc8c010a020 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c006d2f0 con 0x7fc8c010aa20 2024-09-10T22:57:59.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.462+0000 7fc8c5746640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c006d5f0 con 0x7fc8c01135d0 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.466+0000 7fc8be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.466+0000 7fc8be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38034/0 (socket says 172.21.15.96:38034) 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.466+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3724037885 learned_addr learned my addr 172.21.15.96:0/3724037885 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.466+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c01135d0 0x7fc8c010cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.466+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3724037885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c01135d0 msgr2=0x7fc8c010cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.467+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c01135d0 0x7fc8c010cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.467+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c010a020 msgr2=0x7fc8c0112900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.467+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c010a020 0x7fc8c0112900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.467+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3724037885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c010d500 con 0x7fc8c010aa20 2024-09-10T22:57:59.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.470+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b450 tx=0x7fc8b400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.471+0000 7fc8a7fff640 1 -- 172.21.15.96:0/3724037885 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4015020 con 0x7fc8c010aa20 2024-09-10T22:57:59.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.471+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c010d790 con 0x7fc8c010aa20 2024-09-10T22:57:59.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.471+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c007dc10 con 0x7fc8c010aa20 2024-09-10T22:57:59.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.471+0000 7fc8a7fff640 1 -- 172.21.15.96:0/3724037885 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b4011d40 con 0x7fc8c010aa20 2024-09-10T22:57:59.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.471+0000 7fc8a7fff640 1 -- 172.21.15.96:0/3724037885 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b400f340 con 0x7fc8c010aa20 2024-09-10T22:57:59.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 Processor -- start 2024-09-10T22:57:59.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 -- start start 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48a822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e489e9e50 con 0x558e48a73650 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e489d9280 con 0x558e48a7ff00 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e489dd090 con 0x558e48a73f70 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8a7fff640 1 -- 172.21.15.96:0/3724037885 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc8b400f4e0 con 0x7fc8c010aa20 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.476+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.477+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39092/0 (socket says 172.21.15.96:39092) 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.477+0000 7fc8beffd640 1 -- 172.21.15.96:0/2011692722 learned_addr learned my addr 172.21.15.96:0/2011692722 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:59.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.477+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.477+0000 7fc8beffd640 1 -- 172.21.15.96:0/2011692722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 msgr2=0x558e48a822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8a7fff640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 0x7fc8a80b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48a822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48a822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8beffd640 1 -- 172.21.15.96:0/2011692722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 msgr2=0x558e48a73a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a73a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8beffd640 1 -- 172.21.15.96:0/2011692722 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e48737650 con 0x558e48a73f70 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a73a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48a822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:57:59.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8a7fff640 1 -- 172.21.15.96:0/3724037885 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc8b40e5580 con 0x7fc8c010aa20 2024-09-10T22:57:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 0x7fc8a80b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bb10 tx=0x7fc8b000d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75267a8c8dd71286 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8a57fa640 1 -- 172.21.15.96:0/2011692722 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0016070 con 0x558e48a73f70 2024-09-10T22:57:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.478+0000 7fc8a57fa640 1 -- 172.21.15.96:0/2011692722 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b000dce0 con 0x558e48a73f70 2024-09-10T22:57:59.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8a57fa640 1 -- 172.21.15.96:0/2011692722 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0010460 con 0x558e48a73f70 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2011692722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 msgr2=0x558e48a7f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 0x7fc8a80b79f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009a60 tx=0x7fc8b80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bb10 tx=0x7fc8b000d2e0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2011692722 shutdown_connections 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48a822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a7f9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2011692722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a73a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2011692722 >> 172.21.15.96:0/2011692722 conn(0x558e48a71480 msgr2=0x558e48a72620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.479+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2011692722 shutdown_connections 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2011692722 wait complete. 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 -- start start 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 Processor -- start 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73f70 0x558e48a7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a7ff00 0x558e48a7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e4883a700 con 0x558e48a73f70 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e48974110 con 0x558e48a73650 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e48a82b80 con 0x558e48a7ff00 2024-09-10T22:57:59.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.480+0000 7fc8c7b1b740 10 client.0 _ll_register_callbacks cb 0x558e4898d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8be7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73f70 0x558e48a7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38046/0 (socket says 172.21.15.96:38046) 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 -- 172.21.15.96:0/2308493715 learned_addr learned my addr 172.21.15.96:0/2308493715 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a7ff00 0x558e48a7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 -- 172.21.15.96:0/2308493715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a7ff00 msgr2=0x558e48a7d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a7ff00 0x558e48a7d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 -- 172.21.15.96:0/2308493715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73f70 msgr2=0x558e48a7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73f70 0x558e48a7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 -- 172.21.15.96:0/2308493715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e489a1c40 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8beffd640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bae0 tx=0x7fc8b000dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c098fc2b75463b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.481+0000 7fc8a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0016070 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc8b0004030 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b000dce0 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc8b0004e40 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8c7b1b740 10 client.27281 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e48a8e410 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.482+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b000dce0 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc8b00035f0 con 0x558e48a73650 2024-09-10T22:57:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 10 client.27281 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 20 client.27281 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48a8f7c0 0x558e48a91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 4 client.27281 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e489ebda0 con 0x558e48a8f7c0 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.483+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48a8f7c0 0x558e48a91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.484+0000 7fc8a2ff5640 10 client.27281 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.484+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48a8f7c0 0x558e48a91bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782af0bb359b4c35 server_cookie=de0ed356c672106a in_seq=0 out_seq=0 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.484+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/2308493715 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e489ebda0 con 0x558e48a8f7c0 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.484+0000 7fc8a2ff5640 10 client.27281 handle_command_reply: tid=0 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.484+0000 7fc8c7b1b740 1 client.27281 shutdown 2024-09-10T22:57:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 msgr2=0x558e48a7f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bae0 tx=0x7fc8b000dfb0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 shutdown_connections 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48a8f7c0 0x558e48a91bb0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a7ff00 0x558e48a7d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73f70 0x558e48a7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/2308493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73650 0x558e48a7f740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8a2ff5640 0 client.27281 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 >> 172.21.15.96:0/2308493715 conn(0x558e48a71480 msgr2=0x558e48a76ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 shutdown_connections 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/2308493715 wait complete. 2024-09-10T22:57:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.485+0000 7fc8c7b1b740 20 client.27281 trim_cache size 0 max 16384 2024-09-10T22:57:59.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.508+0000 7fc8c7b1b740 1 Processor -- start 2024-09-10T22:57:59.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.508+0000 7fc8c7b1b740 1 -- start start 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a7ff00 0x558e48a97210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00e370 con 0x558e48a7ff00 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00feb0 con 0x558e48a73f70 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8c7b1b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00ea40 con 0x558e48a73650 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a7ff00 0x558e48a97210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38060/0 (socket says 172.21.15.96:38060) 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39110/0 (socket says 172.21.15.96:39110) 2024-09-10T22:57:59.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 -- 172.21.15.96:0/847257141 learned_addr learned my addr 172.21.15.96:0/847257141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 -- 172.21.15.96:0/847257141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 msgr2=0x558e48a79da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 -- 172.21.15.96:0/847257141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a7ff00 msgr2=0x558e48a97210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a7ff00 0x558e48a97210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 -- 172.21.15.96:0/847257141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e48a7d460 con 0x558e48a73f70 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8beffd640 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40a9050 tx=0x7fc8b40e5040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3951b48060fcacbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.509+0000 7fc8a4ff9640 1 -- 172.21.15.96:0/847257141 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40b5370 con 0x558e48a73f70 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8a4ff9640 1 -- 172.21.15.96:0/847257141 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b40af040 con 0x558e48a73f70 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8a4ff9640 1 -- 172.21.15.96:0/847257141 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40e50b0 con 0x558e48a73f70 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/847257141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 msgr2=0x558e48a7a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40a9050 tx=0x7fc8b40e5040 comp rx=0 tx=0).stop 2024-09-10T22:57:59.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/847257141 shutdown_connections 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a73f70 0x558e48a7a2e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a7ff00 0x558e48a97210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/847257141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73650 0x558e48a79da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/847257141 >> 172.21.15.96:0/847257141 conn(0x558e48a71480 msgr2=0x558e48a7e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.510+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/847257141 shutdown_connections 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.511+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/847257141 wait complete. 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.511+0000 7fc8c7b1b740 1 -- start start 2024-09-10T22:57:59.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.511+0000 7fc8c7b1b740 1 Processor -- start 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.511+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8c7b1b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41378/0 (socket says 172.21.15.96:41378) 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8beffd640 1 -- 172.21.15.96:0/3164780517 learned_addr learned my addr 172.21.15.96:0/3164780517 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48af5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00f7e0 con 0x558e48a73650 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8bf7fe640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48af5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00f110 con 0x558e48a7ff00 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.512+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac00c350 con 0x558e48a73f70 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8c7b1b740 10 client.0 _ll_register_callbacks cb 0x7fc8b8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 msgr2=0x558e48af5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48af5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3164780517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 msgr2=0x558e48a84990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e48af5c80 con 0x558e48a73f70 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8beffd640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b40af040 con 0x558e48a73f70 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8be7fc640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a84f10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x558e48a82790 tx=0x7fc8b40972b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2a9bea061a303c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:57:59.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8a2ff5640 10 client.27364 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/3164780517 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40b5370 con 0x558e48a73f70 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/3164780517 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc8b40d1050 con 0x558e48a73f70 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/3164780517 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc8b40a8050 con 0x558e48a73f70 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8c7b1b740 10 client.27364 fetch_fsmap learned FSMap version 32 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.513+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x558e48afcfe0 con 0x558e48a73f70 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/3164780517 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc8b40cd320 con 0x558e48a73f70 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 10 client.27364 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:57:59.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 20 client.27364 populate_metadata read hostname 'smithi096' 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48afd600 0x558e48affa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 4 client.27364 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e489e7850 con 0x558e48afd600 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.514+0000 7fc8beffd640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48afd600 0x558e48affa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8beffd640 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48afd600 0x558e48affa20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bb8386be85309d server_cookie=77883905c2905097 in_seq=0 out_seq=0 2024-09-10T22:57:59.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8a2ff5640 10 client.27364 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8a2ff5640 1 -- 172.21.15.96:0/3164780517 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x558e489e7850 con 0x558e48afd600 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8a2ff5640 10 client.27364 handle_command_reply: tid=0 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 client.27364 shutdown 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 msgr2=0x558e48a84f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a84f10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x558e48a82790 tx=0x7fc8b40972b0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 shutdown_connections 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558e48afd600 0x558e48affa20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e48a7ff00 0x558e48af5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8a2ff5640 0 client.27364 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e48a73f70 0x558e48a84f10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 --2- 172.21.15.96:0/3164780517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e48a73650 0x558e48a84990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 >> 172.21.15.96:0/3164780517 conn(0x558e48a71480 msgr2=0x558e48a6fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 shutdown_connections 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.517+0000 7fc8c7b1b740 1 -- 172.21.15.96:0/3164780517 wait complete. 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c7b1b740 20 client.27364 trim_cache size 0 max 16384 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:57:59.527 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.528 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.529 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:57:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:57:59.533 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:57:59.534 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:57:59.535 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 msgr2=0x7fc8a80b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 0x7fc8a80b79f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009a60 tx=0x7fc8b80095c0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 msgr2=0x7fc8c0112e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b450 tx=0x7fc8b400b910 comp rx=0 tx=0).stop 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 shutdown_connections 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8c010aa20 0x7fc8c0112e40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc8a80b5530 0x7fc8a80b79f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8c01135d0 0x7fc8c010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 --2- 172.21.15.96:0/3724037885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8c010a020 0x7fc8c0112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:57:59.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 >> 172.21.15.96:0/3724037885 conn(0x7fc8c00788e0 msgr2=0x7fc8c0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:57:59.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.518+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 shutdown_connections 2024-09-10T22:57:59.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:57:59.519+0000 7fc8c5746640 1 -- 172.21.15.96:0/3724037885 wait complete. 2024-09-10T22:57:59.537 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:57:59 smithi096 ceph-mon[30869]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T22:57:59.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:57:59 smithi117 ceph-mon[35081]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T22:57:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:57:59 smithi121 ceph-mon[35119]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 579 op/s 2024-09-10T22:58:00.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:58:00.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:00.819 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 625.0 s, 2461.4 tps, lat 40.326 ms stddev 45.729 2024-09-10T22:58:01.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.393+0000 7f98e5ff0640 1 -- 172.21.15.96:0/1038327880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e01087c0 msgr2=0x7f98e0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.393+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/1038327880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e01087c0 0x7f98e0108c00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.393+0000 7f98e5ff0640 1 -- 172.21.15.96:0/1038327880 shutdown_connections 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.393+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/1038327880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0109140 0x7f98e010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.393+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/1038327880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e01087c0 0x7f98e0108c00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.394+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/1038327880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e0102db0 0x7f98e0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.394+0000 7f98e5ff0640 1 -- 172.21.15.96:0/1038327880 >> 172.21.15.96:0/1038327880 conn(0x7f98e00fe800 msgr2=0x7f98e0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.394+0000 7f98e5ff0640 1 -- 172.21.15.96:0/1038327880 shutdown_connections 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.394+0000 7f98e5ff0640 1 -- 172.21.15.96:0/1038327880 wait complete. 2024-09-10T22:58:01.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.394+0000 7f98e5ff0640 1 Processor -- start 2024-09-10T22:58:01.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 -- start start 2024-09-10T22:58:01.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e01087c0 0x7f98e010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 0x7f98e01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0112dc0 con 0x7f98e01087c0 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0112c40 con 0x7f98e0102db0 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98e5ff0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0112f40 con 0x7f98e0109140 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e01087c0 0x7f98e010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38096/0 (socket says 172.21.15.96:38096) 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98df7fe640 1 -- 172.21.15.96:0/2243411381 learned_addr learned my addr 172.21.15.96:0/2243411381 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.396+0000 7f98dffff640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 0x7f98e01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 -- 172.21.15.96:0/2243411381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 msgr2=0x7f98e01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 0x7f98e01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 -- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e01087c0 msgr2=0x7f98e010f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e01087c0 0x7f98e010f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 -- 172.21.15.96:0/2243411381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01a2700 con 0x7f98e0102db0 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98dffff640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 0x7f98e01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98df7fe640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98d000d950 tx=0x7f98d000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0017030 con 0x7f98e0102db0 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d0004020 con 0x7f98e0102db0 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e01a29f0 con 0x7f98e0102db0 2024-09-10T22:58:01.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0003c50 con 0x7f98e0102db0 2024-09-10T22:58:01.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.397+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e01ac2d0 con 0x7f98e0102db0 2024-09-10T22:58:01.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.398+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e0103190 con 0x7f98e0102db0 2024-09-10T22:58:01.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.401+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f98d0009a30 con 0x7f98e0102db0 2024-09-10T22:58:01.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.402+0000 7f98dcff9640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 0x7f98c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:01.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.402+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98d00e57c0 con 0x7f98e0102db0 2024-09-10T22:58:01.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.402+0000 7f98deffd640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 0x7f98c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:01.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.403+0000 7f98deffd640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 0x7f98c80b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f98d4031440 tx=0x7f98d4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:01.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.404+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f98d0021070 con 0x7f98e0102db0 2024-09-10T22:58:01.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.700+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e01002f0 con 0x7f98e0102db0 2024-09-10T22:58:01.703 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:01 smithi096 ceph-mon[30869]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-09-10T22:58:01.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.701+0000 7f98dcff9640 1 -- 172.21.15.96:0/2243411381 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f98d00ac680 con 0x7f98e0102db0 2024-09-10T22:58:01.704 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:58:01.704 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:58:01.704 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 msgr2=0x7f98c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 0x7f98c80b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f98d4031440 tx=0x7f98d4005b50 comp rx=0 tx=0).stop 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 msgr2=0x7f98e010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98d000d950 tx=0x7f98d000de10 comp rx=0 tx=0).stop 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 shutdown_connections 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98c80b52e0 0x7f98c80b77a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98e0109140 0x7f98e01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98e01087c0 0x7f98e010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 --2- 172.21.15.96:0/2243411381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0102db0 0x7f98e010ecb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 >> 172.21.15.96:0/2243411381 conn(0x7f98e00fe800 msgr2=0x7f98e0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 shutdown_connections 2024-09-10T22:58:01.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:01.703+0000 7f98e5ff0640 1 -- 172.21.15.96:0/2243411381 wait complete. 2024-09-10T22:58:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:01 smithi121 ceph-mon[35119]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-09-10T22:58:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:01 smithi117 ceph-mon[35081]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-09-10T22:58:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:02 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2243411381' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:02.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:02 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2243411381' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:02.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:02 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2243411381' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:02.806 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:58:03.160 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:03.636 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:03 smithi096 ceph-mon[30869]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-09-10T22:58:03.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 -- 172.21.15.96:0/2973311520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb010f480 msgr2=0x7f3bb0111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 --2- 172.21.15.96:0/2973311520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb010f480 0x7f3bb0111870 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009ca0 tx=0x7f3ba802a950 comp rx=0 tx=0).stop 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 -- 172.21.15.96:0/2973311520 shutdown_connections 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 --2- 172.21.15.96:0/2973311520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb010f480 0x7f3bb0111870 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 --2- 172.21.15.96:0/2973311520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 0x7f3bb010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 --2- 172.21.15.96:0/2973311520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb01018d0 0x7f3bb0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 -- 172.21.15.96:0/2973311520 >> 172.21.15.96:0/2973311520 conn(0x7f3bb00fd740 msgr2=0x7f3bb00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 -- 172.21.15.96:0/2973311520 shutdown_connections 2024-09-10T22:58:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.748+0000 7f3bb491f640 1 -- 172.21.15.96:0/2973311520 wait complete. 2024-09-10T22:58:03.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.749+0000 7f3bb491f640 1 Processor -- start 2024-09-10T22:58:03.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.749+0000 7f3bb491f640 1 -- start start 2024-09-10T22:58:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.749+0000 7f3bb491f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bb491f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 0x7f3bb019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bb491f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 0x7f3bb01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bb491f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0114f90 con 0x7f3bb01018d0 2024-09-10T22:58:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bb491f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0114e10 con 0x7f3bb010f480 2024-09-10T22:58:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bb491f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0115110 con 0x7f3bb01021f0 2024-09-10T22:58:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41418/0 (socket says 172.21.15.96:41418) 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 -- 172.21.15.96:0/1549890812 learned_addr learned my addr 172.21.15.96:0/1549890812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3baed9d640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 0x7f3bb01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 0x7f3bb019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 -- 172.21.15.96:0/1549890812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 msgr2=0x7f3bb019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 0x7f3bb019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 -- 172.21.15.96:0/1549890812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 msgr2=0x7f3bb01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 0x7f3bb01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.750+0000 7f3bae59c640 1 -- 172.21.15.96:0/1549890812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb01a1550 con 0x7f3bb01018d0 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.751+0000 7f3baed9d640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 0x7f3bb01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.751+0000 7f3bae59c640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c920 tx=0x7f3ba000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.752+0000 7f3ba77fe640 1 -- 172.21.15.96:0/1549890812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0007c70 con 0x7f3bb01018d0 2024-09-10T22:58:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.752+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb01a1840 con 0x7f3bb01018d0 2024-09-10T22:58:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.752+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a80d0 con 0x7f3bb01018d0 2024-09-10T22:58:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.753+0000 7f3ba77fe640 1 -- 172.21.15.96:0/1549890812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba0007e10 con 0x7f3bb01018d0 2024-09-10T22:58:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.753+0000 7f3ba77fe640 1 -- 172.21.15.96:0/1549890812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000e750 con 0x7f3bb01018d0 2024-09-10T22:58:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.755+0000 7f3bb6cf4740 1 Processor -- start 2024-09-10T22:58:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 -- start start 2024-09-10T22:58:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4f2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a4f5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c99a45cea0 con 0x55c99a4e6df0 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38112/0 (socket says 172.21.15.96:38112) 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a4f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3baed9d640 1 -- 172.21.15.96:0/1577606039 learned_addr learned my addr 172.21.15.96:0/1577606039 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.756+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c99a44c2d0 con 0x55c99a4e64d0 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c99a4500e0 con 0x55c99a4f2d80 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3ba77fe640 1 -- 172.21.15.96:0/1549890812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3ba000ea30 con 0x7f3bb01018d0 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3baed9d640 1 -- 172.21.15.96:0/1577606039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 msgr2=0x55c99a4f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3baed9d640 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a4f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3baed9d640 1 -- 172.21.15.96:0/1577606039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 msgr2=0x55c99a4f2840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3baed9d640 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3baed9d640 1 -- 172.21.15.96:0/1577606039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99a1aa650 con 0x55c99a4e64d0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.757+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a4f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3ba77fe640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 0x7f3b9c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3baed9d640 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009ca0 tx=0x7f3ba8005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb80f07b0e583842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3ba77fe640 1 -- 172.21.15.96:0/1549890812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3ba00e5a30 con 0x7f3bb01018d0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3ba4ff9640 1 -- 172.21.15.96:0/1577606039 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8048070 con 0x55c99a4e64d0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3bae59c640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 0x7f3b9c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3ba4ff9640 1 -- 172.21.15.96:0/1577606039 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba802fec0 con 0x55c99a4e64d0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.759+0000 7f3ba4ff9640 1 -- 172.21.15.96:0/1577606039 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8034040 con 0x55c99a4e64d0 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 msgr2=0x55c99a4e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bae59c640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 0x7f3b9c0b7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00a9500 tx=0x7f3ba00a8960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009ca0 tx=0x7f3ba8005fa0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 shutdown_connections 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a4f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/1577606039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4e68b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 >> 172.21.15.96:0/1577606039 conn(0x55c99a4e4300 msgr2=0x55c99a4e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 shutdown_connections 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.760+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/1577606039 wait complete. 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3bb6cf4740 1 -- start start 2024-09-10T22:58:03.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3bb6cf4740 1 Processor -- start 2024-09-10T22:58:03.767 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:03 smithi121 ceph-mon[35119]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-09-10T22:58:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 0x55c99a4f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41442/0 (socket says 172.21.15.96:41442) 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.761+0000 7f3ba7fff640 1 -- 172.21.15.96:0/3950021161 learned_addr learned my addr 172.21.15.96:0/3950021161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4f2d80 0x55c99a4f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c99a2ad700 con 0x55c99a4e6df0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c99a3e7160 con 0x55c99a4f2d80 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c99a4f59f0 con 0x55c99a4e64d0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3baed9d640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 0x55c99a4f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 10 client.0 _ll_register_callbacks cb 0x55c99a4009e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3baed9d640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4f2d80 0x55c99a4f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 -- 172.21.15.96:0/3950021161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 msgr2=0x55c99a4f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 0x55c99a4f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 -- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4f2d80 msgr2=0x55c99a4f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4f2d80 0x55c99a4f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99a414c90 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3baed9d640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 0x55c99a4f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3b980126d0 tx=0x7f3b98012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b866d3426bd6e310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3b95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800f920 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba802fec0 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.762+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3b9800fac0 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b9800fcc0 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 10 client.17715 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c99a501430 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba802fec0 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3b9801c420 con 0x55c99a4e6df0 2024-09-10T22:58:03.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 10 client.17715 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 10 client.17715 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 20 client.17715 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5028e0 0x55c99a504d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 4 client.17715 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c99a45edf0 con 0x55c99a5028e0 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.763+0000 7f3baed9d640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5028e0 0x55c99a504d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.765+0000 7f3baed9d640 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5028e0 0x55c99a504d20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee1d228c3d1d66d9 server_cookie=29e618bb5b2bc6fa in_seq=0 out_seq=0 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.765+0000 7f3b95ffb640 10 client.17715 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3b95ffb640 1 -- 172.21.15.96:0/3950021161 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c99a45edf0 con 0x55c99a5028e0 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3b95ffb640 10 client.17715 handle_command_reply: tid=0 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 client.17715 shutdown 2024-09-10T22:58:03.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 msgr2=0x55c99a4efaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3b980126d0 tx=0x7f3b98012b90 comp rx=0 tx=0).stop 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 shutdown_connections 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5028e0 0x55c99a504d20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4f2d80 0x55c99a4f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e6df0 0x55c99a4efaa0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/3950021161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e64d0 0x55c99a4f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3b95ffb640 0 client.17715 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 >> 172.21.15.96:0/3950021161 conn(0x55c99a4e4300 msgr2=0x55c99a4e9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 shutdown_connections 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/3950021161 wait complete. 2024-09-10T22:58:03.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.766+0000 7f3bb6cf4740 20 client.17715 trim_cache size 0 max 16384 2024-09-10T22:58:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:03 smithi117 ceph-mon[35081]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.786+0000 7f3bb6cf4740 1 Processor -- start 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.786+0000 7f3bb6cf4740 1 -- start start 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4f2d80 0x55c99a4eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000e320 con 0x55c99a4f2d80 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000fe60 con 0x55c99a4e64d0 2024-09-10T22:58:03.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000e9f0 con 0x55c99a4e6df0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4f2d80 0x55c99a4eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:39174/0 (socket says 172.21.15.96:39174) 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 -- 172.21.15.96:0/979531087 learned_addr learned my addr 172.21.15.96:0/979531087 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3baed9d640 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 -- 172.21.15.96:0/979531087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 msgr2=0x55c99a4fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 -- 172.21.15.96:0/979531087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4f2d80 msgr2=0x55c99a4eccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4f2d80 0x55c99a4eccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 -- 172.21.15.96:0/979531087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99a4f03f0 con 0x55c99a4e6df0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.787+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000c00 tx=0x7f3b9800f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f246764596e78b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3b97fff640 1 -- 172.21.15.96:0/979531087 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800f920 con 0x55c99a4e6df0 2024-09-10T22:58:03.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3b97fff640 1 -- 172.21.15.96:0/979531087 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b9802b070 con 0x55c99a4e6df0 2024-09-10T22:58:03.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3b97fff640 1 -- 172.21.15.96:0/979531087 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b980054f0 con 0x55c99a4e6df0 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/979531087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 msgr2=0x55c99a4f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000c00 tx=0x7f3b9800f830 comp rx=0 tx=0).stop 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.788+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/979531087 shutdown_connections 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.789+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4f2d80 0x55c99a4eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.789+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4e6df0 0x55c99a4f17e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.789+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/979531087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e64d0 0x55c99a4fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.789+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/979531087 >> 172.21.15.96:0/979531087 conn(0x55c99a4e4300 msgr2=0x55c99a4f10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/979531087 shutdown_connections 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/979531087 wait complete. 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- start start 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 Processor -- start 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 0x55c99a4f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a571010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000f790 con 0x55c99a4e64d0 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000f0c0 con 0x55c99a4e6df0 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9000c300 con 0x55c99a4f2d80 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 0x55c99a4f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38142/0 (socket says 172.21.15.96:38142) 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3baed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 0x55c99a4f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41470/0 (socket says 172.21.15.96:41470) 2024-09-10T22:58:03.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 -- 172.21.15.96:0/699800841 learned_addr learned my addr 172.21.15.96:0/699800841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3bb6cf4740 10 client.0 _ll_register_callbacks cb 0x7f3b98009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3baed9d640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a571010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 -- 172.21.15.96:0/699800841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 msgr2=0x55c99a571010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a571010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 -- 172.21.15.96:0/699800841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 msgr2=0x55c99a4f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 0x55c99a4f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3ba7fff640 1 -- 172.21.15.96:0/699800841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99a571750 con 0x55c99a4e6df0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.791+0000 7f3baed9d640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a571010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3ba7fff640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b9801c810 tx=0x7f3b9800fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a1dddae1534339 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98033020 con 0x55c99a4e6df0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9802b070 con 0x55c99a4e6df0 2024-09-10T22:58:03.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3b9801ca00 con 0x55c99a4e6df0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b9801cc00 con 0x55c99a4e6df0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3bb6cf4740 10 client.27308 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c99a578ac0 con 0x55c99a4e6df0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b9802b070 con 0x55c99a4e6df0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3b98003a40 con 0x55c99a4e6df0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3bb6cf4740 10 client.27308 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.792+0000 7f3bb6cf4740 10 client.27308 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3bb6cf4740 20 client.27308 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5790d0 0x55c99a57b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3bb6cf4740 4 client.27308 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c99a45a8a0 con 0x55c99a5790d0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3baed9d640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5790d0 0x55c99a57b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:03.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3b95ffb640 10 client.27308 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.793+0000 7f3baed9d640 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5790d0 0x55c99a57b4f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8da65a6d125e4759 server_cookie=5b11cfdef0869779 in_seq=0 out_seq=0 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3b95ffb640 1 -- 172.21.15.96:0/699800841 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c99a45a8a0 con 0x55c99a5790d0 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3b95ffb640 10 client.27308 handle_command_reply: tid=0 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 client.27308 shutdown 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 msgr2=0x55c99a4f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3b9801c810 tx=0x7f3b9800fb50 comp rx=0 tx=0).stop 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 shutdown_connections 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55c99a5790d0 0x55c99a57b4f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c99a4e6df0 0x55c99a4f90e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c99a4f2d80 0x55c99a571010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 --2- 172.21.15.96:0/699800841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c99a4e64d0 0x55c99a4f8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3b95ffb640 0 client.27308 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:03.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 >> 172.21.15.96:0/699800841 conn(0x55c99a4e4300 msgr2=0x55c99a4e3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 shutdown_connections 2024-09-10T22:58:03.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 1 -- 172.21.15.96:0/699800841 wait complete. 2024-09-10T22:58:03.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.794+0000 7f3bb6cf4740 20 client.27308 trim_cache size 0 max 16384 2024-09-10T22:58:03.802 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 msgr2=0x7f3b9c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 0x7f3b9c0b7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00a9500 tx=0x7f3ba00a8960 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 msgr2=0x7f3bb019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c920 tx=0x7f3ba000cde0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 shutdown_connections 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb010f480 0x7f3bb01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3b9c0b5360 0x7f3b9c0b7820 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3bb01021f0 0x7f3bb019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 --2- 172.21.15.96:0/1549890812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3bb01018d0 0x7f3bb019b4a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.795+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 >> 172.21.15.96:0/1549890812 conn(0x7f3bb00fd740 msgr2=0x7f3bb0110040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.796+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 shutdown_connections 2024-09-10T22:58:03.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:03.796+0000 7f3bb491f640 1 -- 172.21.15.96:0/1549890812 wait complete. 2024-09-10T22:58:05.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:58:05.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:05 smithi096 ceph-mon[30869]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 660 op/s 2024-09-10T22:58:05.559 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:05 smithi121 ceph-mon[35119]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 660 op/s 2024-09-10T22:58:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:05 smithi117 ceph-mon[35081]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 660 op/s 2024-09-10T22:58:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 630.0 s, 1659.4 tps, lat 60.027 ms stddev 75.114 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 -- 172.21.15.96:0/1735639399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 msgr2=0x7fd9041031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 --2- 172.21.15.96:0/1735639399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 0x7fd9041031c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009900 tx=0x7fd8f802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 -- 172.21.15.96:0/1735639399 shutdown_connections 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 --2- 172.21.15.96:0/1735639399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9041087f0 0x7fd904108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 --2- 172.21.15.96:0/1735639399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd904109170 0x7fd90410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 --2- 172.21.15.96:0/1735639399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 0x7fd9041031c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 -- 172.21.15.96:0/1735639399 >> 172.21.15.96:0/1735639399 conn(0x7fd9040fe830 msgr2=0x7fd904100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 -- 172.21.15.96:0/1735639399 shutdown_connections 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.063+0000 7fd90b154640 1 -- 172.21.15.96:0/1735639399 wait complete. 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.064+0000 7fd90b154640 1 Processor -- start 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.064+0000 7fd90b154640 1 -- start start 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.064+0000 7fd90b154640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 0x7fd9041992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.064+0000 7fd90b154640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9041087f0 0x7fd904199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.064+0000 7fd90b154640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd90b154640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd904112e30 con 0x7fd904102de0 2024-09-10T22:58:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd90b154640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd904112cb0 con 0x7fd904109170 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd90b154640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd904112fb0 con 0x7fd9041087f0 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd9096f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd9096f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:38166/0 (socket says 172.21.15.96:38166) 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.065+0000 7fd9096f1640 1 -- 172.21.15.96:0/3052165527 learned_addr learned my addr 172.21.15.96:0/3052165527 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 -- 172.21.15.96:0/3052165527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9041087f0 msgr2=0x7fd904199820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9041087f0 0x7fd904199820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 -- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 msgr2=0x7fd9041992e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 0x7fd9041992e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 -- 172.21.15.96:0/3052165527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90419f390 con 0x7fd904109170 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd9096f1640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00eb10 tx=0x7fd8fc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd901ffb640 1 -- 172.21.15.96:0/3052165527 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc004b20 con 0x7fd904109170 2024-09-10T22:58:06.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.066+0000 7fd901ffb640 1 -- 172.21.15.96:0/3052165527 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc004cc0 con 0x7fd904109170 2024-09-10T22:58:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.067+0000 7fd901ffb640 1 -- 172.21.15.96:0/3052165527 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc010690 con 0x7fd904109170 2024-09-10T22:58:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.067+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90419f680 con 0x7fd904109170 2024-09-10T22:58:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.067+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041a5d40 con 0x7fd904109170 2024-09-10T22:58:06.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd901ffb640 1 -- 172.21.15.96:0/3052165527 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd8fc005000 con 0x7fd904109170 2024-09-10T22:58:06.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 Processor -- start 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 -- start start 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f860f0 0x55e824f864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e824efcea0 con 0x55e824f929a0 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e824eec2d0 con 0x55e824f86a10 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd90d529740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e824ef00e0 con 0x55e824f860f0 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd903fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd908ef0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd903fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f94d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41494/0 (socket says 172.21.15.96:41494) 2024-09-10T22:58:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.070+0000 7fd903fff640 1 -- 172.21.15.96:0/1202008423 learned_addr learned my addr 172.21.15.96:0/1202008423 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd901ffb640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 0x7fd8f40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd903fff640 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f860f0 0x55e824f864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd9096f1640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 0x7fd8f40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 -- 172.21.15.96:0/1202008423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f860f0 msgr2=0x55e824f864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f860f0 0x55e824f864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 -- 172.21.15.96:0/1202008423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 msgr2=0x55e824f94d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f94d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 -- 172.21.15.96:0/1202008423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e824c4a650 con 0x55e824f86a10 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd901ffb640 1 -- 172.21.15.96:0/3052165527 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd8fc0e6210 con 0x7fd904109170 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f92460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005940 tx=0x7fd8f8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4f12c63b7a2044 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd9096f1640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 0x7fd8f40b78f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0049d0 tx=0x7fd8fc0711b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd8eeffd640 1 -- 172.21.15.96:0/1202008423 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8035040 con 0x55e824f86a10 2024-09-10T22:58:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd8eeffd640 1 -- 172.21.15.96:0/1202008423 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f8032da0 con 0x55e824f86a10 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.071+0000 7fd8eeffd640 1 -- 172.21.15.96:0/1202008423 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8003900 con 0x55e824f86a10 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.072+0000 7fd90d529740 1 -- 172.21.15.96:0/1202008423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 msgr2=0x55e824f92460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.072+0000 7fd90d529740 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f92460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005940 tx=0x7fd8f8009520 comp rx=0 tx=0).stop 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 -- 172.21.15.96:0/1202008423 shutdown_connections 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f92460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f94d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 --2- 172.21.15.96:0/1202008423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f860f0 0x55e824f864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 -- 172.21.15.96:0/1202008423 >> 172.21.15.96:0/1202008423 conn(0x55e824f83f20 msgr2=0x55e824f850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 -- 172.21.15.96:0/1202008423 shutdown_connections 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 -- 172.21.15.96:0/1202008423 wait complete. 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 -- start start 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.073+0000 7fd90d529740 1 Processor -- start 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f8f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f8fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e824d4d700 con 0x55e824f860f0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41504/0 (socket says 172.21.15.96:41504) 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e824e87160 con 0x55e824f929a0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e824f955c0 con 0x55e824f86a10 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 -- 172.21.15.96:0/3168059838 learned_addr learned my addr 172.21.15.96:0/3168059838 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd90d529740 10 client.0 _ll_register_callbacks cb 0x55e824ea09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f8fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd908ef0640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f8f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 -- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 msgr2=0x55e824f8f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f8f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 -- 172.21.15.96:0/3168059838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 msgr2=0x55e824f8fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f8fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.074+0000 7fd903fff640 1 -- 172.21.15.96:0/3168059838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e824eb4c90 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd903fff640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0013e80 tx=0x7fd8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9fb0163145e213 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd8e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00130d0 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f8032da0 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd8f0004020 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd8f0009790 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd90d529740 10 client.17730 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.075+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e824fa0f40 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f8032da0 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd8f0029020 con 0x55e824f860f0 2024-09-10T22:58:06.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 10 client.17730 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 10 client.17730 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 20 client.17730 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e824fa2450 0x55e824fa4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 4 client.17730 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e824efedf0 con 0x55e824fa2450 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.076+0000 7fd908ef0640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e824fa2450 0x55e824fa4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd8e3fff640 10 client.17730 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd908ef0640 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e824fa2450 0x55e824fa4870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f635dc16bb486e0 server_cookie=7935da2a9836945c in_seq=0 out_seq=0 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd8e3fff640 1 -- 172.21.15.96:0/3168059838 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e824efedf0 con 0x55e824fa2450 2024-09-10T22:58:06.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd8e3fff640 10 client.17730 handle_command_reply: tid=0 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd90d529740 1 client.17730 shutdown 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.077+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 msgr2=0x55e824f92230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0013e80 tx=0x7fd8f00046c0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 shutdown_connections 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e824fa2450 0x55e824fa4870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd8e3fff640 0 client.17730 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f8fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f8f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 --2- 172.21.15.96:0/3168059838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f860f0 0x55e824f92230 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 >> 172.21.15.96:0/3168059838 conn(0x55e824f83f20 msgr2=0x55e824f89a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 shutdown_connections 2024-09-10T22:58:06.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 1 -- 172.21.15.96:0/3168059838 wait complete. 2024-09-10T22:58:06.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.078+0000 7fd90d529740 20 client.17730 trim_cache size 0 max 16384 2024-09-10T22:58:06.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.095+0000 7fd90d529740 1 Processor -- start 2024-09-10T22:58:06.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.095+0000 7fd90d529740 1 -- start start 2024-09-10T22:58:06.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824fa2450 0x55e824f8e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041ae290 con 0x55e824fa2450 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041afdd0 con 0x55e824f929a0 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd90d529740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041ae960 con 0x55e824f86a10 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51476/0 (socket says 172.21.15.96:51476) 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 -- 172.21.15.96:0/3441800995 learned_addr learned my addr 172.21.15.96:0/3441800995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824fa2450 0x55e824f8e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd908ef0640 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 -- 172.21.15.96:0/3441800995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 msgr2=0x55e824f83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 -- 172.21.15.96:0/3441800995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824fa2450 msgr2=0x55e824f8e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824fa2450 0x55e824f8e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 -- 172.21.15.96:0/3441800995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e824f8ff90 con 0x55e824f86a10 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.096+0000 7fd903fff640 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009760 tx=0x7fd8f0001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2868c33f4a6edf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.097+0000 7fd8ee7fc640 1 -- 172.21.15.96:0/3441800995 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0009b00 con 0x55e824f86a10 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.097+0000 7fd8ee7fc640 1 -- 172.21.15.96:0/3441800995 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f00130d0 con 0x55e824f86a10 2024-09-10T22:58:06.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.097+0000 7fd8ee7fc640 1 -- 172.21.15.96:0/3441800995 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0004280 con 0x55e824f86a10 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- 172.21.15.96:0/3441800995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 msgr2=0x55e824f7ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009760 tx=0x7fd8f0001e70 comp rx=0 tx=0).stop 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- 172.21.15.96:0/3441800995 shutdown_connections 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f929a0 0x55e824f83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824fa2450 0x55e824f8e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 --2- 172.21.15.96:0/3441800995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824f86a10 0x55e824f7ee70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- 172.21.15.96:0/3441800995 >> 172.21.15.96:0/3441800995 conn(0x55e824f83f20 msgr2=0x55e824f8df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- 172.21.15.96:0/3441800995 shutdown_connections 2024-09-10T22:58:06.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- 172.21.15.96:0/3441800995 wait complete. 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 -- start start 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.098+0000 7fd90d529740 1 Processor -- start 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824fa2450 0x55e824f8ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041af700 con 0x55e824f929a0 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041af030 con 0x55e824f86a10 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd90d529740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041ac2b0 con 0x55e824fa2450 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd903fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd903fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39908/0 (socket says 172.21.15.96:39908) 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd903fff640 1 -- 172.21.15.96:0/1055155313 learned_addr learned my addr 172.21.15.96:0/1055155313 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:06.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.099+0000 7fd903fff640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824fa2450 0x55e824f8ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd90d529740 10 client.0 _ll_register_callbacks cb 0x7fd8f0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 -- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824fa2450 msgr2=0x55e824f8ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824fa2450 0x55e824f8ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 -- 172.21.15.96:0/1055155313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 msgr2=0x55e824f9b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f9b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 -- 172.21.15.96:0/1055155313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e824f8c280 con 0x55e824f86a10 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd903fff640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0006010 tx=0x7fd8f0003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7569c75b125905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd8e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0003800 con 0x55e824f86a10 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd8f00050e0 con 0x55e824f86a10 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.100+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f00130d0 con 0x55e824f86a10 2024-09-10T22:58:06.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd8f00052e0 con 0x55e824f86a10 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 10 client.27326 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e824f8d560 con 0x55e824f86a10 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f00130d0 con 0x55e824f86a10 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd8f0029070 con 0x55e824f86a10 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 10 client.27326 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 10 client.27326 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 20 client.27326 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8250106a0 0x55e825012ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 4 client.27326 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.101+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e824efa8a0 con 0x55e8250106a0 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8250106a0 0x55e825012ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd908ef0640 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8250106a0 0x55e825012ac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3085d9af8516704d server_cookie=5efd7e702cfb860e in_seq=0 out_seq=0 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd8e3fff640 10 client.27326 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:06.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd8e3fff640 1 -- 172.21.15.96:0/1055155313 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e824efa8a0 con 0x55e8250106a0 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd8e3fff640 10 client.27326 handle_command_reply: tid=0 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd90d529740 1 client.27326 shutdown 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.102+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 msgr2=0x55e824f9acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0006010 tx=0x7fd8f0003de0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 shutdown_connections 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e8250106a0 0x55e825012ac0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd8e3fff640 0 client.27326 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e824fa2450 0x55e824f8ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e824f929a0 0x55e824f9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 --2- 172.21.15.96:0/1055155313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e824f86a10 0x55e824f9acb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 >> 172.21.15.96:0/1055155313 conn(0x55e824f83f20 msgr2=0x55e824f8e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 shutdown_connections 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 1 -- 172.21.15.96:0/1055155313 wait complete. 2024-09-10T22:58:06.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.103+0000 7fd90d529740 20 client.27326 trim_cache size 0 max 16384 2024-09-10T22:58:06.112 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 msgr2=0x7fd8f40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 0x7fd8f40b78f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0049d0 tx=0x7fd8fc0711b0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 msgr2=0x7fd90419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00eb10 tx=0x7fd8fc00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 shutdown_connections 2024-09-10T22:58:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd904109170 0x7fd90419ebc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8f40b5430 0x7fd8f40b78f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9041087f0 0x7fd904199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 --2- 172.21.15.96:0/3052165527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd904102de0 0x7fd9041992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 >> 172.21.15.96:0/3052165527 conn(0x7fd9040fe830 msgr2=0x7fd9041001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 shutdown_connections 2024-09-10T22:58:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:06.104+0000 7fd90b154640 1 -- 172.21.15.96:0/3052165527 wait complete. 2024-09-10T22:58:06.964 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:58:07.342 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:07.642 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:07 smithi096 ceph-mon[30869]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 684 op/s 2024-09-10T22:58:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:07 smithi121 ceph-mon[35119]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 684 op/s 2024-09-10T22:58:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:07 smithi117 ceph-mon[35081]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 684 op/s 2024-09-10T22:58:07.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 -- 172.21.15.96:0/1085516862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98075540 msgr2=0x7f9f98075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 --2- 172.21.15.96:0/1085516862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98075540 0x7f9f98075920 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c01a910 tx=0x7f9f8c03b420 comp rx=0 tx=0).stop 2024-09-10T22:58:07.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 -- 172.21.15.96:0/1085516862 shutdown_connections 2024-09-10T22:58:07.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 --2- 172.21.15.96:0/1085516862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075e60 0x7f9f981135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 --2- 172.21.15.96:0/1085516862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98113b10 0x7f9f98115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 --2- 172.21.15.96:0/1085516862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98075540 0x7f9f98075920 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 -- 172.21.15.96:0/1085516862 >> 172.21.15.96:0/1085516862 conn(0x7f9f980fe860 msgr2=0x7f9f98100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 -- 172.21.15.96:0/1085516862 shutdown_connections 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.915+0000 7f9f9f187640 1 -- 172.21.15.96:0/1085516862 wait complete. 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9f187640 1 Processor -- start 2024-09-10T22:58:07.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9f187640 1 -- start start 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9f187640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9f187640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98075e60 0x7f9f9810f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9f187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98113b10 0x7f9f9819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.916+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39924/0 (socket says 172.21.15.96:39924) 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9f187640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981095e0 con 0x7f9f98113b10 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9f187640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98109460 con 0x7f9f98075540 2024-09-10T22:58:07.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98109760 con 0x7f9f98075e60 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2083948565 learned_addr learned my addr 172.21.15.96:0/2083948565 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9d724640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98113b10 0x7f9f9819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f97fff640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98075e60 0x7f9f9810f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2083948565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98075e60 msgr2=0x7f9f9810f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98075e60 0x7f9f9810f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98113b10 msgr2=0x7f9f9819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98113b10 0x7f9f9819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.917+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2083948565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9819d0f0 con 0x7f9f98075540 2024-09-10T22:58:07.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c03b940 tx=0x7f9f8c0051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f95ffb640 1 -- 172.21.15.96:0/2083948565 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c04f020 con 0x7f9f98075540 2024-09-10T22:58:07.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f95ffb640 1 -- 172.21.15.96:0/2083948565 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c040e50 con 0x7f9f98075540 2024-09-10T22:58:07.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f95ffb640 1 -- 172.21.15.96:0/2083948565 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c04f020 con 0x7f9f98075540 2024-09-10T22:58:07.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9819d380 con 0x7f9f98075540 2024-09-10T22:58:07.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.918+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a3b10 con 0x7f9f98075540 2024-09-10T22:58:07.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.921+0000 7f9f95ffb640 1 -- 172.21.15.96:0/2083948565 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9f8c004a90 con 0x7f9f98075540 2024-09-10T22:58:07.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.921+0000 7f9fa155c740 1 Processor -- start 2024-09-10T22:58:07.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.921+0000 7f9fa155c740 1 -- start start 2024-09-10T22:58:07.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a328840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55564a292ea0 con 0x55564a31c4d0 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55564a2822d0 con 0x55564a31cdf0 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9fa155c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55564a2860e0 con 0x55564a328d80 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35818/0 (socket says 172.21.15.96:35818) 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a328840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 -- 172.21.15.96:0/3930562197 learned_addr learned my addr 172.21.15.96:0/3930562197 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f97fff640 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f95ffb640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 0x7f9f880b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 -- 172.21.15.96:0/3930562197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 msgr2=0x55564a32b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f9d724640 1 -- 172.21.15.96:0/3930562197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 msgr2=0x55564a328840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.922+0000 7f9f95ffb640 1 -- 172.21.15.96:0/2083948565 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9f8c019030 con 0x7f9f98075540 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.923+0000 7f9f9d724640 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a328840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.923+0000 7f9f9d724640 1 -- 172.21.15.96:0/3930562197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555649fe0650 con 0x55564a31c4d0 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.923+0000 7f9f9d724640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 0x7f9f880b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.923+0000 7f9f9d724640 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011590 tx=0x7f9f90011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce1676a71163338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.924+0000 7f9f82ffd640 1 -- 172.21.15.96:0/3930562197 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90017050 con 0x55564a31c4d0 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.924+0000 7f9f82ffd640 1 -- 172.21.15.96:0/3930562197 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9001bc50 con 0x55564a31c4d0 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.924+0000 7f9f82ffd640 1 -- 172.21.15.96:0/3930562197 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000e3b0 con 0x55564a31c4d0 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- 172.21.15.96:0/3930562197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 msgr2=0x55564a31c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f90011590 tx=0x7f9f90011a50 comp rx=0 tx=0).stop 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9f9d724640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 0x7f9f880b7b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f90022000 tx=0x7f9f90023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- 172.21.15.96:0/3930562197 shutdown_connections 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a328840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 --2- 172.21.15.96:0/3930562197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a31c8b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- 172.21.15.96:0/3930562197 >> 172.21.15.96:0/3930562197 conn(0x55564a31a300 msgr2=0x55564a31b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- 172.21.15.96:0/3930562197 shutdown_connections 2024-09-10T22:58:07.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- 172.21.15.96:0/3930562197 wait complete. 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 -- start start 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.925+0000 7f9fa155c740 1 Processor -- start 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a3286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a326180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55564a0e3700 con 0x55564a328d80 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55564a21d160 con 0x55564a31cdf0 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55564a32ba00 con 0x55564a31c4d0 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9fa155c740 10 client.0 _ll_register_callbacks cb 0x55564a2369e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f9d724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a326180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39930/0 (socket says 172.21.15.96:39930) 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 -- 172.21.15.96:0/391094374 learned_addr learned my addr 172.21.15.96:0/391094374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:07.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a3286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 -- 172.21.15.96:0/391094374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 msgr2=0x55564a3286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a3286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 -- 172.21.15.96:0/391094374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 msgr2=0x55564a326180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a326180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 -- 172.21.15.96:0/391094374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564a24ac90 con 0x55564a31cdf0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f97fff640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009f40 tx=0x7f9f84011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4271763fc44ff79b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.926+0000 7f9f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84012920 con 0x55564a31cdf0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9f84012ac0 con 0x55564a31cdf0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9001bc50 con 0x55564a31cdf0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f84012d00 con 0x55564a31cdf0 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 10 client.27338 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55564a3373c0 con 0x55564a31cdf0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9001bc50 con 0x55564a31cdf0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f8401a3f0 con 0x55564a31cdf0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 10 client.27338 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 10 client.27338 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 20 client.27338 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a338920 0x55564a33ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 4 client.27338 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55564a294df0 con 0x55564a338920 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.927+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a338920 0x55564a33ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.928+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a338920 0x55564a33ad40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58ec8ec9c1740634 server_cookie=4dbc4fbb553d5509 in_seq=0 out_seq=0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.928+0000 7f9f77fff640 10 client.27338 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9f77fff640 1 -- 172.21.15.96:0/391094374 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55564a294df0 con 0x55564a338920 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9f77fff640 10 client.27338 handle_command_reply: tid=0 2024-09-10T22:58:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 client.27338 shutdown 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 msgr2=0x55564a325b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009f40 tx=0x7f9f84011870 comp rx=0 tx=0).stop 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 shutdown_connections 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a338920 0x55564a33ad40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a325b00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a326180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 --2- 172.21.15.96:0/391094374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a3286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9f77fff640 0 client.27338 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 >> 172.21.15.96:0/391094374 conn(0x55564a31a300 msgr2=0x55564a31fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 shutdown_connections 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 1 -- 172.21.15.96:0/391094374 wait complete. 2024-09-10T22:58:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.929+0000 7f9fa155c740 20 client.27338 trim_cache size 0 max 16384 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 Processor -- start 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 -- start start 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a319310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a32a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981ac090 con 0x55564a328d80 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981adbd0 con 0x55564a31cdf0 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9fa155c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981ac760 con 0x55564a31c4d0 2024-09-10T22:58:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f9d724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a32a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51520/0 (socket says 172.21.15.96:51520) 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f9d724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a32a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35844/0 (socket says 172.21.15.96:35844) 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2069581931 learned_addr learned my addr 172.21.15.96:0/2069581931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.946+0000 7f9f97fff640 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a319310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2069581931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 msgr2=0x55564a319310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a319310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2069581931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 msgr2=0x55564a32a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a32a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 -- 172.21.15.96:0/2069581931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564a3282e0 con 0x55564a31c4d0 2024-09-10T22:58:07.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c0d9cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c94e87f5605c882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f827fc640 1 -- 172.21.15.96:0/2069581931 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c04f040 con 0x55564a31c4d0 2024-09-10T22:58:07.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f827fc640 1 -- 172.21.15.96:0/2069581931 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c0dd100 con 0x55564a31c4d0 2024-09-10T22:58:07.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.947+0000 7f9f827fc640 1 -- 172.21.15.96:0/2069581931 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c046040 con 0x55564a31c4d0 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.948+0000 7f9fa155c740 1 -- 172.21.15.96:0/2069581931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 msgr2=0x55564a334c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.948+0000 7f9fa155c740 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c0d9cc0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 -- 172.21.15.96:0/2069581931 shutdown_connections 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a319310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a328d80 0x55564a32a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 --2- 172.21.15.96:0/2069581931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a31c4d0 0x55564a334c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 -- 172.21.15.96:0/2069581931 >> 172.21.15.96:0/2069581931 conn(0x55564a31a300 msgr2=0x55564a318c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 -- 172.21.15.96:0/2069581931 shutdown_connections 2024-09-10T22:58:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 -- 172.21.15.96:0/2069581931 wait complete. 2024-09-10T22:58:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 -- start start 2024-09-10T22:58:07.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.949+0000 7f9fa155c740 1 Processor -- start 2024-09-10T22:58:07.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a32c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9fa155c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9fa155c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981ad500 con 0x55564a31c4d0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9f9cf23640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35854/0 (socket says 172.21.15.96:35854) 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9f9cf23640 1 -- 172.21.15.96:0/1010639051 learned_addr learned my addr 172.21.15.96:0/1010639051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9f97fff640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a32c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9f9d724640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.950+0000 7f9fa155c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981ace30 con 0x55564a31cdf0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981aa070 con 0x55564a328d80 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9fa155c740 10 client.0 _ll_register_callbacks cb 0x7f9f84009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 -- 172.21.15.96:0/1010639051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 msgr2=0x55564a32c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 -- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 msgr2=0x55564a32c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a32c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 -- 172.21.15.96:0/1010639051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564a3a5a10 con 0x55564a31c4d0 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9d724640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f97fff640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a32c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f9cf23640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c114900 tx=0x7f9f8c0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902198e3daf53d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.951+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c046040 con 0x55564a31c4d0 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9f8c0e66f0 con 0x55564a31c4d0 2024-09-10T22:58:07.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f8c0dd100 con 0x55564a31c4d0 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f8c04f040 con 0x55564a31c4d0 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 10 client.17751 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55564a3a6d10 con 0x55564a31c4d0 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c0dd100 con 0x55564a31c4d0 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9f8c040950 con 0x55564a31c4d0 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 10 client.17751 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 20 client.17751 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a3a7170 0x55564a3a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 4 client.17751 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55564a2908a0 con 0x55564a3a7170 2024-09-10T22:58:07.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.952+0000 7f9f97fff640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a3a7170 0x55564a3a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9f77fff640 10 client.17751 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9f97fff640 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a3a7170 0x55564a3a9590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88d3c8506cbb70aa server_cookie=b566319d3af0b74a in_seq=0 out_seq=0 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9f77fff640 1 -- 172.21.15.96:0/1010639051 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55564a2908a0 con 0x55564a3a7170 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9f77fff640 10 client.17751 handle_command_reply: tid=0 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 client.17751 shutdown 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 msgr2=0x55564a32bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c114900 tx=0x7f9f8c0053e0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 shutdown_connections 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55564a3a7170 0x55564a3a9590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55564a31cdf0 0x55564a32c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55564a328d80 0x55564a32c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 --2- 172.21.15.96:0/1010639051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55564a31c4d0 0x55564a32bdf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 >> 172.21.15.96:0/1010639051 conn(0x55564a31a300 msgr2=0x55564a319420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9f77fff640 0 client.17751 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:07.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 shutdown_connections 2024-09-10T22:58:07.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.953+0000 7f9fa155c740 1 -- 172.21.15.96:0/1010639051 wait complete. 2024-09-10T22:58:07.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9fa155c740 20 client.17751 trim_cache size 0 max 16384 2024-09-10T22:58:07.960 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 msgr2=0x7f9f880b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 0x7f9f880b7b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f90022000 tx=0x7f9f90023000 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 msgr2=0x7f9f9810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c03b940 tx=0x7f9f8c0051f0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 shutdown_connections 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9f880b56d0 0x7f9f880b7b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f98113b10 0x7f9f9819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98075e60 0x7f9f9810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 --2- 172.21.15.96:0/2083948565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f98075540 0x7f9f9810f330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:07.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 >> 172.21.15.96:0/2083948565 conn(0x7f9f980fe860 msgr2=0x7f9f980ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:07.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 shutdown_connections 2024-09-10T22:58:07.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:07.954+0000 7f9f9f187640 1 -- 172.21.15.96:0/2083948565 wait complete. 2024-09-10T22:58:08.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:58:09.237 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:09 smithi096 ceph-mon[30869]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 MiB/s wr, 600 op/s 2024-09-10T22:58:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:09 smithi121 ceph-mon[35119]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 MiB/s wr, 600 op/s 2024-09-10T22:58:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:09 smithi117 ceph-mon[35081]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 MiB/s wr, 600 op/s 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3053588949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8103380 msgr2=0x7f9bd8109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3053588949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8103380 0x7f9bd8109d00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd0032d90 comp rx=0 tx=0).stop 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3053588949 shutdown_connections 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3053588949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8103380 0x7f9bd8109d00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3053588949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd81029c0 0x7f9bd8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3053588949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd8101870 0x7f9bd8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3053588949 >> 172.21.15.96:0/3053588949 conn(0x7f9bd80fd130 msgr2=0x7f9bd80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3053588949 shutdown_connections 2024-09-10T22:58:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.886+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3053588949 wait complete. 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 Processor -- start 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 -- start start 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 0x7f9bd8196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd8103380 0x7f9bd8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd810c400 con 0x7f9bd8103380 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd810c280 con 0x7f9bd8101870 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.887+0000 7f9bdd2e6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd810c580 con 0x7f9bd81029c0 2024-09-10T22:58:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 0x7f9bd8196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51548/0 (socket says 172.21.15.96:51548) 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 -- 172.21.15.96:0/3912046549 learned_addr learned my addr 172.21.15.96:0/3912046549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 0x7f9bd8196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39952/0 (socket says 172.21.15.96:39952) 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd8103380 0x7f9bd8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 -- 172.21.15.96:0/3912046549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 msgr2=0x7f9bd8196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 0x7f9bd8196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 -- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd8103380 msgr2=0x7f9bd8197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd8103380 0x7f9bd8197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 -- 172.21.15.96:0/3912046549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd8198680 con 0x7f9bd81029c0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d950 tx=0x7f9bc000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd4ff9640 1 -- 172.21.15.96:0/3912046549 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd81029c0 2024-09-10T22:58:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd4ff9640 1 -- 172.21.15.96:0/3912046549 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0009bb0 con 0x7f9bd81029c0 2024-09-10T22:58:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd8198970 con 0x7f9bd81029c0 2024-09-10T22:58:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bd4ff9640 1 -- 172.21.15.96:0/3912046549 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000f620 con 0x7f9bd81029c0 2024-09-10T22:58:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.888+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd819f040 con 0x7f9bd81029c0 2024-09-10T22:58:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 Processor -- start 2024-09-10T22:58:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 -- start start 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 0x56052dbc0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56052db306f0 con 0x56052dbc1190 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56052db1fb20 con 0x56052dbbe860 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9be00ba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56052db23930 con 0x56052dbc3bd0 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.893+0000 7f9bd4ff9640 1 -- 172.21.15.96:0/3912046549 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9bc0010460 con 0x7f9bd81029c0 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 0x56052dbc0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35876/0 (socket says 172.21.15.96:35876) 2024-09-10T22:58:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/966190357 learned_addr learned my addr 172.21.15.96:0/966190357 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bcffff640 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/966190357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 msgr2=0x56052dbc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/966190357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 msgr2=0x56052dbc0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 0x56052dbc0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/966190357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052dae5530 con 0x56052dbc1190 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 0x56052dbc0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ad10 tx=0x7f9bc800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835471322c58e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/966190357 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8016620 con 0x56052dbc1190 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/966190357 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc801b070 con 0x56052dbc1190 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.894+0000 7f9bcd7fa640 1 -- 172.21.15.96:0/966190357 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8015a50 con 0x56052dbc1190 2024-09-10T22:58:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9bd4ff9640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 0x7f9bc40b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 0x7f9bc40b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9bd4ff9640 1 -- 172.21.15.96:0/3912046549 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bc00e55d0 con 0x7f9bd81029c0 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 -- 172.21.15.96:0/966190357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 msgr2=0x56052dbc3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ad10 tx=0x7f9bc800a380 comp rx=0 tx=0).stop 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 -- 172.21.15.96:0/966190357 shutdown_connections 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc3580 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 --2- 172.21.15.96:0/966190357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbbe860 0x56052dbc0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.895+0000 7f9be00ba740 1 -- 172.21.15.96:0/966190357 >> 172.21.15.96:0/966190357 conn(0x56052dbb99b0 msgr2=0x56052dbba740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.896+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 0x7f9bc40b7860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009bf0 tx=0x7f9bd0035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 -- 172.21.15.96:0/966190357 shutdown_connections 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 -- 172.21.15.96:0/966190357 wait complete. 2024-09-10T22:58:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 -- start start 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 Processor -- start 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbbe860 0x56052dbc7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.897+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbc91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9be00ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56052da57ab0 con 0x56052dbc1190 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9be00ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56052dac5730 con 0x56052dbc3bd0 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9be00ba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56052dbc6bb0 con 0x56052dbbe860 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9be00ba740 10 client.0 _ll_register_callbacks cb 0x56052db196b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35888/0 (socket says 172.21.15.96:35888) 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 -- 172.21.15.96:0/3583375709 learned_addr learned my addr 172.21.15.96:0/3583375709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbc91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbbe860 0x56052dbc7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 -- 172.21.15.96:0/3583375709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbbe860 msgr2=0x56052dbc7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbbe860 0x56052dbc7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 -- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 msgr2=0x56052dbc91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbc91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bcffff640 1 -- 172.21.15.96:0/3583375709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052d881790 con 0x56052dbc1190 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.898+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bcffff640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00a84b0 tx=0x7f9bc0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209d156e4f60208c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0020070 con 0x56052dbc1190 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc801b070 con 0x56052dbc1190 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bc0010780 con 0x56052dbc1190 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9bc00ab650 con 0x56052dbc1190 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9be00ba740 10 client.17763 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56052d8fa620 con 0x56052dbc1190 2024-09-10T22:58:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.899+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc801b070 con 0x56052dbc1190 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc00e50d0 con 0x56052dbc1190 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 10 client.17763 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 20 client.17763 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dbd3c70 0x56052dbd6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 4 client.17763 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052db32640 con 0x56052dbd3c70 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dbd3c70 0x56052dbd6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9bbe7fc640 10 client.17763 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.900+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dbd3c70 0x56052dbd6110 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56673a4c4cad5ed7 server_cookie=e82f3f1fcaf91de7 in_seq=0 out_seq=0 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/3583375709 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56052db32640 con 0x56052dbd3c70 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9bbe7fc640 10 client.17763 handle_command_reply: tid=0 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 client.17763 shutdown 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 msgr2=0x56052dbc85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00a84b0 tx=0x7f9bc0009f60 comp rx=0 tx=0).stop 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 shutdown_connections 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dbd3c70 0x56052dbd6110 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbc91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9bbe7fc640 0 client.17763 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbc1190 0x56052dbc85f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 --2- 172.21.15.96:0/3583375709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbbe860 0x56052dbc7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 >> 172.21.15.96:0/3583375709 conn(0x56052dbb99b0 msgr2=0x56052dbb8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 shutdown_connections 2024-09-10T22:58:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 1 -- 172.21.15.96:0/3583375709 wait complete. 2024-09-10T22:58:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.901+0000 7f9be00ba740 20 client.17763 trim_cache size 0 max 16384 2024-09-10T22:58:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.920+0000 7f9be00ba740 1 Processor -- start 2024-09-10T22:58:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.920+0000 7f9be00ba740 1 -- start start 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc1190 0x56052dbc44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbbb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a7c80 con 0x56052dbbe860 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a97c0 con 0x56052dbc3bd0 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9be00ba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a8350 con 0x56052dbc1190 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35892/0 (socket says 172.21.15.96:35892) 2024-09-10T22:58:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/262807628 learned_addr learned my addr 172.21.15.96:0/262807628 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.921+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbbb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/262807628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc1190 msgr2=0x56052dbc44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc1190 0x56052dbc44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/262807628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 msgr2=0x56052dbbb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbbb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/262807628 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052dbc9530 con 0x56052dbbe860 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.922+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ad10 tx=0x7f9bc800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e9b9593d93a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.923+0000 7f9bccff9640 1 -- 172.21.15.96:0/262807628 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc802e020 con 0x56052dbbe860 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.923+0000 7f9bccff9640 1 -- 172.21.15.96:0/262807628 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc801b040 con 0x56052dbbe860 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.923+0000 7f9bccff9640 1 -- 172.21.15.96:0/262807628 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8016460 con 0x56052dbbe860 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.923+0000 7f9be00ba740 1 -- 172.21.15.96:0/262807628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 msgr2=0x56052dbc3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.923+0000 7f9be00ba740 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ad10 tx=0x7f9bc800a970 comp rx=0 tx=0).stop 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 -- 172.21.15.96:0/262807628 shutdown_connections 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc3bd0 0x56052dbbb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc1190 0x56052dbc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 --2- 172.21.15.96:0/262807628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbc3fb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 -- 172.21.15.96:0/262807628 >> 172.21.15.96:0/262807628 conn(0x56052dbb99b0 msgr2=0x56052dbbdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 -- 172.21.15.96:0/262807628 shutdown_connections 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 -- 172.21.15.96:0/262807628 wait complete. 2024-09-10T22:58:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 -- start start 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.924+0000 7f9be00ba740 1 Processor -- start 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc1190 0x56052dbcea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbcf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a90f0 con 0x56052dbbe860 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a8a20 con 0x56052dbc1190 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd81a5c60 con 0x56052dbc3bd0 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 10 client.0 _ll_register_callbacks cb 0x7f9bc000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35898/0 (socket says 172.21.15.96:35898) 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/1448793655 learned_addr learned my addr 172.21.15.96:0/1448793655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bcffff640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc1190 0x56052dbcea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd77fe640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbcf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/1448793655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 msgr2=0x56052dbcf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbcf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc1190 msgr2=0x56052dbcea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc1190 0x56052dbcea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 -- 172.21.15.96:0/1448793655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052dbcd450 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bd6ffd640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800a810 tx=0x7f9bc8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7d8db4c5d0c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80035f0 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9bc8003790 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.925+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc801b040 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9bc803c430 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 10 client.17769 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56052dc40ec0 con 0x56052dbbe860 2024-09-10T22:58:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc801b040 con 0x56052dbbe860 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc8033420 con 0x56052dbbe860 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 10 client.17769 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 20 client.17769 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dc41750 0x56052dc43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 4 client.17769 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56052db2e0f0 con 0x56052dc41750 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.926+0000 7f9bcffff640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dc41750 0x56052dc43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.927+0000 7f9bbe7fc640 10 client.17769 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:09.927+0000 7f9bcffff640 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dc41750 0x56052dc43b70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab76082af655c56 server_cookie=15cfecefb45d7e3 in_seq=0 out_seq=0 2024-09-10T22:58:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.012+0000 7f9bbe7fc640 1 -- 172.21.15.96:0/1448793655 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56052db2e0f0 con 0x56052dc41750 2024-09-10T22:58:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.012+0000 7f9bbe7fc640 10 client.17769 handle_command_reply: tid=0 2024-09-10T22:58:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.012+0000 7f9be00ba740 1 client.17769 shutdown 2024-09-10T22:58:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 msgr2=0x56052dbcfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800a810 tx=0x7f9bc8005850 comp rx=0 tx=0).stop 2024-09-10T22:58:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 shutdown_connections 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56052dc41750 0x56052dc43b70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9bbe7fc640 0 client.17769 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56052dbc1190 0x56052dbcea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56052dbc3bd0 0x56052dbcf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 --2- 172.21.15.96:0/1448793655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56052dbbe860 0x56052dbcfc30 secure :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800a810 tx=0x7f9bc8005850 comp rx=0 tx=0).stop 2024-09-10T22:58:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.014+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 >> 172.21.15.96:0/1448793655 conn(0x56052dbb99b0 msgr2=0x56052dbbdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.017+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 shutdown_connections 2024-09-10T22:58:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.017+0000 7f9be00ba740 1 -- 172.21.15.96:0/1448793655 wait complete. 2024-09-10T22:58:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:58:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:58:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:58:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:58:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.018+0000 7f9be00ba740 20 client.17769 trim_cache size 0 max 16384 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 msgr2=0x7f9bc40b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 0x7f9bc40b7860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009bf0 tx=0x7f9bd0035040 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 msgr2=0x7f9bd8197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000d950 tx=0x7f9bc000de10 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 shutdown_connections 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9bc40b53a0 0x7f9bc40b7860 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bd8103380 0x7f9bd8197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd81029c0 0x7f9bd8197190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 --2- 172.21.15.96:0/3912046549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bd8101870 0x7f9bd8196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.020+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 >> 172.21.15.96:0/3912046549 conn(0x7f9bd80fd130 msgr2=0x7f9bd8105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.023+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 shutdown_connections 2024-09-10T22:58:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:10.023+0000 7f9bdd2e6640 1 -- 172.21.15.96:0/3912046549 wait complete. 2024-09-10T22:58:10.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 -- 172.21.15.96:0/3710618606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc1087b0 msgr2=0x7f4dcc108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/3710618606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc108bf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009900 tx=0x7f4dbc02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:10.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 -- 172.21.15.96:0/3710618606 shutdown_connections 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/3710618606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 0x7f4dcc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/3710618606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc108bf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/3710618606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc102da0 0x7f4dcc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 -- 172.21.15.96:0/3710618606 >> 172.21.15.96:0/3710618606 conn(0x7f4dcc0fe850 msgr2=0x7f4dcc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 -- 172.21.15.96:0/3710618606 shutdown_connections 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.074+0000 7f4dd3bea640 1 -- 172.21.15.96:0/3710618606 wait complete. 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 Processor -- start 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 -- start start 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 0x7f4dcc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112db0 con 0x7f4dcc102da0 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112c30 con 0x7f4dcc1087b0 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd3bea640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112f30 con 0x7f4dcc109130 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd1986640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd1986640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35910/0 (socket says 172.21.15.96:35910) 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd1986640 1 -- 172.21.15.96:0/2282128922 learned_addr learned my addr 172.21.15.96:0/2282128922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd1185640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.075+0000 7f4dd2187640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 0x7f4dcc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 -- 172.21.15.96:0/2282128922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 msgr2=0x7f4dcc19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 0x7f4dcc19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 -- 172.21.15.96:0/2282128922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc1087b0 msgr2=0x7f4dcc199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 -- 172.21.15.96:0/2282128922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc19f2f0 con 0x7f4dcc102da0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e9f0 tx=0x7f4dc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dc2ffd640 1 -- 172.21.15.96:0/2282128922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800cd80 con 0x7f4dcc102da0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dc2ffd640 1 -- 172.21.15.96:0/2282128922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80108e0 con 0x7f4dcc102da0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.076+0000 7f4dc2ffd640 1 -- 172.21.15.96:0/2282128922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8018980 con 0x7f4dcc102da0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.077+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc19f5e0 con 0x7f4dcc102da0 2024-09-10T22:58:10.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.077+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a5ca0 con 0x7f4dcc102da0 2024-09-10T22:58:10.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.080+0000 7f4dc2ffd640 1 -- 172.21.15.96:0/2282128922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4dc8010430 con 0x7f4dcc102da0 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.080+0000 7f4dd5fbf740 1 Processor -- start 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.080+0000 7f4dd5fbf740 1 -- start start 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf904d0 0x55858cf908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 0x55858cf90df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55858cf06ea0 con 0x55858cfa5430 2024-09-10T22:58:10.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55858cef62d0 con 0x55858cf9a830 2024-09-10T22:58:10.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55858cefa0e0 con 0x55858cf904d0 2024-09-10T22:58:10.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd2187640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf904d0 0x55858cf908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1986640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 0x55858cf90df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35924/0 (socket says 172.21.15.96:35924) 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 -- 172.21.15.96:0/3695793166 learned_addr learned my addr 172.21.15.96:0/3695793166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 -- 172.21.15.96:0/3695793166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf904d0 msgr2=0x55858cf908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf904d0 0x55858cf908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 -- 172.21.15.96:0/3695793166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 msgr2=0x55858cf90df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 0x55858cf90df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.081+0000 7f4dd1185640 1 -- 172.21.15.96:0/3695793166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858cc54650 con 0x55858cfa5430 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4dd1986640 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 0x55858cf90df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009900 tx=0x7f4dbc032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0745c628d03343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4dc2ffd640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 0x7f4db80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4db7fff640 1 -- 172.21.15.96:0/3695793166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc040070 con 0x55858cfa5430 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4dd2187640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 0x7f4db80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4db7fff640 1 -- 172.21.15.96:0/3695793166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc004030 con 0x55858cfa5430 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4db7fff640 1 -- 172.21.15.96:0/3695793166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc035040 con 0x55858cfa5430 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.082+0000 7f4dc2ffd640 1 -- 172.21.15.96:0/2282128922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4dc80e6120 con 0x7f4dcc102da0 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3695793166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 msgr2=0x55858cfa7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009900 tx=0x7f4dbc032b20 comp rx=0 tx=0).stop 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd2187640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 0x7f4db80b78a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005fd0 tx=0x7f4dc4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3695793166 shutdown_connections 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf9a830 0x55858cf90df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cfa7820 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3695793166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf904d0 0x55858cf908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3695793166 >> 172.21.15.96:0/3695793166 conn(0x55858cf8e300 msgr2=0x55858cf8f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3695793166 shutdown_connections 2024-09-10T22:58:10.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3695793166 wait complete. 2024-09-10T22:58:10.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 -- start start 2024-09-10T22:58:10.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.083+0000 7f4dd5fbf740 1 Processor -- start 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf97550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf97b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55858cd57700 con 0x55858cfa5430 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55858ce91160 con 0x55858cf904d0 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55858cfa81c0 con 0x55858cf9a830 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd5fbf740 10 client.0 _ll_register_callbacks cb 0x55858ceaa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40034/0 (socket says 172.21.15.96:40034) 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd2187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf97b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 -- 172.21.15.96:0/3639589548 learned_addr learned my addr 172.21.15.96:0/3639589548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf97550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 -- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 msgr2=0x55858cf97550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf97550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 -- 172.21.15.96:0/3639589548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 msgr2=0x55858cf97b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf97b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.084+0000 7f4dd1986640 1 -- 172.21.15.96:0/3639589548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858cebec90 con 0x55858cf904d0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd1986640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80ac870 tx=0x7f4dc80a8f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724bc01dcd679c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80ac6b0 con 0x55858cf904d0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc004030 con 0x55858cf904d0 2024-09-10T22:58:10.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4dc80e82d0 con 0x55858cf904d0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4dc80ac050 con 0x55858cf904d0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 10 client.27374 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55858cfb3b30 con 0x55858cf904d0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc004030 con 0x55858cf904d0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4dc8018b20 con 0x55858cf904d0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 10 client.27374 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 10 client.27374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 20 client.27374 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858cfb5310 0x55858cfb7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 4 client.27374 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858cf08df0 con 0x55858cfb5310 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.085+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858cfb5310 0x55858cfb7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.087+0000 7f4dd1185640 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858cfb5310 0x55858cfb7700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b73d94297aa0d9a5 server_cookie=6fbecb976f571b29 in_seq=0 out_seq=0 2024-09-10T22:58:10.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.087+0000 7f4db57fa640 10 client.27374 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.087+0000 7f4db57fa640 1 -- 172.21.15.96:0/3639589548 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55858cf08df0 con 0x55858cfb5310 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.087+0000 7f4db57fa640 10 client.27374 handle_command_reply: tid=0 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 client.27374 shutdown 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 msgr2=0x55858cf9a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80ac870 tx=0x7f4dc80a8f30 comp rx=0 tx=0).stop 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 shutdown_connections 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf97b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf97550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858cfb5310 0x55858cfb7700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/3639589548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf9a100 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 >> 172.21.15.96:0/3639589548 conn(0x55858cf8e300 msgr2=0x55858cf8cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4db57fa640 0 client.27374 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:58:10.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 shutdown_connections 2024-09-10T22:58:10.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/3639589548 wait complete. 2024-09-10T22:58:10.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.088+0000 7f4dd5fbf740 20 client.27374 trim_cache size 0 max 16384 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 Processor -- start 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 -- start start 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf95dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf8cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ac060 con 0x55858cfa5430 2024-09-10T22:58:10.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1adba0 con 0x55858cf904d0 2024-09-10T22:58:10.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ac730 con 0x55858cf9a830 2024-09-10T22:58:10.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd2187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf8cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd2187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf8cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35946/0 (socket says 172.21.15.96:35946) 2024-09-10T22:58:10.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd2187640 1 -- 172.21.15.96:0/437794512 learned_addr learned my addr 172.21.15.96:0/437794512 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:10.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.104+0000 7f4dd1986640 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf95dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 -- 172.21.15.96:0/437794512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 msgr2=0x55858cf96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 -- 172.21.15.96:0/437794512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 msgr2=0x55858cf8cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf8cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 -- 172.21.15.96:0/437794512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858cfa5810 con 0x55858cf904d0 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4dd1986640 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf95dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c6a0 tx=0x7f4dc8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426922127ed49b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4db77fe640 1 -- 172.21.15.96:0/437794512 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80ac230 con 0x55858cf904d0 2024-09-10T22:58:10.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4db77fe640 1 -- 172.21.15.96:0/437794512 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80ac3f0 con 0x55858cf904d0 2024-09-10T22:58:10.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.105+0000 7f4db77fe640 1 -- 172.21.15.96:0/437794512 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80e93a0 con 0x55858cf904d0 2024-09-10T22:58:10.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.106+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/437794512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 msgr2=0x55858cf95dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.106+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf95dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c6a0 tx=0x7f4dc8000bc0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/437794512 shutdown_connections 2024-09-10T22:58:10.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cfa5430 0x55858cf8cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cf9a830 0x55858cf96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/437794512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cf95dd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/437794512 >> 172.21.15.96:0/437794512 conn(0x55858cf8e300 msgr2=0x55858cfa5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/437794512 shutdown_connections 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/437794512 wait complete. 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 -- start start 2024-09-10T22:58:10.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.107+0000 7f4dd5fbf740 1 Processor -- start 2024-09-10T22:58:10.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cfa9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cfa5430 0x55858d01af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ad4d0 con 0x55858cf9a830 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ace00 con 0x55858cf904d0 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1aa040 con 0x55858cfa5430 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd5fbf740 10 client.0 _ll_register_callbacks cb 0x7f4dc8010750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd1185640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd1185640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35950/0 (socket says 172.21.15.96:35950) 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.108+0000 7f4dd1185640 1 -- 172.21.15.96:0/2111661045 learned_addr learned my addr 172.21.15.96:0/2111661045 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd2187640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cfa5430 0x55858d01af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cfa9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 -- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cfa5430 msgr2=0x55858d01af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cfa5430 0x55858d01af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 -- 172.21.15.96:0/2111661045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 msgr2=0x55858cfa9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cfa9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 -- 172.21.15.96:0/2111661045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858d01b640 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cfa9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4dd1185640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009900 tx=0x7f4dbc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37acb04b7660a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4db57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.109+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc05b060 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4dbc0314a0 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc80ac3f0 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc035040 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 10 client.17787 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55858d0229f0 con 0x55858cf9a830 2024-09-10T22:58:10.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80ac3f0 con 0x55858cf9a830 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4dbc060070 con 0x55858cf9a830 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 10 client.17787 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 10 client.17787 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 20 client.17787 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858d022d60 0x55858d025150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 4 client.17787 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55858cf048a0 con 0x55858d022d60 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.110+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858d022d60 0x55858d025150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.112+0000 7f4db57fa640 10 client.17787 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:58:10.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.112+0000 7f4dd1986640 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858d022d60 0x55858d025150 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acd94ee7aa3403e8 server_cookie=490b7fe87fc8443b in_seq=0 out_seq=0 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4db57fa640 1 -- 172.21.15.96:0/2111661045 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55858cf048a0 con 0x55858d022d60 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4db57fa640 10 client.17787 handle_command_reply: tid=0 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 client.17787 shutdown 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 msgr2=0x55858cfaa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009900 tx=0x7f4dbc031400 comp rx=0 tx=0).stop 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 shutdown_connections 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55858cfa5430 0x55858d01af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55858cf9a830 0x55858cfaa290 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55858d022d60 0x55858d025150 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4db57fa640 0 client.17787 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 --2- 172.21.15.96:0/2111661045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55858cf904d0 0x55858cfa9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 >> 172.21.15.96:0/2111661045 conn(0x55858cf8e300 msgr2=0x55858cf95d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 shutdown_connections 2024-09-10T22:58:10.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 1 -- 172.21.15.96:0/2111661045 wait complete. 2024-09-10T22:58:10.121 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.113+0000 7f4dd5fbf740 20 client.17787 trim_cache size 0 max 16384 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 msgr2=0x7f4db80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 0x7f4db80b78a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005fd0 tx=0x7f4dc4009500 comp rx=0 tx=0).stop 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 msgr2=0x7f4dcc199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e9f0 tx=0x7f4dc800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 shutdown_connections 2024-09-10T22:58:10.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dcc1087b0 0x7f4dcc199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4db80b53e0 0x7f4db80b78a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dcc109130 0x7f4dcc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 --2- 172.21.15.96:0/2282128922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dcc102da0 0x7f4dcc199240 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 >> 172.21.15.96:0/2282128922 conn(0x7f4dcc0fe850 msgr2=0x7f4dcc100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 shutdown_connections 2024-09-10T22:58:10.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:10.114+0000 7f4dd3bea640 1 -- 172.21.15.96:0/2282128922 wait complete. 2024-09-10T22:58:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 635.0 s, 1988.0 tps, lat 50.761 ms stddev 58.857 2024-09-10T22:58:11.183 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:58:11.504 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:11 smithi096 ceph-mon[30869]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 748 op/s 2024-09-10T22:58:11.567 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:11 smithi121 ceph-mon[35119]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 748 op/s 2024-09-10T22:58:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:11 smithi117 ceph-mon[35081]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 748 op/s 2024-09-10T22:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.116+0000 7ff241663640 1 -- 172.21.15.96:0/2953845597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c106d40 msgr2=0x7ff23c107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.116+0000 7ff241663640 1 --2- 172.21.15.96:0/2953845597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c106d40 0x7ff23c107190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff234009ca0 tx=0x7ff23402a8f0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.119+0000 7ff241663640 1 -- 172.21.15.96:0/2953845597 shutdown_connections 2024-09-10T22:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.119+0000 7ff241663640 1 --2- 172.21.15.96:0/2953845597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c106d40 0x7ff23c107190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.119+0000 7ff241663640 1 --2- 172.21.15.96:0/2953845597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.119+0000 7ff241663640 1 --2- 172.21.15.96:0/2953845597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff23c101900 0x7ff23c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.119+0000 7ff241663640 1 -- 172.21.15.96:0/2953845597 >> 172.21.15.96:0/2953845597 conn(0x7ff23c0fd5b0 msgr2=0x7ff23c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.123+0000 7ff241663640 1 -- 172.21.15.96:0/2953845597 shutdown_connections 2024-09-10T22:58:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.125+0000 7ff241663640 1 -- 172.21.15.96:0/2953845597 wait complete. 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.125+0000 7ff241663640 1 Processor -- start 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 -- start start 2024-09-10T22:58:12.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 0x7ff23c0749c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff23c106d40 0x7ff23c06ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c06d470 con 0x7ff23c102220 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c06d2f0 con 0x7ff23c101900 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.126+0000 7ff241663640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c06d5f0 con 0x7ff23c106d40 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c074f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35968/0 (socket says 172.21.15.96:35968) 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23b7fe640 1 -- 172.21.15.96:0/2031262096 learned_addr learned my addr 172.21.15.96:0/2031262096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 0x7ff23c0749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 -- 172.21.15.96:0/2031262096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff23c106d40 msgr2=0x7ff23c06ece0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff23c106d40 0x7ff23c06ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 -- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 msgr2=0x7ff23c074f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c074f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.127+0000 7ff23bfff640 1 -- 172.21.15.96:0/2031262096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c06f5a0 con 0x7ff23c101900 2024-09-10T22:58:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.128+0000 7ff23bfff640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 0x7ff23c0749c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009fd0 tx=0x7ff22c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.131+0000 7ff2397fa640 1 -- 172.21.15.96:0/2031262096 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c015020 con 0x7ff23c101900 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.132+0000 7ff2397fa640 1 -- 172.21.15.96:0/2031262096 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22c011ce0 con 0x7ff23c101900 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.132+0000 7ff2397fa640 1 -- 172.21.15.96:0/2031262096 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c010e00 con 0x7ff23c101900 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.133+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c06f890 con 0x7ff23c101900 2024-09-10T22:58:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.133+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1b48f0 con 0x7ff23c101900 2024-09-10T22:58:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.136+0000 7ff244c9a740 1 Processor -- start 2024-09-10T22:58:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 -- start start 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fad6a0 0x558a35fada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fadfc0 0x558a35fb9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a35f23ea0 con 0x558a35fad6a0 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a35f132d0 con 0x558a35fadfc0 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.137+0000 7ff244c9a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558a35f170e0 con 0x558a35fb9f50 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.138+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.138+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51648/0 (socket says 172.21.15.96:51648) 2024-09-10T22:58:12.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.138+0000 7ff23b7fe640 1 -- 172.21.15.96:0/2639424357 learned_addr learned my addr 172.21.15.96:0/2639424357 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.139+0000 7ff2397fa640 1 -- 172.21.15.96:0/2031262096 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff22c022060 con 0x7ff23c101900 2024-09-10T22:58:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff240e62640 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fad6a0 0x558a35fada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff23b7fe640 1 -- 172.21.15.96:0/2639424357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fadfc0 msgr2=0x558a35fb9a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:58:12.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fadfc0 0x558a35fb9a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff23b7fe640 1 -- 172.21.15.96:0/2639424357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fad6a0 msgr2=0x558a35fada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fad6a0 0x558a35fada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.140+0000 7ff23b7fe640 1 -- 172.21.15.96:0/2639424357 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a35c71650 con 0x558a35fb9f50 2024-09-10T22:58:12.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.141+0000 7ff2397fa640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 0x7ff2240b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.141+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff230009cf0 tx=0x7ff230011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6748a18805dd0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.141+0000 7ff240e62640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 0x7ff2240b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.141+0000 7ff2227fc640 1 -- 172.21.15.96:0/2639424357 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230004220 con 0x558a35fb9f50 2024-09-10T22:58:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.141+0000 7ff2227fc640 1 -- 172.21.15.96:0/2639424357 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2300043c0 con 0x558a35fb9f50 2024-09-10T22:58:12.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.142+0000 7ff2397fa640 1 -- 172.21.15.96:0/2031262096 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff22c0e4f90 con 0x7ff23c101900 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.142+0000 7ff244c9a740 1 -- 172.21.15.96:0/2639424357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 msgr2=0x558a35fbc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.142+0000 7ff244c9a740 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff230009cf0 tx=0x7ff230011810 comp rx=0 tx=0).stop 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff240e62640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 0x7ff2240b79f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff234009bf0 tx=0x7ff234005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 -- 172.21.15.96:0/2639424357 shutdown_connections 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fadfc0 0x558a35fb9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a35fbc340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 --2- 172.21.15.96:0/2639424357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fad6a0 0x558a35fada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 -- 172.21.15.96:0/2639424357 >> 172.21.15.96:0/2639424357 conn(0x558a35fab4d0 msgr2=0x558a35fac670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 -- 172.21.15.96:0/2639424357 shutdown_connections 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 -- 172.21.15.96:0/2639424357 wait complete. 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.143+0000 7ff244c9a740 1 -- start start 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.144+0000 7ff244c9a740 1 Processor -- start 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a35d74700 con 0x558a35fadfc0 2024-09-10T22:58:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a35eae160 con 0x558a35fb9f50 2024-09-10T22:58:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558a35fcdc30 con 0x558a35fad6a0 2024-09-10T22:58:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff244c9a740 10 client.0 _ll_register_callbacks cb 0x558a35ec79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:12.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51660/0 (socket says 172.21.15.96:51660) 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.145+0000 7ff23bfff640 1 -- 172.21.15.96:0/743724212 learned_addr learned my addr 172.21.15.96:0/743724212 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff240e62640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 -- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 msgr2=0x558a35fb7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 -- 172.21.15.96:0/743724212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 msgr2=0x558a35fb6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 -- 172.21.15.96:0/743724212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a35edbc90 con 0x558a35fad6a0 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff23bfff640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0ab4f0 tx=0x7ff22c0ab5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49b752b41e7e786 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff21b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0ab300 con 0x558a35fad6a0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.146+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff22c011830 con 0x558a35fad6a0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.147+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2300043c0 con 0x558a35fad6a0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.147+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff22c0abef0 con 0x558a35fad6a0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.147+0000 7ff244c9a740 10 client.27451 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.147+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a35fd9590 con 0x558a35fad6a0 2024-09-10T22:58:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.147+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2300043c0 con 0x558a35fad6a0 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff22c0aaa50 con 0x558a35fad6a0 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 10 client.27451 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 10 client.27451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 20 client.27451 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a35fda9f0 0x558a35fdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 4 client.27451 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.148+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a35f25df0 con 0x558a35fda9f0 2024-09-10T22:58:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.149+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a35fda9f0 0x558a35fdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.151+0000 7ff21b7fe640 10 client.27451 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:12.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.151+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a35fda9f0 0x558a35fdcde0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69f932e5273cd66 server_cookie=5a639bc79983c321 in_seq=0 out_seq=0 2024-09-10T22:58:12.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.152+0000 7ff21b7fe640 1 -- 172.21.15.96:0/743724212 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a35f25df0 con 0x558a35fda9f0 2024-09-10T22:58:12.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.152+0000 7ff21b7fe640 10 client.27451 handle_command_reply: tid=0 2024-09-10T22:58:12.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.152+0000 7ff244c9a740 1 client.27451 shutdown 2024-09-10T22:58:12.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 msgr2=0x558a35fb97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0ab4f0 tx=0x7ff22c0ab5c0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 shutdown_connections 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a35fda9f0 0x558a35fdcde0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 --2- 172.21.15.96:0/743724212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fb97b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 >> 172.21.15.96:0/743724212 conn(0x558a35fab4d0 msgr2=0x558a35fb1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff21b7fe640 0 client.27451 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 shutdown_connections 2024-09-10T22:58:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.153+0000 7ff244c9a740 1 -- 172.21.15.96:0/743724212 wait complete. 2024-09-10T22:58:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.154+0000 7ff244c9a740 20 client.27451 trim_cache size 0 max 16384 2024-09-10T22:58:12.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.171+0000 7ff244c9a740 1 Processor -- start 2024-09-10T22:58:12.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.171+0000 7ff244c9a740 1 -- start start 2024-09-10T22:58:12.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10ff70 con 0x558a35fadfc0 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1b61c0 con 0x558a35fb9f50 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff244c9a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1b4d50 con 0x558a35fad6a0 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51670/0 (socket says 172.21.15.96:51670) 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 -- 172.21.15.96:0/641829756 learned_addr learned my addr 172.21.15.96:0/641829756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff240e62640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 -- 172.21.15.96:0/641829756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 msgr2=0x558a35fb4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 -- 172.21.15.96:0/641829756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 msgr2=0x558a35fb3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 -- 172.21.15.96:0/641829756 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a35fb74f0 con 0x558a35fad6a0 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.172+0000 7ff23bfff640 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0a7dc0 tx=0x7ff22c0ab6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e6adba4b94fd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff221ffb640 1 -- 172.21.15.96:0/641829756 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0ab300 con 0x558a35fad6a0 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff221ffb640 1 -- 172.21.15.96:0/641829756 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22c0ab7f0 con 0x558a35fad6a0 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff221ffb640 1 -- 172.21.15.96:0/641829756 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0ceb40 con 0x558a35fad6a0 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 -- 172.21.15.96:0/641829756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 msgr2=0x558a35fd6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0a7dc0 tx=0x7ff22c0ab6f0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 -- 172.21.15.96:0/641829756 shutdown_connections 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fb9f50 0x558a35fb4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fb3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 --2- 172.21.15.96:0/641829756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fad6a0 0x558a35fd6f10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.173+0000 7ff244c9a740 1 -- 172.21.15.96:0/641829756 >> 172.21.15.96:0/641829756 conn(0x558a35fab4d0 msgr2=0x558a35fa6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.174+0000 7ff244c9a740 1 -- 172.21.15.96:0/641829756 shutdown_connections 2024-09-10T22:58:12.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.174+0000 7ff244c9a740 1 -- 172.21.15.96:0/641829756 wait complete. 2024-09-10T22:58:12.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.174+0000 7ff244c9a740 1 -- start start 2024-09-10T22:58:12.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.174+0000 7ff244c9a740 1 Processor -- start 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fcfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40102/0 (socket says 172.21.15.96:40102) 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff23bfff640 1 -- 172.21.15.96:0/4291241524 learned_addr learned my addr 172.21.15.96:0/4291241524 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a360408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1b5af0 con 0x558a35fadfc0 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1b5420 con 0x558a35fad6a0 2024-09-10T22:58:12.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1066c0 con 0x558a35fb9f50 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff244c9a740 10 client.0 _ll_register_callbacks cb 0x7ff22c0a7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fcfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.175+0000 7ff240e62640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a360408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 -- 172.21.15.96:0/4291241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 msgr2=0x558a360408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a360408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 -- 172.21.15.96:0/4291241524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 msgr2=0x558a35fcfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fcfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a35fbc6b0 con 0x558a35fad6a0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff23bfff640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0a8a60 tx=0x7ff22c0aae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c58b83237f03d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff21b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0ab300 con 0x558a35fad6a0 2024-09-10T22:58:12.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff22c0aaa50 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22c0ab7f0 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff22c0b02d0 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff244c9a740 10 client.27395 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.176+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a35fbc6b0 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22c0ab7f0 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff22c0a7340 con 0x558a35fad6a0 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 10 client.27395 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 20 client.27395 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a36048530 0x558a3604a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 4 client.27395 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a35f218a0 con 0x558a36048530 2024-09-10T22:58:12.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a36048530 0x558a3604a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff21b7fe640 10 client.27395 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.177+0000 7ff23b7fe640 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a36048530 0x558a3604a950 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7582afadd7e4aef server_cookie=6ccd2f8672a3483a in_seq=0 out_seq=0 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff21b7fe640 1 -- 172.21.15.96:0/4291241524 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a35f218a0 con 0x558a36048530 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff21b7fe640 10 client.27395 handle_command_reply: tid=0 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 client.27395 shutdown 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 msgr2=0x558a35fcf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0a8a60 tx=0x7ff22c0aae70 comp rx=0 tx=0).stop 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 shutdown_connections 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558a36048530 0x558a3604a950 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a35fb9f50 0x558a360408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a35fadfc0 0x558a35fcfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 --2- 172.21.15.96:0/4291241524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a35fad6a0 0x558a35fcf890 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 >> 172.21.15.96:0/4291241524 conn(0x558a35fab4d0 msgr2=0x558a35fb8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff21b7fe640 0 client.27395 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:12.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 shutdown_connections 2024-09-10T22:58:12.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.178+0000 7ff244c9a740 1 -- 172.21.15.96:0/4291241524 wait complete. 2024-09-10T22:58:12.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff244c9a740 20 client.27395 trim_cache size 0 max 16384 2024-09-10T22:58:12.186 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 msgr2=0x7ff2240b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 0x7ff2240b79f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff234009bf0 tx=0x7ff234005f40 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 msgr2=0x7ff23c0749c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 0x7ff23c0749c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009fd0 tx=0x7ff22c00b870 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 shutdown_connections 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2240b5530 0x7ff2240b79f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff23c106d40 0x7ff23c06ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff23c102220 0x7ff23c074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 --2- 172.21.15.96:0/2031262096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff23c101900 0x7ff23c0749c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.179+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 >> 172.21.15.96:0/2031262096 conn(0x7ff23c0fd5b0 msgr2=0x7ff23c1104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.180+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 shutdown_connections 2024-09-10T22:58:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:12.180+0000 7ff241663640 1 -- 172.21.15.96:0/2031262096 wait complete. 2024-09-10T22:58:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:13 smithi096 ceph-mon[30869]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-09-10T22:58:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:13 smithi121 ceph-mon[35119]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-09-10T22:58:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:13 smithi117 ceph-mon[35081]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-09-10T22:58:14.419 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:58:14.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:58:14.763 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:15 smithi096 ceph-mon[30869]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-09-10T22:58:15.577 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:15 smithi117 ceph-mon[35081]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 -- 172.21.15.96:0/1956567434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc1021d0 msgr2=0x7f9dbc10ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/1956567434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc10ef20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009900 tx=0x7f9dac02a410 comp rx=0 tx=0).stop 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 -- 172.21.15.96:0/1956567434 shutdown_connections 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/1956567434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc10f460 0x7f9dbc111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/1956567434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc10ef20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/1956567434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1018b0 0x7f9dbc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 -- 172.21.15.96:0/1956567434 >> 172.21.15.96:0/1956567434 conn(0x7f9dbc0fd7c0 msgr2=0x7f9dbc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.662+0000 7f9dc3a23640 1 -- 172.21.15.96:0/1956567434 shutdown_connections 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.663+0000 7f9dc3a23640 1 -- 172.21.15.96:0/1956567434 wait complete. 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.663+0000 7f9dc3a23640 1 Processor -- start 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.663+0000 7f9dc3a23640 1 -- start start 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.663+0000 7f9dc3a23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc1018b0 0x7f9dbc19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.663+0000 7f9dc3a23640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc3a23640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc3a23640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc114fd0 con 0x7f9dbc10f460 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc3a23640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc114e50 con 0x7f9dbc1018b0 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc3a23640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc115150 con 0x7f9dbc1021d0 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36010/0 (socket says 172.21.15.96:36010) 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 -- 172.21.15.96:0/3882277176 learned_addr learned my addr 172.21.15.96:0/3882277176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc1018b0 0x7f9dbc19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 -- 172.21.15.96:0/3882277176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1021d0 msgr2=0x7f9dbc19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 -- 172.21.15.96:0/3882277176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc1018b0 msgr2=0x7f9dbc19b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc1018b0 0x7f9dbc19b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 -- 172.21.15.96:0/3882277176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc1a1650 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.664+0000 7f9dc1fc0640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9db400eb40 tx=0x7f9db400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.665+0000 7f9db27fc640 1 -- 172.21.15.96:0/3882277176 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4004b20 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.665+0000 7f9db27fc640 1 -- 172.21.15.96:0/3882277176 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db4004cc0 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.665+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc1a1940 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.665+0000 7f9db27fc640 1 -- 172.21.15.96:0/3882277176 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40186f0 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.665+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc1a8000 con 0x7f9dbc10f460 2024-09-10T22:58:15.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.668+0000 7f9db27fc640 1 -- 172.21.15.96:0/3882277176 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9db4020030 con 0x7f9dbc10f460 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 Processor -- start 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 -- start start 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d80250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d80790 0x5624c9d82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624c9ceae60 con 0x5624c9d73ee0 2024-09-10T22:58:15.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624c9cda290 con 0x5624c9d74800 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5624c9cde0a0 con 0x5624c9d80790 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36018/0 (socket says 172.21.15.96:36018) 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d80250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/871141925 learned_addr learned my addr 172.21.15.96:0/871141925 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:15.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9db27fc640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 0x7f9da80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc1fc0640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 0x7f9da80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 -- 172.21.15.96:0/871141925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d80790 msgr2=0x5624c9d82b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d80790 0x5624c9d82b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 -- 172.21.15.96:0/871141925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 msgr2=0x5624c9d742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9db27fc640 1 -- 172.21.15.96:0/3882277176 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9db40e5ef0 con 0x7f9dbc10f460 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.669+0000 7f9dc17bf640 1 -- 172.21.15.96:0/871141925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c9a38930 con 0x5624c9d74800 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.670+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.670+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d80250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004630 tx=0x7f9db800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a2d7bbf579a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.670+0000 7f9da77fe640 1 -- 172.21.15.96:0/871141925 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8003bd0 con 0x5624c9d74800 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.670+0000 7f9da77fe640 1 -- 172.21.15.96:0/871141925 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db8015070 con 0x5624c9d74800 2024-09-10T22:58:15.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.670+0000 7f9da77fe640 1 -- 172.21.15.96:0/871141925 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004d50 con 0x5624c9d74800 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- 172.21.15.96:0/871141925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 msgr2=0x5624c9d80250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d80250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004630 tx=0x7f9db800bb20 comp rx=0 tx=0).stop 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc1fc0640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 0x7f9da80b7820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c7d0 tx=0x7f9db40712c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- 172.21.15.96:0/871141925 shutdown_connections 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d80250 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d80790 0x5624c9d82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/871141925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d73ee0 0x5624c9d742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- 172.21.15.96:0/871141925 >> 172.21.15.96:0/871141925 conn(0x5624c9d71d10 msgr2=0x5624c9d72eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- 172.21.15.96:0/871141925 shutdown_connections 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- 172.21.15.96:0/871141925 wait complete. 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 -- start start 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.671+0000 7f9dc5df8740 1 Processor -- start 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d7e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624c9b3b890 con 0x5624c9d80790 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624c9c74d40 con 0x5624c9d74800 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5624c9d83440 con 0x5624c9d73ee0 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.672+0000 7f9dc5df8740 10 client.0 _ll_register_callbacks cb 0x5624c9cd8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51720/0 (socket says 172.21.15.96:51720) 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/3019396002 learned_addr learned my addr 172.21.15.96:0/3019396002 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 msgr2=0x5624c9d7dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d7dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/3019396002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 msgr2=0x5624c9d7e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d7e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/3019396002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c9d8c850 con 0x5624c9d73ee0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.673+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9dac032d50 tx=0x7f9dac032d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4534d1ae88923587 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.674+0000 7f9da4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.674+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac035040 con 0x5624c9d73ee0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.674+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9dac0318b0 con 0x5624c9d73ee0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.674+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db8015070 con 0x5624c9d73ee0 2024-09-10T22:58:15.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9dac005030 con 0x5624c9d73ee0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 10 client.27466 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5624c9d8f200 con 0x5624c9d73ee0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db8015070 con 0x5624c9d73ee0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9dac046480 con 0x5624c9d73ee0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 10 client.27466 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 20 client.27466 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9d906c0 0x5624c9d92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 4 client.27466 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624c9cecdb0 con 0x5624c9d906c0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.675+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9d906c0 0x5624c9d92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.676+0000 7f9da4ff9640 10 client.27466 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.676+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9d906c0 0x5624c9d92ab0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322adfa5f0d74ebe server_cookie=3025b0e1346c0c9 in_seq=0 out_seq=0 2024-09-10T22:58:15.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.676+0000 7f9da4ff9640 1 -- 172.21.15.96:0/3019396002 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624c9cecdb0 con 0x5624c9d906c0 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.676+0000 7f9da4ff9640 10 client.27466 handle_command_reply: tid=0 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.676+0000 7f9dc5df8740 1 client.27466 shutdown 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 msgr2=0x5624c9d7d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9dac032d50 tx=0x7f9dac032d80 comp rx=0 tx=0).stop 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 shutdown_connections 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9d906c0 0x5624c9d92ab0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d7dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d7e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/3019396002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7d760 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9da4ff9640 0 client.27466 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 >> 172.21.15.96:0/3019396002 conn(0x5624c9d71d10 msgr2=0x5624c9d70f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 shutdown_connections 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 1 -- 172.21.15.96:0/3019396002 wait complete. 2024-09-10T22:58:15.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.677+0000 7f9dc5df8740 20 client.27466 trim_cache size 0 max 16384 2024-09-10T22:58:15.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 Processor -- start 2024-09-10T22:58:15.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 -- start start 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d74800 0x5624c9d7a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d80790 0x5624c9d7aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1b0580 con 0x5624c9d74800 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1b20c0 con 0x5624c9d80790 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.697+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1b0c50 con 0x5624c9d73ee0 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51734/0 (socket says 172.21.15.96:51734) 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/1366589737 learned_addr learned my addr 172.21.15.96:0/1366589737 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d80790 0x5624c9d7aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d74800 0x5624c9d7a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/1366589737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d80790 msgr2=0x5624c9d7aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d80790 0x5624c9d7aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/1366589737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d74800 msgr2=0x5624c9d7a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d74800 0x5624c9d7a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/1366589737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624c9d93090 con 0x5624c9d73ee0 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.698+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005ec0 tx=0x7f9dac000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c12bca5b3f3d86f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.699+0000 7f9da6ffd640 1 -- 172.21.15.96:0/1366589737 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac035070 con 0x5624c9d73ee0 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.700+0000 7f9da6ffd640 1 -- 172.21.15.96:0/1366589737 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac056030 con 0x5624c9d73ee0 2024-09-10T22:58:15.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.700+0000 7f9da6ffd640 1 -- 172.21.15.96:0/1366589737 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac004750 con 0x5624c9d73ee0 2024-09-10T22:58:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.700+0000 7f9dc5df8740 1 -- 172.21.15.96:0/1366589737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 msgr2=0x5624c9d7f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.700+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005ec0 tx=0x7f9dac000bc0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 -- 172.21.15.96:0/1366589737 shutdown_connections 2024-09-10T22:58:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d80790 0x5624c9d7aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d74800 0x5624c9d7a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/1366589737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d7f400 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 -- 172.21.15.96:0/1366589737 >> 172.21.15.96:0/1366589737 conn(0x5624c9d71d10 msgr2=0x5624c9d7def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 -- 172.21.15.96:0/1366589737 shutdown_connections 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 -- 172.21.15.96:0/1366589737 wait complete. 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 -- start start 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 Processor -- start 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.701+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1b19f0 con 0x5624c9d80790 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1b1320 con 0x5624c9d74800 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1ae560 con 0x5624c9d73ee0 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 10 client.0 _ll_register_callbacks cb 0x7f9dac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:15.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36046/0 (socket says 172.21.15.96:36046) 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/2721924264 learned_addr learned my addr 172.21.15.96:0/2721924264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/2721924264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 msgr2=0x5624c9d856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d856d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 msgr2=0x5624c9d85c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d85c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 -- 172.21.15.96:0/2721924264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624c9d71b40 con 0x5624c9d80790 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc0fbe640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0467e0 tx=0x7f9dac005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da75dc2375c01cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:15.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9da4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac035040 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dac056030 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.702+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9dac005760 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9dac004030 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 10 client.17820 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5624c9dfe6c0 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac056030 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9dac0316b0 con 0x5624c9d80790 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 10 client.17820 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 20 client.17820 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9dfedc0 0x5624c9e011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 4 client.17820 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:58:15.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5624c9ce8860 con 0x5624c9dfedc0 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.703+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9dfedc0 0x5624c9e011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.704+0000 7f9da4ff9640 10 client.17820 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.704+0000 7f9dc17bf640 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9dfedc0 0x5624c9e011b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e14fe19302cc03 server_cookie=abd082fe5d50d67f in_seq=0 out_seq=0 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9da4ff9640 1 -- 172.21.15.96:0/2721924264 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5624c9ce8860 con 0x5624c9dfedc0 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9da4ff9640 10 client.17820 handle_command_reply: tid=0 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 client.17820 shutdown 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 msgr2=0x5624c9d86150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0467e0 tx=0x7f9dac005430 comp rx=0 tx=0).stop 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 shutdown_connections 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5624c9dfedc0 0x5624c9e011b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624c9d74800 0x5624c9d85c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624c9d80790 0x5624c9d86150 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 --2- 172.21.15.96:0/2721924264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5624c9d73ee0 0x5624c9d856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 >> 172.21.15.96:0/2721924264 conn(0x5624c9d71d10 msgr2=0x5624c9d70750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.705+0000 7f9da4ff9640 0 client.17820 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 shutdown_connections 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc5df8740 1 -- 172.21.15.96:0/2721924264 wait complete. 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc5df8740 20 client.17820 trim_cache size 0 max 16384 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "6a1f25f9-789b-475d-b1e0-5f3a64165683", 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:58:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 msgr2=0x7f9da80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 0x7f9da80b7820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c7d0 tx=0x7f9db40712c0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 msgr2=0x7f9dbc1a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9db400eb40 tx=0x7f9db400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 shutdown_connections 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9da80b5360 0x7f9da80b7820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.706+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dbc10f460 0x7f9dbc1a0e80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.707+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dbc1021d0 0x7f9dbc19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.707+0000 7f9dc3a23640 1 --2- 172.21.15.96:0/3882277176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc1018b0 0x7f9dbc19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.707+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 >> 172.21.15.96:0/3882277176 conn(0x7f9dbc0fd7c0 msgr2=0x7f9dbc110790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:15.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.707+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 shutdown_connections 2024-09-10T22:58:15.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:15.707+0000 7f9dc3a23640 1 -- 172.21.15.96:0/3882277176 wait complete. 2024-09-10T22:58:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:15 smithi121 ceph-mon[35119]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 MiB/s wr, 593 op/s 2024-09-10T22:58:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 640.0 s, 1535.8 tps, lat 64.656 ms stddev 66.520 2024-09-10T22:58:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:16 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:58:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:16 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:16.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:16 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:58:16.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:16 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:16 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:58:16.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:16 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:17.254 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:58:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:17 smithi096 ceph-mon[30869]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 722 op/s 2024-09-10T22:58:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:17 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:58:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:17 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:17.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:17 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:58:17.615 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:17 smithi121 ceph-mon[35119]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 722 op/s 2024-09-10T22:58:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:17 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:58:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:17 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:17.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:17 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:58:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:17 smithi117 ceph-mon[35081]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 722 op/s 2024-09-10T22:58:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:17 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:58:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:17 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:17 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:58:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:18 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:18.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:18 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:18.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:18 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 -- 172.21.15.96:0/862385598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce81076a0 msgr2=0x7fcce8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 --2- 172.21.15.96:0/862385598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce81076a0 0x7fcce8107ae0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 -- 172.21.15.96:0/862385598 shutdown_connections 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 --2- 172.21.15.96:0/862385598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8108020 0x7fcce810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 --2- 172.21.15.96:0/862385598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce81076a0 0x7fcce8107ae0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 --2- 172.21.15.96:0/862385598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce8101c90 0x7fcce8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.911+0000 7fccf0167640 1 -- 172.21.15.96:0/862385598 >> 172.21.15.96:0/862385598 conn(0x7fcce80fd700 msgr2=0x7fcce80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 -- 172.21.15.96:0/862385598 shutdown_connections 2024-09-10T22:58:18.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 -- 172.21.15.96:0/862385598 wait complete. 2024-09-10T22:58:18.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 Processor -- start 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 -- start start 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce81076a0 0x7fcce8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 0x7fcce819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.912+0000 7fccf0167640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111ca0 con 0x7fcce8108020 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccf0167640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111b20 con 0x7fcce8101c90 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccf0167640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111e20 con 0x7fcce81076a0 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 0x7fcce819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccedf03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccedf03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:42980/0 (socket says 172.21.15.96:42980) 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccedf03640 1 -- 172.21.15.96:0/1213257513 learned_addr learned my addr 172.21.15.96:0/1213257513 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fcced702640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce81076a0 0x7fcce8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 -- 172.21.15.96:0/1213257513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce81076a0 msgr2=0x7fcce8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce81076a0 0x7fcce8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 -- 172.21.15.96:0/1213257513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 msgr2=0x7fcce8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 -- 172.21.15.96:0/1213257513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce819e2b0 con 0x7fcce8108020 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccedf03640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccee704640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 0x7fcce819dae0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fd0 tx=0x7fcce000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccdeffd640 1 -- 172.21.15.96:0/1213257513 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce000ce20 con 0x7fcce8108020 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccdeffd640 1 -- 172.21.15.96:0/1213257513 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce0005020 con 0x7fcce8108020 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccdeffd640 1 -- 172.21.15.96:0/1213257513 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0010690 con 0x7fcce8108020 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce819e5a0 con 0x7fcce8108020 2024-09-10T22:58:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.913+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce806aab0 con 0x7fcce8108020 2024-09-10T22:58:18.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.917+0000 7fccf253c740 1 Processor -- start 2024-09-10T22:58:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.917+0000 7fccdeffd640 1 -- 172.21.15.96:0/1213257513 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fcce00054d0 con 0x7fcce8108020 2024-09-10T22:58:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 -- start start 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073eff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561073e75e50 con 0x561073f0b950 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561073e65280 con 0x561073eff0a0 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccf253c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561073e69090 con 0x561073eff9c0 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fcced702640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fcced702640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47274/0 (socket says 172.21.15.96:47274) 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fcced702640 1 -- 172.21.15.96:0/2568348605 learned_addr learned my addr 172.21.15.96:0/2568348605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fcced702640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073eff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.918+0000 7fccedf03640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 -- 172.21.15.96:0/2568348605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 msgr2=0x561073eff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073eff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 -- 172.21.15.96:0/2568348605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 msgr2=0x561073f0dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 -- 172.21.15.96:0/2568348605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561073bc3650 con 0x561073eff9c0 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccdeffd640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 0x7fccd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fcced702640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073eff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccedf03640 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0b410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcce400f3c0 tx=0x7fcce400f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2e73778086e7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccdeffd640 1 -- 172.21.15.96:0/1213257513 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcce00e61c0 con 0x7fcce8108020 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fcced702640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 0x7fccd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccd3fff640 1 -- 172.21.15.96:0/2568348605 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce401d070 con 0x561073eff9c0 2024-09-10T22:58:18.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccd3fff640 1 -- 172.21.15.96:0/2568348605 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce4010db0 con 0x561073eff9c0 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.919+0000 7fccd3fff640 1 -- 172.21.15.96:0/2568348605 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4016340 con 0x561073eff9c0 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 -- 172.21.15.96:0/2568348605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 msgr2=0x561073f0b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fcced702640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 0x7fccd40b77a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fccd80096c0 tx=0x7fccd8005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0b410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcce400f3c0 tx=0x7fcce400f880 comp rx=0 tx=0).stop 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 -- 172.21.15.96:0/2568348605 shutdown_connections 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0b410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 --2- 172.21.15.96:0/2568348605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073eff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 -- 172.21.15.96:0/2568348605 >> 172.21.15.96:0/2568348605 conn(0x561073efced0 msgr2=0x561073efe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 -- 172.21.15.96:0/2568348605 shutdown_connections 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.920+0000 7fccf253c740 1 -- 172.21.15.96:0/2568348605 wait complete. 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 -- start start 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 Processor -- start 2024-09-10T22:58:18.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073eff9c0 0x561073f08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073f0b950 0x561073f08c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561073cc6700 con 0x561073eff9c0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561073e00110 con 0x561073f0b950 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561073f129e0 con 0x561073eff0a0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccf253c740 10 client.0 _ll_register_callbacks cb 0x561073e19990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccedf03640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccedf03640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:34628/0 (socket says 172.21.15.96:34628) 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccedf03640 1 -- 172.21.15.96:0/4082263073 learned_addr learned my addr 172.21.15.96:0/4082263073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fccee704640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073f0b950 0x561073f08c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.921+0000 7fcced702640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073eff9c0 0x561073f08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 -- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073f0b950 msgr2=0x561073f08c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073f0b950 0x561073f08c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 -- 172.21.15.96:0/4082263073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073eff9c0 msgr2=0x561073f08660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073eff9c0 0x561073f08660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 -- 172.21.15.96:0/4082263073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561073e2dc40 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccedf03640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005cf0 tx=0x7fcce4018670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff23821cec16817 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4027020 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcce4010700 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce4010db0 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcce401d040 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccf253c740 10 client.27481 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561073f1e2c0 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.922+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce4010db0 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcce40046c0 con 0x561073eff0a0 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 10 client.27481 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 10 client.27481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:18.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 20 client.27481 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 4 client.27481 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561073e77da0 con 0x561073f1f670 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fcced702640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fcced702640 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f21a60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259c1157bbfd73ab server_cookie=6bba921f9cb9263 in_seq=0 out_seq=0 2024-09-10T22:58:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.923+0000 7fccd17fa640 10 client.27481 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:18.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.925+0000 7fccd17fa640 1 -- 172.21.15.96:0/4082263073 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561073e77da0 con 0x561073f1f670 2024-09-10T22:58:18.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.925+0000 7fccd17fa640 10 client.27481 handle_command_reply: tid=0 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.925+0000 7fccf253c740 1 client.27481 shutdown 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.925+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 msgr2=0x561073f0b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005cf0 tx=0x7fcce4018670 comp rx=0 tx=0).stop 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 shutdown_connections 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f21a60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073f0b950 0x561073f08c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccd17fa640 0 client.27481 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073eff9c0 0x561073f08660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 --2- 172.21.15.96:0/4082263073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f0b210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 >> 172.21.15.96:0/4082263073 conn(0x561073efced0 msgr2=0x561073f02a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 shutdown_connections 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 1 -- 172.21.15.96:0/4082263073 wait complete. 2024-09-10T22:58:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.926+0000 7fccf253c740 20 client.27481 trim_cache size 0 max 16384 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.948+0000 7fccf253c740 1 Processor -- start 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.948+0000 7fccf253c740 1 -- start start 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073f1bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ac040 con 0x561073f0b950 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81adb80 con 0x561073eff0a0 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccf253c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ac710 con 0x561073eff9c0 2024-09-10T22:58:18.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47296/0 (socket says 172.21.15.96:47296) 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 -- 172.21.15.96:0/3866359481 learned_addr learned my addr 172.21.15.96:0/3866359481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccedf03640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073f1bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fcced702640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 -- 172.21.15.96:0/3866359481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 msgr2=0x561073f0cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 -- 172.21.15.96:0/3866359481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 msgr2=0x561073f1bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073f1bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 -- 172.21.15.96:0/3866359481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561073f08410 con 0x561073f0b950 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccedf03640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073f1bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.949+0000 7fccee704640 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcce00a8ff0 tx=0x7fcce00a90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7731cfa9f1abc7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.950+0000 7fccd37fe640 1 -- 172.21.15.96:0/3866359481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00d3070 con 0x561073f0b950 2024-09-10T22:58:18.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.950+0000 7fccd37fe640 1 -- 172.21.15.96:0/3866359481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce00e5040 con 0x561073f0b950 2024-09-10T22:58:18.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.950+0000 7fccd37fe640 1 -- 172.21.15.96:0/3866359481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00b5b20 con 0x561073f0b950 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- 172.21.15.96:0/3866359481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 msgr2=0x561073f0d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcce00a8ff0 tx=0x7fcce00a90f0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- 172.21.15.96:0/3866359481 shutdown_connections 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f0d280 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff9c0 0x561073f0cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 --2- 172.21.15.96:0/3866359481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff0a0 0x561073f1bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- 172.21.15.96:0/3866359481 >> 172.21.15.96:0/3866359481 conn(0x561073efced0 msgr2=0x561073efb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- 172.21.15.96:0/3866359481 shutdown_connections 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- 172.21.15.96:0/3866359481 wait complete. 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 -- start start 2024-09-10T22:58:18.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.951+0000 7fccf253c740 1 Processor -- start 2024-09-10T22:58:18.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ad4b0 con 0x561073f0b950 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81acde0 con 0x561073eff9c0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81aa020 con 0x561073eff0a0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.953+0000 7fccf253c740 10 client.0 _ll_register_callbacks cb 0x7fcce401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fcced702640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fcced702640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43024/0 (socket says 172.21.15.96:43024) 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fcced702640 1 -- 172.21.15.96:0/2257365900 learned_addr learned my addr 172.21.15.96:0/2257365900 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccedf03640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 -- 172.21.15.96:0/2257365900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 msgr2=0x561073f17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 -- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 msgr2=0x561073f181b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 -- 172.21.15.96:0/2257365900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561073f118f0 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fcced702640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccee704640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f113b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcce00a8ff0 tx=0x7fcce00053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf25d98c3f56d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00cc060 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fcce00b5d30 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce00e5040 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcce00c6030 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccf253c740 10 client.17838 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.954+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561073f04c30 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce00e5040 con 0x561073f0b950 2024-09-10T22:58:18.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fcce00a8570 con 0x561073f0b950 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 10 client.17838 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 20 client.17838 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 4 client.17838 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561073e73850 con 0x561073f1f670 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccedf03640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccd17fa640 10 client.17838 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.955+0000 7fccedf03640 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f05310 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c812e09b7e67dd2a server_cookie=e7757474828f06bb in_seq=0 out_seq=0 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccd17fa640 1 -- 172.21.15.96:0/2257365900 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x561073e73850 con 0x561073f1f670 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccd17fa640 10 client.17838 handle_command_reply: tid=0 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 client.17838 shutdown 2024-09-10T22:58:18.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 msgr2=0x561073f113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f113b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcce00a8ff0 tx=0x7fcce00053b0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 shutdown_connections 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x561073f1f670 0x561073f05310 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561073eff9c0 0x561073f181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561073f0b950 0x561073f113b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccd17fa640 0 client.17838 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 --2- 172.21.15.96:0/2257365900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561073eff0a0 0x561073f17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 >> 172.21.15.96:0/2257365900 conn(0x561073efced0 msgr2=0x561073f09c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 shutdown_connections 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 1 -- 172.21.15.96:0/2257365900 wait complete. 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.956+0000 7fccf253c740 20 client.17838 trim_cache size 0 max 16384 2024-09-10T22:58:18.962 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "6a1f25f9-789b-475d-b1e0-5f3a64165683", 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T22:58:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:58:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:58:18.971 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:58:18.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 msgr2=0x7fccd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 0x7fccd40b77a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fccd80096c0 tx=0x7fccd8005db0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 msgr2=0x7fcce819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 0x7fcce819dae0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fd0 tx=0x7fcce000ef10 comp rx=0 tx=0).stop 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 shutdown_connections 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fccd40b52e0 0x7fccd40b77a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcce8108020 0x7fcce819dae0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcce81076a0 0x7fcce8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 --2- 172.21.15.96:0/1213257513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce8101c90 0x7fcce8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 >> 172.21.15.96:0/1213257513 conn(0x7fcce80fd700 msgr2=0x7fcce80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 shutdown_connections 2024-09-10T22:58:18.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:18.957+0000 7fccf0167640 1 -- 172.21.15.96:0/1213257513 wait complete. 2024-09-10T22:58:19.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:19 smithi096 ceph-mon[30869]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 540 op/s 2024-09-10T22:58:19.664 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:19 smithi121 ceph-mon[35119]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 540 op/s 2024-09-10T22:58:19.737 INFO:tasks.cephfs.filesystem:scrub status for tag:6a1f25f9-789b-475d-b1e0-5f3a64165683 - {'path': '/', 'tag': '6a1f25f9-789b-475d-b1e0-5f3a64165683', 'options': 'recursive,force'} 2024-09-10T22:58:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:19 smithi117 ceph-mon[35081]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 540 op/s 2024-09-10T22:58:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 645.0 s, 1630.6 tps, lat 61.650 ms stddev 67.435 2024-09-10T22:58:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:21 smithi096 ceph-mon[30869]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.8 MiB/s wr, 769 op/s 2024-09-10T22:58:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:21 smithi121 ceph-mon[35119]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.8 MiB/s wr, 769 op/s 2024-09-10T22:58:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:21 smithi117 ceph-mon[35081]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.8 MiB/s wr, 769 op/s 2024-09-10T22:58:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.146+0000 7f013d61f640 1 -- 172.21.15.96:0/3338346257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 msgr2=0x7f0138102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.146+0000 7f0135ffb640 1 -- 172.21.15.96:0/3338346257 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c035020 con 0x7f01381029b0 2024-09-10T22:58:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.146+0000 7f013d61f640 1 --2- 172.21.15.96:0/3338346257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f0138102e30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f012c009930 tx=0x7f012c02a480 comp rx=0 tx=0).stop 2024-09-10T22:58:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.152+0000 7f013d61f640 1 -- 172.21.15.96:0/3338346257 shutdown_connections 2024-09-10T22:58:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.152+0000 7f013d61f640 1 --2- 172.21.15.96:0/3338346257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0138103370 0x7f0138109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.152+0000 7f013d61f640 1 --2- 172.21.15.96:0/3338346257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f0138102e30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.152+0000 7f013d61f640 1 --2- 172.21.15.96:0/3338346257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.153+0000 7f013d61f640 1 -- 172.21.15.96:0/3338346257 >> 172.21.15.96:0/3338346257 conn(0x7f01380fd140 msgr2=0x7f01380ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.156+0000 7f013d61f640 1 -- 172.21.15.96:0/3338346257 shutdown_connections 2024-09-10T22:58:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.163+0000 7f013d61f640 1 -- 172.21.15.96:0/3338346257 wait complete. 2024-09-10T22:58:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.163+0000 7f013d61f640 1 Processor -- start 2024-09-10T22:58:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 -- start start 2024-09-10T22:58:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f01380722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0138103370 0x7f0138072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f013810c540 con 0x7f0138103370 2024-09-10T22:58:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f013810c3c0 con 0x7f0138101860 2024-09-10T22:58:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.170+0000 7f013d61f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f013810c6c0 con 0x7f01381029b0 2024-09-10T22:58:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.171+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.171+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43044/0 (socket says 172.21.15.96:43044) 2024-09-10T22:58:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.171+0000 7f0136ffd640 1 -- 172.21.15.96:0/1209065469 learned_addr learned my addr 172.21.15.96:0/1209065469 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.171+0000 7f01367fc640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f01380722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.173+0000 7f0136ffd640 1 -- 172.21.15.96:0/1209065469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 msgr2=0x7f01380722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.173+0000 7f0136ffd640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f01380722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.173+0000 7f0136ffd640 1 -- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0138103370 msgr2=0x7f0138072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.173+0000 7f0136ffd640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0138103370 0x7f0138072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.173+0000 7f0136ffd640 1 -- 172.21.15.96:0/1209065469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01381a6fd0 con 0x7f0138101860 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f0136ffd640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f013000ba50 tx=0x7f013000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f0127fff640 1 -- 172.21.15.96:0/1209065469 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130019070 con 0x7f0138101860 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f0127fff640 1 -- 172.21.15.96:0/1209065469 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f013000cc10 con 0x7f0138101860 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01381a72c0 con 0x7f0138101860 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f01367fc640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f01380722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f0127fff640 1 -- 172.21.15.96:0/1209065469 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300075f0 con 0x7f0138101860 2024-09-10T22:58:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.174+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a78e0 con 0x7f0138101860 2024-09-10T22:58:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 Processor -- start 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 -- start start 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f0127fff640 1 -- 172.21.15.96:0/1209065469 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0130014980 con 0x7f0138101860 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 0x55949817b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x55949817e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5594980e86f0 con 0x559498179190 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5594980d7b20 con 0x55949817bbd0 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01403ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5594980db930 con 0x559498176860 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 0x55949817b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01377fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x55949817e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01377fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:34660/0 (socket says 172.21.15.96:34660) 2024-09-10T22:58:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.178+0000 7f01377fe640 1 -- 172.21.15.96:0/599932543 learned_addr learned my addr 172.21.15.96:0/599932543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f0127fff640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 0x7f01200b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f0127fff640 1 -- 172.21.15.96:0/1209065469 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f01300e53a0 con 0x7f0138101860 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01377fe640 1 -- 172.21.15.96:0/599932543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 msgr2=0x55949817e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01377fe640 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x55949817e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01377fe640 1 -- 172.21.15.96:0/599932543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 msgr2=0x55949817b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01377fe640 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 0x55949817b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01377fe640 1 -- 172.21.15.96:0/599932543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55949809d530 con 0x559498176860 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f01367fc640 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x55949817e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.179+0000 7f0136ffd640 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 0x55949817b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.180+0000 7f01377fe640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 0x7f01200b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.180+0000 7f01377fe640 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0128009770 tx=0x7f0128009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39251b3b01a9cdf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.180+0000 7f01257fa640 1 -- 172.21.15.96:0/599932543 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128017030 con 0x559498176860 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.180+0000 7f01257fa640 1 -- 172.21.15.96:0/599932543 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012800bd50 con 0x559498176860 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.180+0000 7f01257fa640 1 -- 172.21.15.96:0/599932543 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280107f0 con 0x559498176860 2024-09-10T22:58:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- 172.21.15.96:0/599932543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 msgr2=0x559498178c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01377fe640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 0x7f01200b7850 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f012801b000 tx=0x7f012801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0128009770 tx=0x7f0128009c30 comp rx=0 tx=0).stop 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- 172.21.15.96:0/599932543 shutdown_connections 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x55949817e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498179190 0x55949817b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 --2- 172.21.15.96:0/599932543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498178c50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- 172.21.15.96:0/599932543 >> 172.21.15.96:0/599932543 conn(0x5594981719b0 msgr2=0x559498172740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- 172.21.15.96:0/599932543 shutdown_connections 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- 172.21.15.96:0/599932543 wait complete. 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 -- start start 2024-09-10T22:58:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.181+0000 7f01403ed740 1 Processor -- start 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01403ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498176860 0x55949817fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01403ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 0x5594981804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01403ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x5594981810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01367fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 0x5594981804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01367fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 0x5594981804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:34674/0 (socket says 172.21.15.96:34674) 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01367fc640 1 -- 172.21.15.96:0/1869011114 learned_addr learned my addr 172.21.15.96:0/1869011114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55949800fab0 con 0x559498176860 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.182+0000 7f01377fe640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x5594981810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55949807d730 con 0x55949817bbd0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55949817ebb0 con 0x559498179190 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01377fe640 1 -- 172.21.15.96:0/1869011114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 msgr2=0x5594981804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01377fe640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 0x5594981804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01377fe640 1 -- 172.21.15.96:0/1869011114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498176860 msgr2=0x55949817fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01377fe640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498176860 0x55949817fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.183+0000 7f01377fe640 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559497e39790 con 0x55949817bbd0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f01377fe640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x5594981810a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0128009770 tx=0x7f0128010580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06fd9cdfcccc5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f011a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f011a7fc640 1 -- 172.21.15.96:0/1869011114 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012802b020 con 0x55949817bbd0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f011a7fc640 1 -- 172.21.15.96:0/1869011114 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f01280105f0 con 0x55949817bbd0 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f011a7fc640 10 client.0 inactive, discarding osd_map(77..77 src has 1..77) 2024-09-10T22:58:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.184+0000 7f01403ed740 10 client.0 _ll_register_callbacks cb 0x5594980d16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012800bd50 con 0x55949817bbd0 2024-09-10T22:58:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f011a7fc640 1 -- 172.21.15.96:0/1869011114 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0128009340 con 0x55949817bbd0 2024-09-10T22:58:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 10 client.27440 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x559497eb2620 con 0x55949817bbd0 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f011a7fc640 1 -- 172.21.15.96:0/1869011114 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f01280107f0 con 0x55949817bbd0 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 10 client.27440 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 10 client.27440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 20 client.27440 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.185+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x55949818d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.186+0000 7f01403ed740 4 client.27440 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.186+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f012800bd50 con 0x55949818b240 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.186+0000 7f0136ffd640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x55949818d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.186+0000 7f011a7fc640 10 client.27440 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.186+0000 7f0136ffd640 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x55949818d6b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885527547bbff350 server_cookie=8eebed3d0a9b28c7 in_seq=0 out_seq=0 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f011a7fc640 1 -- 172.21.15.96:0/1869011114 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f012800bd50 con 0x55949818b240 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f011a7fc640 10 client.27440 handle_command_reply: tid=0 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 client.27440 shutdown 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 msgr2=0x5594981810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x5594981810a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0128009770 tx=0x7f0128010580 comp rx=0 tx=0).stop 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 shutdown_connections 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x55949818d6b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f011a7fc640 0 client.27440 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949817bbd0 0x5594981810a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498179190 0x5594981804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 --2- 172.21.15.96:0/1869011114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559498176860 0x55949817fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 >> 172.21.15.96:0/1869011114 conn(0x5594981719b0 msgr2=0x559498170bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 shutdown_connections 2024-09-10T22:58:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 1 -- 172.21.15.96:0/1869011114 wait complete. 2024-09-10T22:58:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.187+0000 7f01403ed740 20 client.27440 trim_cache size 0 max 16384 2024-09-10T22:58:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.206+0000 7f01403ed740 1 Processor -- start 2024-09-10T22:58:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.206+0000 7f01403ed740 1 -- start start 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x5594981804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x55949817cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01381ae520 con 0x55949817bbd0 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x5594981804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x55949817cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47370/0 (socket says 172.21.15.96:47370) 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01377fe640 1 -- 172.21.15.96:0/862415408 learned_addr learned my addr 172.21.15.96:0/862415408 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01381c3080 con 0x559498179190 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.207+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01381aebf0 con 0x559498176860 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 -- 172.21.15.96:0/862415408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 msgr2=0x5594981804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x5594981804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 -- 172.21.15.96:0/862415408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 msgr2=0x55949817d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 -- 172.21.15.96:0/862415408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594981758e0 con 0x559498179190 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01377fe640 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f01367fc640 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x55949817cb60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f012c005c40 tx=0x7f012c004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de191a7e8bd41a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f0124ff9640 1 -- 172.21.15.96:0/862415408 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c035040 con 0x559498179190 2024-09-10T22:58:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f0124ff9640 1 -- 172.21.15.96:0/862415408 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c003ca0 con 0x559498179190 2024-09-10T22:58:23.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.208+0000 7f0124ff9640 1 -- 172.21.15.96:0/862415408 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c031d80 con 0x559498179190 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 msgr2=0x55949817cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x55949817cb60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f012c005c40 tx=0x7f012c004340 comp rx=0 tx=0).stop 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 shutdown_connections 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x55949817cb60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x55949817d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 --2- 172.21.15.96:0/862415408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x5594981804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 >> 172.21.15.96:0/862415408 conn(0x5594981719b0 msgr2=0x559498175c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.209+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 shutdown_connections 2024-09-10T22:58:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.210+0000 7f01403ed740 1 -- 172.21.15.96:0/862415408 wait complete. 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.210+0000 7f01403ed740 1 -- start start 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.210+0000 7f01403ed740 1 Processor -- start 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x5594981859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x559498186580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01381c29b0 con 0x55949817bbd0 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01381af2c0 con 0x559498179190 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01381ac500 con 0x559498176860 2024-09-10T22:58:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f01403ed740 10 client.0 _ll_register_callbacks cb 0x7f01280095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f0136ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:34698/0 (socket says 172.21.15.96:34698) 2024-09-10T22:58:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.211+0000 7f0136ffd640 1 -- 172.21.15.96:0/3638200455 learned_addr learned my addr 172.21.15.96:0/3638200455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f01367fc640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x5594981859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f0136ffd640 1 -- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 msgr2=0x5594981859b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f0136ffd640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x5594981859b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f0136ffd640 1 -- 172.21.15.96:0/3638200455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 msgr2=0x559498186580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f0136ffd640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x559498186580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f0136ffd640 1 -- 172.21.15.96:0/3638200455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559498186db0 con 0x559498176860 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.212+0000 7f01367fc640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x5594981859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f0136ffd640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f01300abb90 tx=0x7f013000c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a6ae728d7569a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f011a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300930b0 con 0x559498176860 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f01300b4d00 con 0x559498176860 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c003ca0 con 0x559498176860 2024-09-10T22:58:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01300ab710 con 0x559498176860 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f01403ed740 10 client.27505 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559498188190 con 0x559498176860 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.213+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c003ca0 con 0x559498176860 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f01300aae60 con 0x559498176860 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 10 client.27505 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 10 client.27505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 20 client.27505 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x5594981fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 4 client.27505 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594980ea640 con 0x55949818b240 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01367fc640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x5594981fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f011a7fc640 10 client.27505 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.214+0000 7f01367fc640 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x5594981fa670 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d45b192e005563c server_cookie=de7cf9e344031af8 in_seq=0 out_seq=0 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f011a7fc640 1 -- 172.21.15.96:0/3638200455 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594980ea640 con 0x55949818b240 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f011a7fc640 10 client.27505 handle_command_reply: tid=0 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 client.27505 shutdown 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 msgr2=0x559498185310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f01300abb90 tx=0x7f013000c650 comp rx=0 tx=0).stop 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 shutdown_connections 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55949818b240 0x5594981fa670 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559498179190 0x5594981859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f011a7fc640 0 client.27505 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55949817bbd0 0x559498186580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 --2- 172.21.15.96:0/3638200455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559498176860 0x559498185310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.230+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 >> 172.21.15.96:0/3638200455 conn(0x5594981719b0 msgr2=0x559498170bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.231+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 shutdown_connections 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.231+0000 7f01403ed740 1 -- 172.21.15.96:0/3638200455 wait complete. 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.231+0000 7f01403ed740 20 client.27505 trim_cache size 0 max 16384 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:58:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.231+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 msgr2=0x7f01200b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.231+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 0x7f01200b7850 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f012801b000 tx=0x7f012801c000 comp rx=0 tx=0).stop 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 msgr2=0x7f0138071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f013000ba50 tx=0x7f013000bf10 comp rx=0 tx=0).stop 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 shutdown_connections 2024-09-10T22:58:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f01200b5390 0x7f01200b7850 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0138103370 0x7f0138072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01381029b0 0x7f01380722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 --2- 172.21.15.96:0/1209065469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0138101860 0x7f0138071c60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 >> 172.21.15.96:0/1209065469 conn(0x7f01380fd140 msgr2=0x7f0138105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 shutdown_connections 2024-09-10T22:58:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:23.232+0000 7f013d61f640 1 -- 172.21.15.96:0/1209065469 wait complete. 2024-09-10T22:58:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:23 smithi096 ceph-mon[30869]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.4 MiB/s wr, 701 op/s 2024-09-10T22:58:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:23 smithi121 ceph-mon[35119]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.4 MiB/s wr, 701 op/s 2024-09-10T22:58:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:23 smithi117 ceph-mon[35081]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.4 MiB/s wr, 701 op/s 2024-09-10T22:58:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:25 smithi096 ceph-mon[30869]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.9 MiB/s wr, 730 op/s 2024-09-10T22:58:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:25 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:58:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:25 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:58:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:25 smithi121 ceph-mon[35119]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.9 MiB/s wr, 730 op/s 2024-09-10T22:58:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:25 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:58:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:25 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:58:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:25 smithi117 ceph-mon[35081]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.9 MiB/s wr, 730 op/s 2024-09-10T22:58:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:25 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:58:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:25 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:58:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 650.0 s, 1470.2 tps, lat 67.278 ms stddev 68.986 2024-09-10T22:58:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:27 smithi096 ceph-mon[30869]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.1 MiB/s wr, 824 op/s 2024-09-10T22:58:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:27 smithi121 ceph-mon[35119]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.1 MiB/s wr, 824 op/s 2024-09-10T22:58:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:27 smithi117 ceph-mon[35081]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.1 MiB/s wr, 824 op/s 2024-09-10T22:58:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:29 smithi096 ceph-mon[30869]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:58:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:29 smithi121 ceph-mon[35119]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:58:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:29 smithi117 ceph-mon[35081]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.3 MiB/s wr, 640 op/s 2024-09-10T22:58:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 655.0 s, 1641.2 tps, lat 61.514 ms stddev 63.580 2024-09-10T22:58:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:31 smithi096 ceph-mon[30869]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.1 MiB/s wr, 825 op/s 2024-09-10T22:58:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:31 smithi121 ceph-mon[35119]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.1 MiB/s wr, 825 op/s 2024-09-10T22:58:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:31 smithi117 ceph-mon[35081]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.1 MiB/s wr, 825 op/s 2024-09-10T22:58:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:33 smithi096 ceph-mon[30869]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 623 op/s 2024-09-10T22:58:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:33 smithi121 ceph-mon[35119]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 623 op/s 2024-09-10T22:58:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:33 smithi117 ceph-mon[35081]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 623 op/s 2024-09-10T22:58:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:35 smithi096 ceph-mon[30869]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-09-10T22:58:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:35 smithi121 ceph-mon[35119]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-09-10T22:58:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:35 smithi117 ceph-mon[35081]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-09-10T22:58:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 660.0 s, 2067.0 tps, lat 48.549 ms stddev 54.138 2024-09-10T22:58:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.353+0000 7fc05dc33640 1 -- 172.21.15.96:0/1206037353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058102210 msgr2=0x7fc058106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.353+0000 7fc05dc33640 1 --2- 172.21.15.96:0/1206037353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058102210 0x7fc058106ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc050009ca0 tx=0x7fc05002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 -- 172.21.15.96:0/1206037353 shutdown_connections 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 --2- 172.21.15.96:0/1206037353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058102210 0x7fc058106ce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 --2- 172.21.15.96:0/1206037353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058101670 0x7fc058101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 --2- 172.21.15.96:0/1206037353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc0581010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 -- 172.21.15.96:0/1206037353 >> 172.21.15.96:0/1206037353 conn(0x7fc0580fc880 msgr2=0x7fc0580fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 -- 172.21.15.96:0/1206037353 shutdown_connections 2024-09-10T22:58:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 -- 172.21.15.96:0/1206037353 wait complete. 2024-09-10T22:58:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.354+0000 7fc05dc33640 1 Processor -- start 2024-09-10T22:58:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 -- start start 2024-09-10T22:58:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc05819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 0x7fc05819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc058114c20 con 0x7fc058102210 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc058114aa0 con 0x7fc058101670 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc05dc33640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc058114da0 con 0x7fc058100ce0 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc057fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 0x7fc05819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc057fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45848/0 (socket says 172.21.15.96:45848) 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc057fff640 1 -- 172.21.15.96:0/2101935530 learned_addr learned my addr 172.21.15.96:0/2101935530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.355+0000 7fc0577fe640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc05819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 -- 172.21.15.96:0/2101935530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 msgr2=0x7fc05819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc05819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 -- 172.21.15.96:0/2101935530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 msgr2=0x7fc05819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 0x7fc05819f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 -- 172.21.15.96:0/2101935530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0581a0eb0 con 0x7fc058102210 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc0577fe640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc05819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 0x7fc05819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc057fff640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc050031a20 tx=0x7fc050003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc054ff9640 1 -- 172.21.15.96:0/2101935530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050004240 con 0x7fc058102210 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581a1140 con 0x7fc058102210 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc054ff9640 1 -- 172.21.15.96:0/2101935530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc050003e10 con 0x7fc058102210 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc054ff9640 1 -- 172.21.15.96:0/2101935530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05002fd90 con 0x7fc058102210 2024-09-10T22:58:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.356+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581a7870 con 0x7fc058102210 2024-09-10T22:58:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.359+0000 7fc060a02740 1 Processor -- start 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc054ff9640 1 -- 172.21.15.96:0/2101935530 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc0500043e0 con 0x7fc058102210 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 -- start start 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d428080580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d428080bd0 0x55d428083090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d427fed6f0 con 0x55d42807b860 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d427fdcb20 con 0x55d42807e190 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.360+0000 7fc060a02740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d427fe0930 con 0x55d428080bd0 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d428080580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d428080580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39128/0 (socket says 172.21.15.96:39128) 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45854/0 (socket says 172.21.15.96:45854) 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc0577fe640 1 -- 172.21.15.96:0/2697056333 learned_addr learned my addr 172.21.15.96:0/2697056333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d428080bd0 0x55d428083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc054ff9640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 0x7fc0400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 -- 172.21.15.96:0/2697056333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d428080bd0 msgr2=0x55d428083090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d428080bd0 0x55d428083090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 -- 172.21.15.96:0/2697056333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 msgr2=0x55d428080580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d428080580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 -- 172.21.15.96:0/2697056333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d427fa2530 con 0x55d42807b860 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc054ff9640 1 -- 172.21.15.96:0/2101935530 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0501071f0 con 0x7fc058102210 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc057fff640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 0x7fc0400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0159e0 tx=0x7fc04c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72343dc93eb9b623 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc03dffb640 1 -- 172.21.15.96:0/2697056333 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c00e7e0 con 0x55d42807b860 2024-09-10T22:58:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc03dffb640 1 -- 172.21.15.96:0/2697056333 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c01d070 con 0x55d42807b860 2024-09-10T22:58:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.361+0000 7fc03dffb640 1 -- 172.21.15.96:0/2697056333 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c018af0 con 0x55d42807b860 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 -- 172.21.15.96:0/2697056333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 msgr2=0x55d42807dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc057fff640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 0x7fc0400b77a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc050104c30 tx=0x7fc0500095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0159e0 tx=0x7fc04c009850 comp rx=0 tx=0).stop 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 -- 172.21.15.96:0/2697056333 shutdown_connections 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d428080580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d428080bd0 0x55d428083090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 --2- 172.21.15.96:0/2697056333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807b860 0x55d42807dc50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 -- 172.21.15.96:0/2697056333 >> 172.21.15.96:0/2697056333 conn(0x55d4280769b0 msgr2=0x55d428077740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 -- 172.21.15.96:0/2697056333 shutdown_connections 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.362+0000 7fc060a02740 1 -- 172.21.15.96:0/2697056333 wait complete. 2024-09-10T22:58:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc060a02740 1 -- start start 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc060a02740 1 Processor -- start 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc060a02740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc060a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc060a02740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d4280860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.363+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39142/0 (socket says 172.21.15.96:39142) 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d4280860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53636/0 (socket says 172.21.15.96:53636) 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc060a02740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d427f14ab0 con 0x55d428080bd0 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc060a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d427f82730 con 0x55d42807e190 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc0577fe640 1 -- 172.21.15.96:0/2497440343 learned_addr learned my addr 172.21.15.96:0/2497440343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.364+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d428083b50 con 0x55d42807b860 2024-09-10T22:58:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc060a02740 10 client.0 _ll_register_callbacks cb 0x55d427fd66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 -- 172.21.15.96:0/2497440343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 msgr2=0x55d428084e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 -- 172.21.15.96:0/2497440343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 msgr2=0x55d4280860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d4280860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 -- 172.21.15.96:0/2497440343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d427d3e790 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc0577fe640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc048011a30 tx=0x7fc048011ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1caa1a6b587f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c01d070 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc036ffd640 1 -- 172.21.15.96:0/2497440343 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04801e060 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc036ffd640 1 -- 172.21.15.96:0/2497440343 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0480193f0 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.365+0000 7fc036ffd640 1 -- 172.21.15.96:0/2497440343 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc04800fe70 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 10 client.27455 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55d427db7620 con 0x55d42807e190 2024-09-10T22:58:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc036ffd640 1 -- 172.21.15.96:0/2497440343 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc048023070 con 0x55d42807e190 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 10 client.27455 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 10 client.27455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 20 client.27455 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 4 client.27455 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d427fef640 con 0x55d428090500 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc036ffd640 10 client.27455 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.366+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280928f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=393cb3401b469028 server_cookie=4d288aa00ca947fe in_seq=0 out_seq=0 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc036ffd640 1 -- 172.21.15.96:0/2497440343 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d427fef640 con 0x55d428090500 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc036ffd640 10 client.27455 handle_command_reply: tid=0 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 client.27455 shutdown 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 msgr2=0x55d4280854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc048011a30 tx=0x7fc048011ef0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 shutdown_connections 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280928f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280854d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc036ffd640 0 client.27455 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d4280860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 --2- 172.21.15.96:0/2497440343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428084e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.367+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 >> 172.21.15.96:0/2497440343 conn(0x55d4280769b0 msgr2=0x55d428075bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.368+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 shutdown_connections 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.368+0000 7fc060a02740 1 -- 172.21.15.96:0/2497440343 wait complete. 2024-09-10T22:58:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.368+0000 7fc060a02740 20 client.27455 trim_cache size 0 max 16384 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 Processor -- start 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 -- start start 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428085540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d428079d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b0300 con 0x55d428080bd0 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b1e40 con 0x55d42807e190 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.386+0000 7fc060a02740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b09d0 con 0x55d42807b860 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39148/0 (socket says 172.21.15.96:39148) 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc0577fe640 1 -- 172.21.15.96:0/2417416509 learned_addr learned my addr 172.21.15.96:0/2417416509 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d428079d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428085540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 -- 172.21.15.96:0/2417416509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 msgr2=0x55d428085540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428085540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 -- 172.21.15.96:0/2417416509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 msgr2=0x55d4280797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 -- 172.21.15.96:0/2417416509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d428092c60 con 0x55d428080bd0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc056ffd640 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d428079d30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009e00 tx=0x7fc04c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed9f76c14a1905b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc03d7fa640 1 -- 172.21.15.96:0/2417416509 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c018ac0 con 0x55d428080bd0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc03d7fa640 1 -- 172.21.15.96:0/2417416509 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c00ec90 con 0x55d428080bd0 2024-09-10T22:58:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.387+0000 7fc03d7fa640 1 -- 172.21.15.96:0/2417416509 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c034810 con 0x55d428080bd0 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- 172.21.15.96:0/2417416509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 msgr2=0x55d428079d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d428079d30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009e00 tx=0x7fc04c0046c0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- 172.21.15.96:0/2417416509 shutdown_connections 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42807e190 0x55d4280797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d428080bd0 0x55d428079d30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 --2- 172.21.15.96:0/2417416509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d428085540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- 172.21.15.96:0/2417416509 >> 172.21.15.96:0/2417416509 conn(0x55d4280769b0 msgr2=0x55d428084e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- 172.21.15.96:0/2417416509 shutdown_connections 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- 172.21.15.96:0/2417416509 wait complete. 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.388+0000 7fc060a02740 1 -- start start 2024-09-10T22:58:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 Processor -- start 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d42808bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 0x55d42808a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b1770 con 0x55d42807e190 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45892/0 (socket says 172.21.15.96:45892) 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 -- 172.21.15.96:0/204598457 learned_addr learned my addr 172.21.15.96:0/204598457 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d42808bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b10a0 con 0x55d428080bd0 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581ae320 con 0x55d42807b860 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc056ffd640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 0x55d42808a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 -- 172.21.15.96:0/204598457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 msgr2=0x55d42808bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d42808bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 -- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 msgr2=0x55d42808a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 0x55d42808a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc0577fe640 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d428086410 con 0x55d42807e190 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.389+0000 7fc056ffd640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 0x55d42808a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 10 client.0 _ll_register_callbacks cb 0x7fc04800c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c00ec90 con 0x55d42807e190 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc0577fe640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d010 tx=0x7fc0480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b803e44b2f0c3d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc036ffd640 10 client.17859 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc036ffd640 1 -- 172.21.15.96:0/204598457 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04801e050 con 0x55d42807e190 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc036ffd640 1 -- 172.21.15.96:0/204598457 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc04800fc70 con 0x55d42807e190 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc036ffd640 1 -- 172.21.15.96:0/204598457 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc04800f790 con 0x55d42807e190 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 10 client.17859 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55d4280fd730 con 0x55d42807e190 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc036ffd640 1 -- 172.21.15.96:0/204598457 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc048023040 con 0x55d42807e190 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 10 client.17859 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 20 client.17859 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 4 client.17859 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d427feb0f0 con 0x55d428090500 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.390+0000 7fc056ffd640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.391+0000 7fc056ffd640 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280fff30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f570406965e5d1 server_cookie=b89d7b6df3b1c25b in_seq=0 out_seq=0 2024-09-10T22:58:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.391+0000 7fc036ffd640 10 client.17859 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.420+0000 7fc036ffd640 1 -- 172.21.15.96:0/204598457 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d427feb0f0 con 0x55d428090500 2024-09-10T22:58:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.420+0000 7fc036ffd640 10 client.17859 handle_command_reply: tid=0 2024-09-10T22:58:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.420+0000 7fc060a02740 1 client.17859 shutdown 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.420+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 msgr2=0x55d42808c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.420+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc04800d010 tx=0x7fc0480046c0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 shutdown_connections 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d428090500 0x55d4280fff30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc036ffd640 0 client.17859 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d428080bd0 0x55d42808a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d42807e190 0x55d42808c0f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 --2- 172.21.15.96:0/204598457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d42807b860 0x55d42808bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 >> 172.21.15.96:0/204598457 conn(0x55d4280769b0 msgr2=0x55d428084070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 shutdown_connections 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 1 -- 172.21.15.96:0/204598457 wait complete. 2024-09-10T22:58:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.421+0000 7fc060a02740 20 client.17859 trim_cache size 0 max 16384 2024-09-10T22:58:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:58:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:58:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:58:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 msgr2=0x7fc0400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 0x7fc0400b77a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc050104c30 tx=0x7fc0500095c0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 msgr2=0x7fc0581a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc050031a20 tx=0x7fc050003c20 comp rx=0 tx=0).stop 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 shutdown_connections 2024-09-10T22:58:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc058101670 0x7fc05819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0400b52e0 0x7fc0400b77a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc058102210 0x7fc0581a05f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 --2- 172.21.15.96:0/2101935530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc058100ce0 0x7fc05819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 >> 172.21.15.96:0/2101935530 conn(0x7fc0580fc880 msgr2=0x7fc058102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 shutdown_connections 2024-09-10T22:58:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:36.422+0000 7fc05dc33640 1 -- 172.21.15.96:0/2101935530 wait complete. 2024-09-10T22:58:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:37 smithi096 ceph-mon[30869]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 688 op/s 2024-09-10T22:58:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:37 smithi121 ceph-mon[35119]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 688 op/s 2024-09-10T22:58:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:37 smithi117 ceph-mon[35081]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.3 MiB/s wr, 688 op/s 2024-09-10T22:58:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:39 smithi096 ceph-mon[30869]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-09-10T22:58:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:39 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:39 smithi121 ceph-mon[35119]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-09-10T22:58:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:39 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:39 smithi117 ceph-mon[35081]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 545 op/s 2024-09-10T22:58:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:39 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:58:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 665.0 s, 2306.8 tps, lat 43.474 ms stddev 50.097 2024-09-10T22:58:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-10T22:58:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:41.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:41.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:41.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:58:41.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:58:41.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:58:43.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:42 smithi096 ceph-mon[30869]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:58:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:42 smithi121 ceph-mon[35119]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:58:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:42 smithi117 ceph-mon[35081]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-10T22:58:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:44 smithi096 ceph-mon[30869]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-10T22:58:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:44 smithi121 ceph-mon[35119]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-10T22:58:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:44 smithi117 ceph-mon[35081]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-10T22:58:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 670.0 s, 2197.6 tps, lat 45.201 ms stddev 49.932 2024-09-10T22:58:47.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:46 smithi096 ceph-mon[30869]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-09-10T22:58:47.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:46 smithi121 ceph-mon[35119]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-09-10T22:58:47.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:46 smithi117 ceph-mon[35081]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 746 op/s 2024-09-10T22:58:49.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:48 smithi096 ceph-mon[30869]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2024-09-10T22:58:49.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:48 smithi121 ceph-mon[35119]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2024-09-10T22:58:49.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:48 smithi117 ceph-mon[35081]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- 172.21.15.96:0/1271799020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac106460 msgr2=0x7fc7ac106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 --2- 172.21.15.96:0/1271799020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac106460 0x7fc7ac106910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009ca0 tx=0x7fc7a402a8b0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- 172.21.15.96:0/1271799020 shutdown_connections 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 --2- 172.21.15.96:0/1271799020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac106460 0x7fc7ac106910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 --2- 172.21.15.96:0/1271799020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac1038b0 0x7fc7ac105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 --2- 172.21.15.96:0/1271799020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac06b470 0x7fc7ac103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- 172.21.15.96:0/1271799020 >> 172.21.15.96:0/1271799020 conn(0x7fc7ac0fd140 msgr2=0x7fc7ac0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- 172.21.15.96:0/1271799020 shutdown_connections 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- 172.21.15.96:0/1271799020 wait complete. 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 Processor -- start 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.547+0000 7fc7b403c640 1 -- start start 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 0x7fc7ac19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac1038b0 0x7fc7ac19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 0x7fc7ac1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac114be0 con 0x7fc7ac106460 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac114a60 con 0x7fc7ac1038b0 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b403c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac114d60 con 0x7fc7ac06b470 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 0x7fc7ac1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 0x7fc7ac1a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49654/0 (socket says 172.21.15.96:49654) 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 0x7fc7ac19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac1038b0 0x7fc7ac19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b25d9640 1 -- 172.21.15.96:0/3213496462 learned_addr learned my addr 172.21.15.96:0/3213496462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/3213496462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac1038b0 msgr2=0x7fc7ac19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac1038b0 0x7fc7ac19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 msgr2=0x7fc7ac1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 0x7fc7ac1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/3213496462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac1a0e50 con 0x7fc7ac06b470 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 0x7fc7ac19f380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c970 tx=0x7fc7a800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.548+0000 7fc7a2ffd640 1 -- 172.21.15.96:0/3213496462 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8007bb0 con 0x7fc7ac06b470 2024-09-10T22:58:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.549+0000 7fc7a2ffd640 1 -- 172.21.15.96:0/3213496462 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a8007d50 con 0x7fc7ac06b470 2024-09-10T22:58:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.549+0000 7fc7a2ffd640 1 -- 172.21.15.96:0/3213496462 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8004df0 con 0x7fc7ac06b470 2024-09-10T22:58:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.549+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1a1140 con 0x7fc7ac06b470 2024-09-10T22:58:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.549+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1a7810 con 0x7fc7ac06b470 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.552+0000 7fc7b6e08740 1 Processor -- start 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 -- start start 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131571580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131574090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f1314de6f0 con 0x55f13156f190 2024-09-10T22:58:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1314cdb20 con 0x55f131571bd0 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f1314d1930 con 0x55f13156c860 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131574090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37464/0 (socket says 172.21.15.96:37464) 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b25d9640 1 -- 172.21.15.96:0/1191143434 learned_addr learned my addr 172.21.15.96:0/1191143434 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7a2ffd640 1 -- 172.21.15.96:0/3213496462 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc7a8020060 con 0x7fc7ac06b470 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1191143434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 msgr2=0x55f13156ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1191143434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 msgr2=0x55f131571580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131571580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1191143434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f131493530 con 0x55f131571bd0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.553+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131574090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009c20 tx=0x7fc79c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a773499c694c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc797fff640 1 -- 172.21.15.96:0/1191143434 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004050 con 0x55f131571bd0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc797fff640 1 -- 172.21.15.96:0/1191143434 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc79c0054d0 con 0x55f131571bd0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc797fff640 1 -- 172.21.15.96:0/1191143434 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004cd0 con 0x55f131571bd0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc7a2ffd640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 0x7fc7980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 0x7fc7980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.554+0000 7fc7a2ffd640 1 -- 172.21.15.96:0/3213496462 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc7a80e5070 con 0x7fc7ac06b470 2024-09-10T22:58:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 0x7fc7980b7820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4030480 tx=0x7fc7a4030800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1191143434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 msgr2=0x55f131574090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131574090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009c20 tx=0x7fc79c011810 comp rx=0 tx=0).stop 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1191143434 shutdown_connections 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131574090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131571580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1191143434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f13156ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1191143434 >> 172.21.15.96:0/1191143434 conn(0x55f1315679b0 msgr2=0x55f131568740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1191143434 shutdown_connections 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1191143434 wait complete. 2024-09-10T22:58:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.555+0000 7fc7b6e08740 1 -- start start 2024-09-10T22:58:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 Processor -- start 2024-09-10T22:58:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f131571bd0 0x55f131577170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f131405ab0 con 0x55f131571bd0 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f131473730 con 0x55f13156f190 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f131574bb0 con 0x55f13156c860 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.556+0000 7fc7b6e08740 10 client.0 _ll_register_callbacks cb 0x55f1314c76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.557+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.557+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43312/0 (socket says 172.21.15.96:43312) 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.557+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1443141392 learned_addr learned my addr 172.21.15.96:0/1443141392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.557+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f131571bd0 0x55f131577170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1443141392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 msgr2=0x55f131575f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1443141392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f131571bd0 msgr2=0x55f131577170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f131571bd0 0x55f131577170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 -- 172.21.15.96:0/1443141392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13122f790 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc79c01b040 tx=0x7fc79c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c05932ec3edad202 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.558+0000 7fc7957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c029070 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc79c003ca0 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c0054d0 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc79c003ea0 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 10 client.27470 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f1312a8620 con 0x55f13156f190 2024-09-10T22:58:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc79c035070 con 0x55f13156f190 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc79c0234c0 con 0x55f13156f190 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 10 client.27470 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 10 client.27470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 20 client.27470 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f131581630 0x55f131583a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 4 client.27470 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.559+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1314e0640 con 0x55f131581630 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.560+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f131581630 0x55f131583a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.560+0000 7fc7957fa640 10 client.27470 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.560+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f131581630 0x55f131583a20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c7b290e61b8296 server_cookie=966b42a6de1f1f1c in_seq=0 out_seq=0 2024-09-10T22:58:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7957fa640 1 -- 172.21.15.96:0/1443141392 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1314e0640 con 0x55f131581630 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7957fa640 10 client.27470 handle_command_reply: tid=0 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 client.27470 shutdown 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 msgr2=0x55f1315765a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc79c01b040 tx=0x7fc79c0038a0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 shutdown_connections 2024-09-10T22:58:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f131581630 0x55f131583a20 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f13156f190 0x55f1315765a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7957fa640 0 client.27470 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f131571bd0 0x55f131577170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1443141392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 >> 172.21.15.96:0/1443141392 conn(0x55f1315679b0 msgr2=0x55f131566bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 shutdown_connections 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1443141392 wait complete. 2024-09-10T22:58:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.561+0000 7fc7b6e08740 20 client.27470 trim_cache size 0 max 16384 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.580+0000 7fc7b6e08740 1 Processor -- start 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 -- start start 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131576440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b0400 con 0x55f13156f190 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b1f40 con 0x55f131571bd0 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b0ad0 con 0x55f13156c860 2024-09-10T22:58:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131576440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b25d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43324/0 (socket says 172.21.15.96:43324) 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131576440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49690/0 (socket says 172.21.15.96:49690) 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b25d9640 1 -- 172.21.15.96:0/2755908701 learned_addr learned my addr 172.21.15.96:0/2755908701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37482/0 (socket says 172.21.15.96:37482) 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/2755908701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 msgr2=0x55f131575160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/2755908701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 msgr2=0x55f131576440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131576440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/2755908701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f131583d90 con 0x55f13156c860 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.581+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80e2c60 tx=0x7fc7a80e5370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b4e55605a6751 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7977fe640 1 -- 172.21.15.96:0/2755908701 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a801d040 con 0x55f13156c860 2024-09-10T22:58:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7977fe640 1 -- 172.21.15.96:0/2755908701 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a80ab490 con 0x55f13156c860 2024-09-10T22:58:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7977fe640 1 -- 172.21.15.96:0/2755908701 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80e70d0 con 0x55f13156c860 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 -- 172.21.15.96:0/2755908701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 msgr2=0x55f131575f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80e2c60 tx=0x7fc7a80e5370 comp rx=0 tx=0).stop 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 -- 172.21.15.96:0/2755908701 shutdown_connections 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f131575160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156f190 0x55f131576440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/2755908701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156c860 0x55f131575f00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 -- 172.21.15.96:0/2755908701 >> 172.21.15.96:0/2755908701 conn(0x55f1315679b0 msgr2=0x55f131571fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 -- 172.21.15.96:0/2755908701 shutdown_connections 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.582+0000 7fc7b6e08740 1 -- 172.21.15.96:0/2755908701 wait complete. 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 -- start start 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 Processor -- start 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156f190 0x55f13157afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f13157bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b1870 con 0x55f13156c860 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b11a0 con 0x55f131571bd0 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ae3e0 con 0x55f13156f190 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b6e08740 10 client.0 _ll_register_callbacks cb 0x7fc79c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b15d7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156f190 0x55f13157afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b1dd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.583+0000 7fc7b1dd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49692/0 (socket says 172.21.15.96:49692) 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/1486884456 learned_addr learned my addr 172.21.15.96:0/1486884456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b25d9640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f13157bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/1486884456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156f190 msgr2=0x55f13157afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156f190 0x55f13157afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 msgr2=0x55f13157bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f13157bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 -- 172.21.15.96:0/1486884456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13157c3d0 con 0x55f13156c860 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7b1dd8640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c940 tx=0x7fc7a80e5460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fd96f1678ceb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.584+0000 7fc7957fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80c6060 con 0x55f13156c860 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc7a80e72d0 con 0x55f13156c860 2024-09-10T22:58:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a80ab490 con 0x55f13156c860 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7a80cd050 con 0x55f13156c860 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 10 client.17874 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f13157d7e0 con 0x55f13156c860 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a80b03b0 con 0x55f13156c860 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc7a80a72d0 con 0x55f13156c860 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 10 client.17874 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 20 client.17874 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f1315ee4c0 0x55f1315f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 4 client.17874 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc7a80ab490 con 0x55f1315ee4c0 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.585+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f1315ee4c0 0x55f1315f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.586+0000 7fc7957fa640 10 client.17874 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.586+0000 7fc7b15d7640 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f1315ee4c0 0x55f1315f08e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ef2a98d1f2e8e server_cookie=9b3fdafb0d862858 in_seq=0 out_seq=0 2024-09-10T22:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.600+0000 7fc7957fa640 1 -- 172.21.15.96:0/1486884456 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc7a80ab490 con 0x55f1315ee4c0 2024-09-10T22:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.600+0000 7fc7957fa640 10 client.17874 handle_command_reply: tid=0 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 client.17874 shutdown 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 msgr2=0x55f13157a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c940 tx=0x7fc7a80e5460 comp rx=0 tx=0).stop 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 shutdown_connections 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f1315ee4c0 0x55f1315f08e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f131571bd0 0x55f13157bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f13156f190 0x55f13157afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7957fa640 0 client.17874 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 --2- 172.21.15.96:0/1486884456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f13156c860 0x55f13157a930 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 >> 172.21.15.96:0/1486884456 conn(0x55f1315679b0 msgr2=0x55f1315663d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 shutdown_connections 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 1 -- 172.21.15.96:0/1486884456 wait complete. 2024-09-10T22:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.601+0000 7fc7b6e08740 20 client.17874 trim_cache size 0 max 16384 2024-09-10T22:58:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.602+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 msgr2=0x7fc7980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.602+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 0x7fc7980b7820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4030480 tx=0x7fc7a4030800 comp rx=0 tx=0).stop 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.602+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 msgr2=0x7fc7ac19f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.602+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 0x7fc7ac19f380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c970 tx=0x7fc7a800ce30 comp rx=0 tx=0).stop 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 shutdown_connections 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7ac1038b0 0x7fc7ac19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc7980b5360 0x7fc7980b7820 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7ac106460 0x7fc7ac1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 --2- 172.21.15.96:0/3213496462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac06b470 0x7fc7ac19f380 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.603+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 >> 172.21.15.96:0/3213496462 conn(0x7fc7ac0fd140 msgr2=0x7fc7ac101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.604+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 shutdown_connections 2024-09-10T22:58:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:58:49.604+0000 7fc7b403c640 1 -- 172.21.15.96:0/3213496462 wait complete. 2024-09-10T22:58:49.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:58:50.105 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/3357260186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd281093d0 msgr2=0x7fbd281097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/3357260186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd281093d0 0x7fbd281097b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009900 tx=0x7fbd1802a410 comp rx=0 tx=0).stop 2024-09-10T22:58:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/3357260186 shutdown_connections 2024-09-10T22:58:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/3357260186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd2810ede0 0x7fbd2810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/3357260186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810f760 0x7fbd28115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/3357260186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd281093d0 0x7fbd281097b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/3357260186 >> 172.21.15.96:0/3357260186 conn(0x7fbd28078a40 msgr2=0x7fbd2807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/3357260186 shutdown_connections 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.710+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/3357260186 wait complete. 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 Processor -- start 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 -- start start 2024-09-10T22:58:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 0x7fbd2819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd2810f760 0x7fbd281a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd28119370 con 0x7fbd2810f760 2024-09-10T22:58:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281191f0 con 0x7fbd281093d0 2024-09-10T22:58:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd2e6ce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281194f0 con 0x7fbd2810ede0 2024-09-10T22:58:50.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37508/0 (socket says 172.21.15.96:37508) 2024-09-10T22:58:50.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.711+0000 7fbd277fe640 1 -- 172.21.15.96:0/576108055 learned_addr learned my addr 172.21.15.96:0/576108055 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:50.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 0x7fbd2819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd2810f760 0x7fbd281a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 -- 172.21.15.96:0/576108055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 msgr2=0x7fbd2819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 -- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd2810f760 msgr2=0x7fbd281a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd2810f760 0x7fbd281a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.712+0000 7fbd27fff640 1 -- 172.21.15.96:0/576108055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd281a5950 con 0x7fbd281093d0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd277fe640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd27fff640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 0x7fbd2819f8a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802a930 tx=0x7fbd18004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd257fa640 1 -- 172.21.15.96:0/576108055 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1803f030 con 0x7fbd281093d0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd257fa640 1 -- 172.21.15.96:0/576108055 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd18003aa0 con 0x7fbd281093d0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd281a5be0 con 0x7fbd281093d0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd257fa640 1 -- 172.21.15.96:0/576108055 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18035070 con 0x7fbd281093d0 2024-09-10T22:58:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.713+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd281ac300 con 0x7fbd281093d0 2024-09-10T22:58:50.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.715+0000 7fbd257fa640 1 -- 172.21.15.96:0/576108055 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbd1804c060 con 0x7fbd281093d0 2024-09-10T22:58:50.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.716+0000 7fbd257fa640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 0x7fbd100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.716+0000 7fbd30aa3740 1 Processor -- start 2024-09-10T22:58:50.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.716+0000 7fbd30aa3740 1 -- start start 2024-09-10T22:58:50.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.716+0000 7fbd257fa640 1 -- 172.21.15.96:0/576108055 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd18105a80 con 0x7fbd281093d0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.716+0000 7fbd277fe640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 0x7fbd100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbd0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 0x56367bbd2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56367bb3ae50 con 0x56367bbc40a0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56367bb2a280 con 0x56367bbd0950 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56367bb2e090 con 0x56367bbc49c0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49704/0 (socket says 172.21.15.96:49704) 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 -- 172.21.15.96:0/592063471 learned_addr learned my addr 172.21.15.96:0/592063471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd277fe640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbd0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 0x56367bbd2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd277fe640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 0x7fbd100b7820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbd2810ea30 tx=0x7fbd1c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 -- 172.21.15.96:0/592063471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 msgr2=0x56367bbd0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbd0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 -- 172.21.15.96:0/592063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 msgr2=0x56367bbd2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 0x56367bbd2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 -- 172.21.15.96:0/592063471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56367b888650 con 0x56367bbc40a0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd277fe640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbd0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 0x56367bbd2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd27fff640 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd180c8960 tx=0x7fbd18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4778c23f744ff4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd0e7fc640 1 -- 172.21.15.96:0/592063471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18049040 con 0x56367bbc40a0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.717+0000 7fbd0e7fc640 1 -- 172.21.15.96:0/592063471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd180d51b0 con 0x56367bbc40a0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd0e7fc640 1 -- 172.21.15.96:0/592063471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180cc850 con 0x56367bbc40a0 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 -- 172.21.15.96:0/592063471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 msgr2=0x56367bbc4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd180c8960 tx=0x7fbd18005fd0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 -- 172.21.15.96:0/592063471 shutdown_connections 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbd0950 0x56367bbd2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbd0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/592063471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc40a0 0x56367bbc4480 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 -- 172.21.15.96:0/592063471 >> 172.21.15.96:0/592063471 conn(0x56367bbc1ed0 msgr2=0x56367bbc3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 -- 172.21.15.96:0/592063471 shutdown_connections 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.718+0000 7fbd30aa3740 1 -- 172.21.15.96:0/592063471 wait complete. 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.719+0000 7fbd30aa3740 1 -- start start 2024-09-10T22:58:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.719+0000 7fbd30aa3740 1 Processor -- start 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.719+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.719+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbcd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.719+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bbcdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd2cc6b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bbcdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56367b98b700 con 0x56367bbc49c0 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56367bac5110 con 0x56367bbc40a0 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56367bbd3390 con 0x56367bbd0950 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43386/0 (socket says 172.21.15.96:43386) 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd30aa3740 10 client.0 _ll_register_callbacks cb 0x56367bade990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.720+0000 7fbd27fff640 1 -- 172.21.15.96:0/448419273 learned_addr learned my addr 172.21.15.96:0/448419273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd277fe640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbcd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 -- 172.21.15.96:0/448419273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 msgr2=0x56367bbcdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bbcdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 -- 172.21.15.96:0/448419273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 msgr2=0x56367bbcd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbcd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 -- 172.21.15.96:0/448419273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367baf2c40 con 0x56367bbc40a0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.721+0000 7fbd27fff640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd1803d040 tx=0x7fbd18031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f2c87976c5ddc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd077fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180f3020 con 0x56367bbc40a0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56367bb3cda0 con 0x56367bbc40a0 2024-09-10T22:58:50.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd180f31c0 con 0x56367bbc40a0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd18049070 con 0x56367bbc40a0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd30aa3740 10 client.27488 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.722+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56367bbdec20 con 0x56367bbc40a0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.723+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd180cc390 con 0x56367bbc40a0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.723+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbd180ef0d0 con 0x56367bbc40a0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.723+0000 7fbd30aa3740 10 client.27488 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.723+0000 7fbd30aa3740 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.723+0000 7fbd30aa3740 20 client.27488 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.724+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bbe0170 0x56367bbe2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.724+0000 7fbd30aa3740 4 client.27488 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.724+0000 7fbd277fe640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bbe0170 0x56367bbe2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.724+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56367bb3cda0 con 0x56367bbe0170 2024-09-10T22:58:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.725+0000 7fbd077fe640 10 client.27488 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.725+0000 7fbd277fe640 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bbe0170 0x56367bbe2560 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d7ac03255eee79 server_cookie=f17fcae333c69878 in_seq=0 out_seq=0 2024-09-10T22:58:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.725+0000 7fbd077fe640 1 -- 172.21.15.96:0/448419273 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56367bb3cda0 con 0x56367bbe0170 2024-09-10T22:58:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.725+0000 7fbd077fe640 10 client.27488 handle_command_reply: tid=0 2024-09-10T22:58:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.725+0000 7fbd30aa3740 1 client.27488 shutdown 2024-09-10T22:58:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 msgr2=0x56367bbd0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd1803d040 tx=0x7fbd18031a10 comp rx=0 tx=0).stop 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 shutdown_connections 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bbe0170 0x56367bbe2560 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bbcdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbcd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/448419273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbd0270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 >> 172.21.15.96:0/448419273 conn(0x56367bbc1ed0 msgr2=0x56367bbc7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd077fe640 0 client.27488 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 shutdown_connections 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 1 -- 172.21.15.96:0/448419273 wait complete. 2024-09-10T22:58:50.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.726+0000 7fbd30aa3740 20 client.27488 trim_cache size 0 max 16384 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 Processor -- start 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 -- start start 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbdc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbcf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 0x56367bbd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbcf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbcf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37538/0 (socket says 172.21.15.96:37538) 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd277fe640 1 -- 172.21.15.96:0/1449880156 learned_addr learned my addr 172.21.15.96:0/1449880156 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b4930 con 0x56367bbd0950 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b6470 con 0x56367bbc40a0 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.754+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b5000 con 0x56367bbc49c0 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 0x56367bbd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 -- 172.21.15.96:0/1449880156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 msgr2=0x56367bbcf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbcf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 -- 172.21.15.96:0/1449880156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 msgr2=0x56367bbdc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbdc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.756+0000 7fbd2cc6b640 1 -- 172.21.15.96:0/1449880156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56367bbcfea0 con 0x56367bbd0950 2024-09-10T22:58:50.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.758+0000 7fbd2cc6b640 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 0x56367bbd1d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd200039c0 tx=0x7fbd200096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4360d3d20a4c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.758+0000 7fbd0dffb640 1 -- 172.21.15.96:0/1449880156 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2001f020 con 0x56367bbd0950 2024-09-10T22:58:50.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.758+0000 7fbd0dffb640 1 -- 172.21.15.96:0/1449880156 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd20005020 con 0x56367bbd0950 2024-09-10T22:58:50.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.758+0000 7fbd0dffb640 1 -- 172.21.15.96:0/1449880156 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20004860 con 0x56367bbd0950 2024-09-10T22:58:50.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.759+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 msgr2=0x56367bbd1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.759+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 0x56367bbd1d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd200039c0 tx=0x7fbd200096a0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.760+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 shutdown_connections 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.760+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbd0950 0x56367bbd1d40 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.760+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbc49c0 0x56367bbcf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.760+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/1449880156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbdc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.760+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 >> 172.21.15.96:0/1449880156 conn(0x56367bbc1ed0 msgr2=0x56367bbcecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.768+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 shutdown_connections 2024-09-10T22:58:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.768+0000 7fbd30aa3740 1 -- 172.21.15.96:0/1449880156 wait complete. 2024-09-10T22:58:50.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.768+0000 7fbd30aa3740 1 -- start start 2024-09-10T22:58:50.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.769+0000 7fbd30aa3740 1 Processor -- start 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.769+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbe4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.769+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd30aa3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bc4bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b5da0 con 0x56367bbc49c0 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b56d0 con 0x56367bbc40a0 2024-09-10T22:58:50.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd30aa3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281b2910 con 0x56367bbd0950 2024-09-10T22:58:50.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd30aa3740 10 client.0 _ll_register_callbacks cb 0x7fbd20009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:50.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49722/0 (socket says 172.21.15.96:49722) 2024-09-10T22:58:50.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.770+0000 7fbd277fe640 1 -- 172.21.15.96:0/181430606 learned_addr learned my addr 172.21.15.96:0/181430606 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.771+0000 7fbd277fe640 1 -- 172.21.15.96:0/181430606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 msgr2=0x56367bc4bf10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:58:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.771+0000 7fbd277fe640 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bc4bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.771+0000 7fbd277fe640 1 -- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 msgr2=0x56367bbe4b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:58:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.771+0000 7fbd277fe640 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbe4b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.771+0000 7fbd277fe640 1 -- 172.21.15.96:0/181430606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56367bc4c450 con 0x56367bbc49c0 2024-09-10T22:58:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.772+0000 7fbd277fe640 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0058c0 tx=0x7fbd1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b65310f1381e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.775+0000 7fbd077fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:58:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.775+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c004020 con 0x56367bbc49c0 2024-09-10T22:58:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.775+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd1c0041c0 con 0x56367bbc49c0 2024-09-10T22:58:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.777+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd20005020 con 0x56367bbc49c0 2024-09-10T22:58:50.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.782+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd1c0043c0 con 0x56367bbc49c0 2024-09-10T22:58:50.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.783+0000 7fbd30aa3740 10 client.17892 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:50.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.783+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56367bc4d7c0 con 0x56367bbc49c0 2024-09-10T22:58:50.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd20005020 con 0x56367bbc49c0 2024-09-10T22:58:50.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbd1c052480 con 0x56367bbc49c0 2024-09-10T22:58:50.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 10 client.17892 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:50.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:58:50.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 20 client.17892 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:50.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bc4df50 0x56367bc50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:50.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 4 client.17892 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:58:50.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.791+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56367bb38850 con 0x56367bc4df50 2024-09-10T22:58:50.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.796+0000 7fbd27fff640 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bc4df50 0x56367bc50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:50.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.807+0000 7fbd077fe640 10 client.17892 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:50.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.807+0000 7fbd27fff640 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bc4df50 0x56367bc50370 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5fe49c7bb4573af server_cookie=445518134bebfb2e in_seq=0 out_seq=0 2024-09-10T22:58:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.808+0000 7fbd077fe640 1 -- 172.21.15.96:0/181430606 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x56367bb38850 con 0x56367bc4df50 2024-09-10T22:58:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.808+0000 7fbd077fe640 10 client.17892 handle_command_reply: tid=0 2024-09-10T22:58:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.808+0000 7fbd30aa3740 1 client.17892 shutdown 2024-09-10T22:58:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.809+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 msgr2=0x56367bbe50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.809+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0058c0 tx=0x7fbd1c004730 comp rx=0 tx=0).stop 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.810+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 shutdown_connections 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.810+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56367bc4df50 0x56367bc50370 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.810+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56367bbd0950 0x56367bc4bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56367bbc49c0 0x56367bbe50b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 1 --2- 172.21.15.96:0/181430606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56367bbc40a0 0x56367bbe4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd077fe640 0 client.17892 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 >> 172.21.15.96:0/181430606 conn(0x56367bbc1ed0 msgr2=0x56367bbc0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 shutdown_connections 2024-09-10T22:58:50.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 1 -- 172.21.15.96:0/181430606 wait complete. 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T22:58:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:58:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:58:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:58:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:58:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:58:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:58:50.823 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.811+0000 7fbd30aa3740 20 client.17892 trim_cache size 0 max 16384 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 msgr2=0x7fbd100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 0x7fbd100b7820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbd2810ea30 tx=0x7fbd1c00b480 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 msgr2=0x7fbd2819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 0x7fbd2819f8a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802a930 tx=0x7fbd18004270 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 shutdown_connections 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd100b5360 0x7fbd100b7820 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd2810f760 0x7fbd281a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd2810ede0 0x7fbd2819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 --2- 172.21.15.96:0/576108055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd281093d0 0x7fbd2819f8a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.812+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 >> 172.21.15.96:0/576108055 conn(0x7fbd28078a40 msgr2=0x7fbd28079340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.813+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 shutdown_connections 2024-09-10T22:58:50.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:50.814+0000 7fbd2e6ce640 1 -- 172.21.15.96:0/576108055 wait complete. 2024-09-10T22:58:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 675.0 s, 2040.8 tps, lat 48.970 ms stddev 51.133 2024-09-10T22:58:51.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:50 smithi096 ceph-mon[30869]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-10T22:58:51.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:50 smithi121 ceph-mon[35119]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-10T22:58:51.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:50 smithi117 ceph-mon[35081]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-10T22:58:51.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:58:51.625 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:58:51.977 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.559+0000 7fe2c52c8640 1 -- 172.21.15.96:0/3668162837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 msgr2=0x7fe2c010f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.559+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/3668162837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 0x7fe2c010f7f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009ca0 tx=0x7fe2b802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 -- 172.21.15.96:0/3668162837 shutdown_connections 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/3668162837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 0x7fe2c010f7f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/3668162837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0108700 0x7fe2c0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/3668162837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0100c90 0x7fe2c0101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 -- 172.21.15.96:0/3668162837 >> 172.21.15.96:0/3668162837 conn(0x7fe2c00fc8c0 msgr2=0x7fe2c00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 -- 172.21.15.96:0/3668162837 shutdown_connections 2024-09-10T22:58:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.560+0000 7fe2c52c8640 1 -- 172.21.15.96:0/3668162837 wait complete. 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.561+0000 7fe2c52c8640 1 Processor -- start 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.561+0000 7fe2c52c8640 1 -- start start 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.561+0000 7fe2c52c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.561+0000 7fe2c52c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0108700 0x7fe2c006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.561+0000 7fe2c52c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 0x7fe2c0072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2c52c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0112c20 con 0x7fe2c0100c90 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2c52c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0112aa0 con 0x7fe2c0108700 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2c52c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0112da0 con 0x7fe2c01090a0 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49740/0 (socket says 172.21.15.96:49740) 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 -- 172.21.15.96:0/2155157918 learned_addr learned my addr 172.21.15.96:0/2155157918 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:52.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2b7fff640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0108700 0x7fe2c006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 -- 172.21.15.96:0/2155157918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 msgr2=0x7fe2c0072740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 0x7fe2c0072740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 -- 172.21.15.96:0/2155157918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0108700 msgr2=0x7fe2c006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0108700 0x7fe2c006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 -- 172.21.15.96:0/2155157918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0072d00 con 0x7fe2c0100c90 2024-09-10T22:58:52.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.562+0000 7fe2beffd640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9f0 tx=0x7fe2b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.563+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000cd80 con 0x7fe2c0100c90 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.563+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b00108e0 con 0x7fe2c0100c90 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.564+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00188e0 con 0x7fe2c0100c90 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.564+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c0072ff0 con 0x7fe2c0100c90 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.564+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c00734e0 con 0x7fe2c0100c90 2024-09-10T22:58:52.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.565+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c0101070 con 0x7fe2c0100c90 2024-09-10T22:58:52.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.568+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe2b0010430 con 0x7fe2c0100c90 2024-09-10T22:58:52.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.569+0000 7fe2bcff9640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 0x7fe2ac0b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:52.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.570+0000 7fe2b7fff640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 0x7fe2ac0b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:52.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.570+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe2b00e63f0 con 0x7fe2c0100c90 2024-09-10T22:58:52.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.570+0000 7fe2b7fff640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 0x7fe2ac0b7a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00737d0 tx=0x7fe2a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:52.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.575+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe2b0026020 con 0x7fe2c0100c90 2024-09-10T22:58:52.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:52 smithi096 ceph-mon[30869]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 MiB/s wr, 638 op/s 2024-09-10T22:58:52.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.902+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c002e5c0 con 0x7fe2c0100c90 2024-09-10T22:58:52.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.908+0000 7fe2bcff9640 1 -- 172.21.15.96:0/2155157918 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fe2b00ad330 con 0x7fe2c0100c90 2024-09-10T22:58:52.911 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:58:52.911 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:58:52.912 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:58:52.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 msgr2=0x7fe2ac0b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 0x7fe2ac0b7a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00737d0 tx=0x7fe2a8005f90 comp rx=0 tx=0).stop 2024-09-10T22:58:52.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 msgr2=0x7fe2c006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:52.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9f0 tx=0x7fe2b000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 shutdown_connections 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c0108700 0x7fe2c006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe2ac0b5560 0x7fe2ac0b7a20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c01090a0 0x7fe2c0072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 --2- 172.21.15.96:0/2155157918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c0100c90 0x7fe2c006ce60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:52.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.920+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 >> 172.21.15.96:0/2155157918 conn(0x7fe2c00fc8c0 msgr2=0x7fe2c00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:52.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.923+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 shutdown_connections 2024-09-10T22:58:52.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:52.923+0000 7fe2c52c8640 1 -- 172.21.15.96:0/2155157918 wait complete. 2024-09-10T22:58:53.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:52 smithi121 ceph-mon[35119]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 MiB/s wr, 638 op/s 2024-09-10T22:58:53.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:52 smithi117 ceph-mon[35081]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 MiB/s wr, 638 op/s 2024-09-10T22:58:53.731 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:58:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:53 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2155157918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:54.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:53 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2155157918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:54.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:53 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2155157918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:58:54.114 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:54.803 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:54 smithi096 ceph-mon[30869]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T22:58:54.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.824+0000 7fd1c3fff640 1 -- 172.21.15.96:0/3158080999 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80318a0 con 0x7fd1cc10a150 2024-09-10T22:58:54.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.825+0000 7fd1d37cd640 1 -- 172.21.15.96:0/3158080999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 msgr2=0x7fd1cc1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.825+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/3158080999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 0x7fd1cc1111b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c802a4a0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.825+0000 7fd1d37cd640 1 -- 172.21.15.96:0/3158080999 shutdown_connections 2024-09-10T22:58:54.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/3158080999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc10a530 0x7fd1cc10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/3158080999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/3158080999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 0x7fd1cc1111b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 -- 172.21.15.96:0/3158080999 >> 172.21.15.96:0/3158080999 conn(0x7fd1cc06dfc0 msgr2=0x7fd1cc06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 -- 172.21.15.96:0/3158080999 shutdown_connections 2024-09-10T22:58:54.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.826+0000 7fd1d37cd640 1 -- 172.21.15.96:0/3158080999 wait complete. 2024-09-10T22:58:54.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.827+0000 7fd1d37cd640 1 Processor -- start 2024-09-10T22:58:54.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.827+0000 7fd1d37cd640 1 -- start start 2024-09-10T22:58:54.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 0x7fd1cc115520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111d50 con 0x7fd1cc10a150 2024-09-10T22:58:54.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d0d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d0d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37574/0 (socket says 172.21.15.96:37574) 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d0d68640 1 -- 172.21.15.96:0/978713243 learned_addr learned my addr 172.21.15.96:0/978713243 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111bd0 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc111ed0 con 0x7fd1cc10a530 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.828+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d1d6a640 1 -- 172.21.15.96:0/978713243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 msgr2=0x7fd1cc115a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d1d6a640 1 -- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 msgr2=0x7fd1cc115520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 0x7fd1cc115520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d1d6a640 1 -- 172.21.15.96:0/978713243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc11b400 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.829+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:54.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.830+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11ad90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400be00 tx=0x7fd1c400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.831+0000 7fd1c27fc640 1 -- 172.21.15.96:0/978713243 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4019070 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.831+0000 7fd1c27fc640 1 -- 172.21.15.96:0/978713243 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c40148f0 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.831+0000 7fd1c27fc640 1 -- 172.21.15.96:0/978713243 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4019070 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.831+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc11b6f0 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.831+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1bd490 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.835+0000 7fd1d5ba2740 1 Processor -- start 2024-09-10T22:58:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.837+0000 7fd1c27fc640 1 -- 172.21.15.96:0/978713243 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd1c4014440 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.837+0000 7fd1d5ba2740 1 -- start start 2024-09-10T22:58:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e68df0 0x561c26e74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c26ddeea0 con 0x561c26e74d80 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c26dce2d0 con 0x561c26e68df0 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561c26dd20e0 con 0x561c26e684d0 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:37586/0 (socket says 172.21.15.96:37586) 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 -- 172.21.15.96:0/213302570 learned_addr learned my addr 172.21.15.96:0/213302570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 -- 172.21.15.96:0/213302570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e68df0 msgr2=0x561c26e74840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:58:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e68df0 0x561c26e74840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.838+0000 7fd1d0d68640 1 -- 172.21.15.96:0/213302570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 msgr2=0x561c26e77170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1d0d68640 1 -- 172.21.15.96:0/213302570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c26b2c650 con 0x561c26e684d0 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1c27fc640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 0x7fd1b80b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009f40 tx=0x7fd1bc009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605ab448447a9fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1b77fe640 1 -- 172.21.15.96:0/213302570 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc004230 con 0x561c26e684d0 2024-09-10T22:58:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.839+0000 7fd1b77fe640 1 -- 172.21.15.96:0/213302570 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc017070 con 0x561c26e684d0 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1c27fc640 1 -- 172.21.15.96:0/978713243 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1c40e51f0 con 0x7fd1cc11c0a0 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/213302570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 msgr2=0x561c26e688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009f40 tx=0x7fd1bc009710 comp rx=0 tx=0).stop 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/213302570 shutdown_connections 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e68df0 0x561c26e74840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e77170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/213302570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e684d0 0x561c26e688b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.840+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/213302570 >> 172.21.15.96:0/213302570 conn(0x561c26e66300 msgr2=0x561c26e674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.841+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/213302570 shutdown_connections 2024-09-10T22:58:54.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.841+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/213302570 wait complete. 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.841+0000 7fd1d5ba2740 1 -- start start 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.841+0000 7fd1d1569640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 0x7fd1b80b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.841+0000 7fd1d5ba2740 1 Processor -- start 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 0x561c26e71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e80dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c26c2f700 con 0x561c26e684d0 2024-09-10T22:58:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c26d69160 con 0x561c26e74d80 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561c26e77a50 con 0x561c26e72370 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d5ba2740 10 client.0 _ll_register_callbacks cb 0x561c26d829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49786/0 (socket says 172.21.15.96:49786) 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d1569640 1 -- 172.21.15.96:0/1184131341 learned_addr learned my addr 172.21.15.96:0/1184131341 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 0x561c26e71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d1569640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 0x7fd1b80b79d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80096c0 tx=0x7fd1c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.842+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e80dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 -- 172.21.15.96:0/1184131341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 msgr2=0x561c26e80dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e80dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 -- 172.21.15.96:0/1184131341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 msgr2=0x561c26e747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 -- 172.21.15.96:0/1184131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c26d96c90 con 0x561c26e74d80 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 0x561c26e71c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009f10 tx=0x7fd1bc005240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393c4b94661b946f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc02a060 con 0x561c26e74d80 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1bc0037f0 con 0x561c26e74d80 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1bc017070 con 0x561c26e74d80 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc009bb0 con 0x561c26e74d80 2024-09-10T22:58:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d5ba2740 10 client.27503 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.843+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c26e838f0 con 0x561c26e74d80 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc017070 con 0x561c26e74d80 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd1bc02f070 con 0x561c26e74d80 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 10 client.27503 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 10 client.27503 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 20 client.27503 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26e84c40 0x561c26e87030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 4 client.27503 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c26de0df0 con 0x561c26e84c40 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26e84c40 0x561c26e87030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1b4ff9640 10 client.27503 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:54.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.844+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26e84c40 0x561c26e87030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d2afe982eb663a0 server_cookie=e86b6c0ca75b61a7 in_seq=0 out_seq=0 2024-09-10T22:58:54.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1184131341 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c26de0df0 con 0x561c26e84c40 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1b4ff9640 10 client.27503 handle_command_reply: tid=0 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 client.27503 shutdown 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 msgr2=0x561c26e71c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 0x561c26e71c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009f10 tx=0x7fd1bc005240 comp rx=0 tx=0).stop 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 shutdown_connections 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26e84c40 0x561c26e87030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e74d80 0x561c26e71c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e80dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1184131341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e684d0 0x561c26e747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 >> 172.21.15.96:0/1184131341 conn(0x561c26e66300 msgr2=0x561c26e6bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.845+0000 7fd1b4ff9640 0 client.27503 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:54.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.846+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 shutdown_connections 2024-09-10T22:58:54.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.846+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1184131341 wait complete. 2024-09-10T22:58:54.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.846+0000 7fd1d5ba2740 20 client.27503 trim_cache size 0 max 16384 2024-09-10T22:58:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.862+0000 7fd1d5ba2740 1 Processor -- start 2024-09-10T22:58:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.862+0000 7fd1d5ba2740 1 -- start start 2024-09-10T22:58:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e6ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c5980 con 0x561c26e74d80 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c74c0 con 0x561c26e684d0 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c6050 con 0x561c26e72370 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43444/0 (socket says 172.21.15.96:43444) 2024-09-10T22:58:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 -- 172.21.15.96:0/222182804 learned_addr learned my addr 172.21.15.96:0/222182804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 -- 172.21.15.96:0/222182804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 msgr2=0x561c26e81280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e81280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 -- 172.21.15.96:0/222182804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 msgr2=0x561c26e6ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e6ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.863+0000 7fd1d1569640 1 -- 172.21.15.96:0/222182804 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c26e6d8e0 con 0x561c26e684d0 2024-09-10T22:58:54.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.864+0000 7fd1d1569640 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c803a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5634d26bf33101e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.864+0000 7fd1b6ffd640 1 -- 172.21.15.96:0/222182804 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035040 con 0x561c26e684d0 2024-09-10T22:58:54.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.864+0000 7fd1b6ffd640 1 -- 172.21.15.96:0/222182804 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c8054070 con 0x561c26e684d0 2024-09-10T22:58:54.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.864+0000 7fd1b6ffd640 1 -- 172.21.15.96:0/222182804 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c804f670 con 0x561c26e684d0 2024-09-10T22:58:54.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.865+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/222182804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 msgr2=0x561c26e80d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.865+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c803a000 comp rx=0 tx=0).stop 2024-09-10T22:58:54.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/222182804 shutdown_connections 2024-09-10T22:58:54.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e6ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e81280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/222182804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e80d40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/222182804 >> 172.21.15.96:0/222182804 conn(0x561c26e66300 msgr2=0x561c26e694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/222182804 shutdown_connections 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/222182804 wait complete. 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 -- start start 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.866+0000 7fd1d5ba2740 1 Processor -- start 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e87910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c6df0 con 0x561c26e74d80 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c6720 con 0x561c26e684d0 2024-09-10T22:58:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1c3960 con 0x561c26e72370 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d5ba2740 10 client.0 _ll_register_callbacks cb 0x7fd1bc009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43454/0 (socket says 172.21.15.96:43454) 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 -- 172.21.15.96:0/1388482825 learned_addr learned my addr 172.21.15.96:0/1388482825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1d6a640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e87910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 -- 172.21.15.96:0/1388482825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 msgr2=0x561c26e87910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e87910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 -- 172.21.15.96:0/1388482825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 msgr2=0x561c26e87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.867+0000 7fd1d1569640 1 -- 172.21.15.96:0/1388482825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c26e88680 con 0x561c26e684d0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d1569640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8031870 tx=0x7fd1c802ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8120cb351bb04ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:54.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035070 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c8054070 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd1c80035f0 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd1c80037f0 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 10 client.27509 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c26ef1810 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c8054070 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd1c8058420 con 0x561c26e684d0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 10 client.27509 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 10 client.27509 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 20 client.27509 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26ef1ee0 0x561c26ef4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 4 client.27509 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.868+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c26ddc8a0 con 0x561c26ef1ee0 2024-09-10T22:58:54.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26ef1ee0 0x561c26ef4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1b4ff9640 10 client.27509 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d0d68640 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26ef1ee0 0x561c26ef4300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c8167763d599cafd server_cookie=441316e0f23b0c77 in_seq=0 out_seq=0 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1b4ff9640 1 -- 172.21.15.96:0/1388482825 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c26ddc8a0 con 0x561c26ef1ee0 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1b4ff9640 10 client.27509 handle_command_reply: tid=0 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 client.27509 shutdown 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 msgr2=0x561c26e8a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8031870 tx=0x7fd1c802ac20 comp rx=0 tx=0).stop 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 shutdown_connections 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x561c26ef1ee0 0x561c26ef4300 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c26e74d80 0x561c26e87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561c26e72370 0x561c26e87910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1b4ff9640 0 client.27509 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 --2- 172.21.15.96:0/1388482825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c26e684d0 0x561c26e8a420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.869+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 >> 172.21.15.96:0/1388482825 conn(0x561c26e66300 msgr2=0x561c26e73260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 shutdown_connections 2024-09-10T22:58:54.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d5ba2740 1 -- 172.21.15.96:0/1388482825 wait complete. 2024-09-10T22:58:54.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d5ba2740 20 client.27509 trim_cache size 0 max 16384 2024-09-10T22:58:54.876 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 msgr2=0x7fd1b80b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 0x7fd1b80b79d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80096c0 tx=0x7fd1c8031400 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 msgr2=0x7fd1cc11ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11ad90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400be00 tx=0x7fd1c400bef0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 shutdown_connections 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc11c0a0 0x7fd1cc11ad90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd1b80b5510 0x7fd1b80b79d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1cc10a530 0x7fd1cc115a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 --2- 172.21.15.96:0/978713243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1cc10a150 0x7fd1cc115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:54.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 >> 172.21.15.96:0/978713243 conn(0x7fd1cc06dfc0 msgr2=0x7fd1cc072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:54.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 shutdown_connections 2024-09-10T22:58:54.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:54.870+0000 7fd1d37cd640 1 -- 172.21.15.96:0/978713243 wait complete. 2024-09-10T22:58:55.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:54 smithi121 ceph-mon[35119]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T22:58:55.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:54 smithi117 ceph-mon[35081]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T22:58:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 680.0 s, 2027.0 tps, lat 49.332 ms stddev 56.273 2024-09-10T22:58:56.336 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:58:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:56.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:56 smithi096 ceph-mon[30869]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-10T22:58:57.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:56 smithi121 ceph-mon[35119]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-10T22:58:57.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:56 smithi117 ceph-mon[35081]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-09-10T22:58:57.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/4239681859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 msgr2=0x7fe0e0113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/4239681859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e0113500 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009900 tx=0x7fe0cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/4239681859 shutdown_connections 2024-09-10T22:58:57.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/4239681859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 0x7fe0e0115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/4239681859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e0113500 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/4239681859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e0107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/4239681859 >> 172.21.15.96:0/4239681859 conn(0x7fe0e0076f20 msgr2=0x7fe0e0077330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.222+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/4239681859 shutdown_connections 2024-09-10T22:58:57.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.223+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/4239681859 wait complete. 2024-09-10T22:58:57.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 Processor -- start 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 -- start start 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 0x7fe0e01a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01193b0 con 0x7fe0e0107fe0 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0119230 con 0x7fe0e01076c0 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e6cb4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0119530 con 0x7fe0e0113a40 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e4a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 0x7fe0e01a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e019fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50450/0 (socket says 172.21.15.96:50450) 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.224+0000 7fe0dffff640 1 -- 172.21.15.96:0/14750145 learned_addr learned my addr 172.21.15.96:0/14750145 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 -- 172.21.15.96:0/14750145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 msgr2=0x7fe0e01a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 0x7fe0e01a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 -- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 msgr2=0x7fe0e019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 -- 172.21.15.96:0/14750145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e01a59d0 con 0x7fe0e01076c0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e4a50640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e019f920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0ddffb640 1 -- 172.21.15.96:0/14750145 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400cd80 con 0x7fe0e01076c0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e01a5cc0 con 0x7fe0e01076c0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0ddffb640 1 -- 172.21.15.96:0/14750145 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d40035f0 con 0x7fe0e01076c0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0ddffb640 1 -- 172.21.15.96:0/14750145 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4010690 con 0x7fe0e01076c0 2024-09-10T22:58:57.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.225+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e01ac550 con 0x7fe0e01076c0 2024-09-10T22:58:57.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0ddffb640 1 -- 172.21.15.96:0/14750145 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe0d4004000 con 0x7fe0e01076c0 2024-09-10T22:58:57.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 Processor -- start 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 -- start start 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557a5b445ea0 con 0x557a5b4cfdf0 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a5b4352d0 con 0x557a5b4dbd80 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e9089740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a5b4390e0 con 0x557a5b4cf4d0 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e4a50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49270/0 (socket says 172.21.15.96:49270) 2024-09-10T22:58:57.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.228+0000 7fe0e5251640 1 -- 172.21.15.96:0/2628349588 learned_addr learned my addr 172.21.15.96:0/2628349588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0dffff640 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0e5251640 1 -- 172.21.15.96:0/2628349588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 msgr2=0x557a5b4de170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0e5251640 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4de170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0e5251640 1 -- 172.21.15.96:0/2628349588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 msgr2=0x557a5b4db840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0e5251640 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4db840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.229+0000 7fe0e5251640 1 -- 172.21.15.96:0/2628349588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a5b193650 con 0x557a5b4cf4d0 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0e5251640 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009f50 tx=0x7fe0d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95136a10071ef00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0ddffb640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 0x7fe0d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0e4a50640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 0x7fe0d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0caffd640 1 -- 172.21.15.96:0/2628349588 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004d20 con 0x557a5b4cf4d0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0caffd640 1 -- 172.21.15.96:0/2628349588 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d8004050 con 0x557a5b4cf4d0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.230+0000 7fe0caffd640 1 -- 172.21.15.96:0/2628349588 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8005280 con 0x557a5b4cf4d0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0ddffb640 1 -- 172.21.15.96:0/14750145 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0d40e5e90 con 0x7fe0e01076c0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 -- 172.21.15.96:0/2628349588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 msgr2=0x557a5b4cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e4a50640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 0x7fe0d00b77a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40a98c0 tx=0x7fe0d40e9600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009f50 tx=0x7fe0d80118d0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 -- 172.21.15.96:0/2628349588 shutdown_connections 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4de170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 --2- 172.21.15.96:0/2628349588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4cf8b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 -- 172.21.15.96:0/2628349588 >> 172.21.15.96:0/2628349588 conn(0x557a5b4cd300 msgr2=0x557a5b4ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 -- 172.21.15.96:0/2628349588 shutdown_connections 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.231+0000 7fe0e9089740 1 -- 172.21.15.96:0/2628349588 wait complete. 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 -- start start 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 Processor -- start 2024-09-10T22:58:57.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557a5b296700 con 0x557a5b4dbd80 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a5b3d0160 con 0x557a5b4cf4d0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a5b4dea00 con 0x557a5b4cfdf0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.232+0000 7fe0e9089740 10 client.0 _ll_register_callbacks cb 0x557a5b3e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60252/0 (socket says 172.21.15.96:60252) 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 -- 172.21.15.96:0/32441773 learned_addr learned my addr 172.21.15.96:0/32441773 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0dffff640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 -- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 msgr2=0x557a5b4d8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 -- 172.21.15.96:0/32441773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 msgr2=0x557a5b4d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 -- 172.21.15.96:0/32441773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a5b3fdc90 con 0x557a5b4cf4d0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e5251640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009d60 tx=0x7fe0d8004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9f001cd9074a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0bffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8004a00 con 0x557a5b4cf4d0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0d8004050 con 0x557a5b4cf4d0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.233+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0d801f040 con 0x557a5b4cf4d0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0d8004ba0 con 0x557a5b4cf4d0 2024-09-10T22:58:57.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 10 client.27521 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a5b4ea3b0 con 0x557a5b4cf4d0 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d8004050 con 0x557a5b4cf4d0 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe0d800f480 con 0x557a5b4cf4d0 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 10 client.27521 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 10 client.27521 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 20 client.27521 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b4eb700 0x557a5b4edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 4 client.27521 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a5b447df0 con 0x557a5b4eb700 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.234+0000 7fe0dffff640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b4eb700 0x557a5b4edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.235+0000 7fe0bffff640 10 client.27521 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:57.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.235+0000 7fe0dffff640 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b4eb700 0x557a5b4edaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81506f9f8a6132cb server_cookie=31750a94303f80cf in_seq=0 out_seq=0 2024-09-10T22:58:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.236+0000 7fe0bffff640 1 -- 172.21.15.96:0/32441773 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a5b447df0 con 0x557a5b4eb700 2024-09-10T22:58:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.236+0000 7fe0bffff640 10 client.27521 handle_command_reply: tid=0 2024-09-10T22:58:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 client.27521 shutdown 2024-09-10T22:58:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 msgr2=0x557a5b4db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009d60 tx=0x7fe0d8004950 comp rx=0 tx=0).stop 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 shutdown_connections 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b4eb700 0x557a5b4edaf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 --2- 172.21.15.96:0/32441773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4db6f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 >> 172.21.15.96:0/32441773 conn(0x557a5b4cd300 msgr2=0x557a5b4d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0bffff640 0 client.27521 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 shutdown_connections 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.237+0000 7fe0e9089740 1 -- 172.21.15.96:0/32441773 wait complete. 2024-09-10T22:58:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.238+0000 7fe0e9089740 20 client.27521 trim_cache size 0 max 16384 2024-09-10T22:58:57.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 Processor -- start 2024-09-10T22:58:57.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 -- start start 2024-09-10T22:58:57.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b4980 con 0x557a5b4dbd80 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b64c0 con 0x557a5b4cf4d0 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e9089740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b5050 con 0x557a5b4cfdf0 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60260/0 (socket says 172.21.15.96:60260) 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e5251640 1 -- 172.21.15.96:0/1111052092 learned_addr learned my addr 172.21.15.96:0/1111052092 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0e5251640 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.254+0000 7fe0dffff640 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 -- 172.21.15.96:0/1111052092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 msgr2=0x557a5b4d8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 -- 172.21.15.96:0/1111052092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 msgr2=0x557a5b4d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 -- 172.21.15.96:0/1111052092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a5b4cdd80 con 0x557a5b4cf4d0 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0e5251640 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8010010 tx=0x7fe0d80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb15aeee4ccbbb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0ca7fc640 1 -- 172.21.15.96:0/1111052092 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800f480 con 0x557a5b4cf4d0 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0ca7fc640 1 -- 172.21.15.96:0/1111052092 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d800f620 con 0x557a5b4cf4d0 2024-09-10T22:58:57.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.255+0000 7fe0ca7fc640 1 -- 172.21.15.96:0/1111052092 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80278f0 con 0x557a5b4cf4d0 2024-09-10T22:58:57.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- 172.21.15.96:0/1111052092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 msgr2=0x557a5b4e7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8010010 tx=0x7fe0d80118d0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- 172.21.15.96:0/1111052092 shutdown_connections 2024-09-10T22:58:57.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4dbd80 0x557a5b4d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cfdf0 0x557a5b4d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 --2- 172.21.15.96:0/1111052092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e7c30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- 172.21.15.96:0/1111052092 >> 172.21.15.96:0/1111052092 conn(0x557a5b4cd300 msgr2=0x557a5b4d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- 172.21.15.96:0/1111052092 shutdown_connections 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- 172.21.15.96:0/1111052092 wait complete. 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 -- start start 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.256+0000 7fe0e9089740 1 Processor -- start 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b5df0 con 0x557a5b4cfdf0 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b5720 con 0x557a5b4dbd80 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e01b2980 con 0x557a5b4cf4d0 2024-09-10T22:58:57.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e9089740 10 client.0 _ll_register_callbacks cb 0x7fe0d8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e5251640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49286/0 (socket says 172.21.15.96:49286) 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e5251640 1 -- 172.21.15.96:0/800262347 learned_addr learned my addr 172.21.15.96:0/800262347 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0dffff640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.257+0000 7fe0e5251640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 -- 172.21.15.96:0/800262347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 msgr2=0x557a5b4dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 -- 172.21.15.96:0/800262347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 msgr2=0x557a5b4dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 -- 172.21.15.96:0/800262347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a5b4dd860 con 0x557a5b4cf4d0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0e5251640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009bb0 tx=0x7fe0d8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de68ce44ebdc6392 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0bffff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:57.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800f480 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.258+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe0d803e470 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0d800f620 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0d803e670 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 10 client.27586 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a5b558380 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d800f620 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe0d803b040 con 0x557a5b4cf4d0 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 10 client.27586 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 10 client.27586 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 20 client.27586 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b5589a0 0x557a5b55adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:57.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 4 client.27586 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a5b4438a0 con 0x557a5b5589a0 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.259+0000 7fe0dffff640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b5589a0 0x557a5b55adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0bffff640 10 client.27586 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0dffff640 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b5589a0 0x557a5b55adc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e1cde1b1a01bdff server_cookie=2533d45a46a61979 in_seq=0 out_seq=0 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0bffff640 1 -- 172.21.15.96:0/800262347 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a5b4438a0 con 0x557a5b5589a0 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0bffff640 10 client.27586 handle_command_reply: tid=0 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 client.27586 shutdown 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 msgr2=0x557a5b4e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009bb0 tx=0x7fe0d8005fd0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 shutdown_connections 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x557a5b5589a0 0x557a5b55adc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a5b4dbd80 0x557a5b4dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557a5b4cfdf0 0x557a5b4dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 --2- 172.21.15.96:0/800262347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a5b4cf4d0 0x557a5b4e42d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 >> 172.21.15.96:0/800262347 conn(0x557a5b4cd300 msgr2=0x557a5b4cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0bffff640 0 client.27586 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 shutdown_connections 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.260+0000 7fe0e9089740 1 -- 172.21.15.96:0/800262347 wait complete. 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e9089740 20 client.27586 trim_cache size 0 max 16384 2024-09-10T22:58:57.267 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:57.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 msgr2=0x7fe0d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 0x7fe0d00b77a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe0d40a98c0 tx=0x7fe0d40e9600 comp rx=0 tx=0).stop 2024-09-10T22:58:57.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 msgr2=0x7fe0e019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:57.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e019f920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 shutdown_connections 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe0d00b52e0 0x7fe0d00b77a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0113a40 0x7fe0e01a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0e0107fe0 0x7fe0e019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 --2- 172.21.15.96:0/14750145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e01076c0 0x7fe0e019f920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 >> 172.21.15.96:0/14750145 conn(0x7fe0e0076f20 msgr2=0x7fe0e0114ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 shutdown_connections 2024-09-10T22:58:57.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:57.261+0000 7fe0e6cb4640 1 -- 172.21.15.96:0/14750145 wait complete. 2024-09-10T22:58:58.120 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:58:58.493 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:58:58.929 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:58:58 smithi096 ceph-mon[30869]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-10T22:58:59.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:58:58 smithi121 ceph-mon[35119]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-10T22:58:59.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:58:58 smithi117 ceph-mon[35081]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2024-09-10T22:58:59.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.044+0000 7f82d81fe640 1 -- 172.21.15.96:0/3478837369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d0101a90 msgr2=0x7f82d0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.044+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3478837369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d0101a90 0x7f82d0101e70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a440 comp rx=0 tx=0).stop 2024-09-10T22:58:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.046+0000 7f82d81fe640 1 -- 172.21.15.96:0/3478837369 shutdown_connections 2024-09-10T22:58:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.046+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3478837369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d0106ed0 0x7f82d0107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.046+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3478837369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d01023b0 0x7f82d0106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.046+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3478837369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d0101a90 0x7f82d0101e70 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a440 comp rx=0 tx=0).stop 2024-09-10T22:58:59.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.046+0000 7f82d81fe640 1 -- 172.21.15.96:0/3478837369 >> 172.21.15.96:0/3478837369 conn(0x7f82d00fd700 msgr2=0x7f82d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.051+0000 7f82d81fe640 1 -- 172.21.15.96:0/3478837369 shutdown_connections 2024-09-10T22:58:59.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.054+0000 7f82d81fe640 1 -- 172.21.15.96:0/3478837369 wait complete. 2024-09-10T22:58:59.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.055+0000 7f82d81fe640 1 Processor -- start 2024-09-10T22:58:59.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 -- start start 2024-09-10T22:58:59.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 0x7f82d01088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d010e1b0 0x7f82d010e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82d006e3c0 con 0x7f82d0106ed0 2024-09-10T22:58:59.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82d006e240 con 0x7f82d01023b0 2024-09-10T22:58:59.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.057+0000 7f82d81fe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82d006e540 con 0x7f82d010e1b0 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.058+0000 7f82d5799640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.058+0000 7f82d5799640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50498/0 (socket says 172.21.15.96:50498) 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.058+0000 7f82d5799640 1 -- 172.21.15.96:0/3770354550 learned_addr learned my addr 172.21.15.96:0/3770354550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.058+0000 7f82d679b640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d010e1b0 0x7f82d010e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.058+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 0x7f82d01088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 -- 172.21.15.96:0/3770354550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d010e1b0 msgr2=0x7f82d010e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d010e1b0 0x7f82d010e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 -- 172.21.15.96:0/3770354550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 msgr2=0x7f82d01088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 0x7f82d01088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 -- 172.21.15.96:0/3770354550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d010ed90 con 0x7f82d0106ed0 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 0x7f82d01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.059+0000 7f82d5799640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d750 tx=0x7f82c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.060+0000 7f82c6ffd640 1 -- 172.21.15.96:0/3770354550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0017070 con 0x7f82d0106ed0 2024-09-10T22:58:59.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.060+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d0115350 con 0x7f82d0106ed0 2024-09-10T22:58:59.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.060+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01158e0 con 0x7f82d0106ed0 2024-09-10T22:58:59.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.060+0000 7f82c6ffd640 1 -- 172.21.15.96:0/3770354550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c0004020 con 0x7f82d0106ed0 2024-09-10T22:58:59.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.060+0000 7f82c6ffd640 1 -- 172.21.15.96:0/3770354550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0020060 con 0x7f82d0106ed0 2024-09-10T22:58:59.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.063+0000 7f82da5d3740 1 Processor -- start 2024-09-10T22:58:59.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.063+0000 7f82da5d3740 1 -- start start 2024-09-10T22:58:59.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 0x5642e21254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e21319a0 0x5642e2133d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642e209bea0 con 0x5642e21250f0 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642e208b2d0 con 0x5642e21319a0 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82da5d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642e208f0e0 con 0x5642e2125a10 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49314/0 (socket says 172.21.15.96:49314) 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 -- 172.21.15.96:0/1824069748 learned_addr learned my addr 172.21.15.96:0/1824069748 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d679b640 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 0x5642e21254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 -- 172.21.15.96:0/1824069748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e21319a0 msgr2=0x5642e2133d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e21319a0 0x5642e2133d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 -- 172.21.15.96:0/1824069748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 msgr2=0x5642e21254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 0x5642e21254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d5f9a640 1 -- 172.21.15.96:0/1824069748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e1de9650 con 0x5642e2125a10 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.064+0000 7f82d679b640 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 0x5642e21254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.065+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82cc031440 tx=0x7f82cc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12b8a1f5e467c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.065+0000 7f82bbfff640 1 -- 172.21.15.96:0/1824069748 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc035070 con 0x5642e2125a10 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.065+0000 7f82bbfff640 1 -- 172.21.15.96:0/1824069748 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc0046c0 con 0x5642e2125a10 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.065+0000 7f82bbfff640 1 -- 172.21.15.96:0/1824069748 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0044e0 con 0x5642e2125a10 2024-09-10T22:58:59.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.065+0000 7f82c6ffd640 1 -- 172.21.15.96:0/3770354550 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f82c00041d0 con 0x7f82d0106ed0 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- 172.21.15.96:0/1824069748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 msgr2=0x5642e2131460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82cc031440 tx=0x7f82cc0042f0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- 172.21.15.96:0/1824069748 shutdown_connections 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e21319a0 0x5642e2133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2125a10 0x5642e2131460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 --2- 172.21.15.96:0/1824069748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21250f0 0x5642e21254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- 172.21.15.96:0/1824069748 >> 172.21.15.96:0/1824069748 conn(0x5642e2122f20 msgr2=0x5642e21240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- 172.21.15.96:0/1824069748 shutdown_connections 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82c6ffd640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 0x7f82bc0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- 172.21.15.96:0/1824069748 wait complete. 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82c6ffd640 1 -- 172.21.15.96:0/3770354550 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82c00e53d0 con 0x7f82d0106ed0 2024-09-10T22:58:59.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.066+0000 7f82da5d3740 1 -- start start 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 Processor -- start 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 0x7f82bc0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e212e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e212ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642e1eec700 con 0x5642e21319a0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642e2026160 con 0x5642e2125a10 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642e21345c0 con 0x5642e21250f0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.067+0000 7f82da5d3740 10 client.0 _ll_register_callbacks cb 0x5642e203f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 0x7f82bc0b7760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0095a0 tx=0x7f82cc005a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49320/0 (socket says 172.21.15.96:49320) 2024-09-10T22:58:59.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2494086024 learned_addr learned my addr 172.21.15.96:0/2494086024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:59.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5799640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e212e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d679b640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e212ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 msgr2=0x5642e212e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e212e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2494086024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 msgr2=0x5642e212ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e212ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.068+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2494086024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e2053c90 con 0x5642e21250f0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.069+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5642e21260f0 tx=0x7f82cc004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c1e11318387ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.070+0000 7f82b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc035030 con 0x5642e21250f0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82cc0316a0 con 0x5642e21250f0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc0046c0 con 0x5642e21250f0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82cc0318a0 con 0x5642e21250f0 2024-09-10T22:58:59.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82da5d3740 10 client.27595 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e213ff10 con 0x5642e21250f0 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.071+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc0046c0 con 0x5642e21250f0 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f82cc058ac0 con 0x5642e21250f0 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 10 client.27595 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 10 client.27595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 20 client.27595 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e2141260 0x5642e2143650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 4 client.27595 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e209ddf0 con 0x5642e2141260 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82d5799640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e2141260 0x5642e2143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82b97fa640 10 client.27595 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:59.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.072+0000 7f82d5799640 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e2141260 0x5642e2143650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a26d358ed6105cc server_cookie=407c062d1b0418c0 in_seq=0 out_seq=0 2024-09-10T22:58:59.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.073+0000 7f82b97fa640 1 -- 172.21.15.96:0/2494086024 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642e209ddf0 con 0x5642e2141260 2024-09-10T22:58:59.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.073+0000 7f82b97fa640 10 client.27595 handle_command_reply: tid=0 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.073+0000 7f82da5d3740 1 client.27595 shutdown 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.073+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 msgr2=0x5642e21312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.073+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5642e21260f0 tx=0x7f82cc004260 comp rx=0 tx=0).stop 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 shutdown_connections 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e2141260 0x5642e2143650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e212e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e212ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2494086024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21250f0 0x5642e21312c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82b97fa640 0 client.27595 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 >> 172.21.15.96:0/2494086024 conn(0x5642e2122f20 msgr2=0x5642e2128a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 shutdown_connections 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 1 -- 172.21.15.96:0/2494086024 wait complete. 2024-09-10T22:58:59.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.074+0000 7f82da5d3740 20 client.27595 trim_cache size 0 max 16384 2024-09-10T22:58:59.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.090+0000 7f82da5d3740 1 Processor -- start 2024-09-10T22:58:59.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.090+0000 7f82da5d3740 1 -- start start 2024-09-10T22:58:59.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21319a0 0x5642e212d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 0x5642e2133220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400e320 con 0x5642e2125a10 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400fe60 con 0x5642e2132d90 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82da5d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400e9f0 con 0x5642e21319a0 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50534/0 (socket says 172.21.15.96:50534) 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2653084473 learned_addr learned my addr 172.21.15.96:0/2653084473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5799640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21319a0 0x5642e212d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d679b640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 0x5642e2133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2653084473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21319a0 msgr2=0x5642e212d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21319a0 0x5642e212d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2653084473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 msgr2=0x5642e2133220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 0x5642e2133220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 -- 172.21.15.96:0/2653084473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e21236c0 con 0x5642e2125a10 2024-09-10T22:58:59.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d679b640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 0x5642e2133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:58:59.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005ab0 tx=0x7f82cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7ff2f96254b51d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.091+0000 7f82bb7fe640 1 -- 172.21.15.96:0/2653084473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc035040 con 0x5642e2125a10 2024-09-10T22:58:59.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82bb7fe640 1 -- 172.21.15.96:0/2653084473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc053070 con 0x5642e2125a10 2024-09-10T22:58:59.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82bb7fe640 1 -- 172.21.15.96:0/2653084473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc058c60 con 0x5642e2125a10 2024-09-10T22:58:59.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 -- 172.21.15.96:0/2653084473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 msgr2=0x5642e2122760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f82cc005ab0 tx=0x7f82cc005fd0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 -- 172.21.15.96:0/2653084473 shutdown_connections 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2132d90 0x5642e2133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e21319a0 0x5642e212d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2653084473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e2125a10 0x5642e2122760 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 -- 172.21.15.96:0/2653084473 >> 172.21.15.96:0/2653084473 conn(0x5642e2122f20 msgr2=0x5642e212cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 -- 172.21.15.96:0/2653084473 shutdown_connections 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.092+0000 7f82da5d3740 1 -- 172.21.15.96:0/2653084473 wait complete. 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 -- start start 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 Processor -- start 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e213a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e21348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400f790 con 0x5642e21319a0 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400f0c0 con 0x5642e2125a10 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82b400c300 con 0x5642e2132d90 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.093+0000 7f82da5d3740 10 client.0 _ll_register_callbacks cb 0x7f82cc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.094+0000 7f82d679b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.094+0000 7f82d679b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49330/0 (socket says 172.21.15.96:49330) 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.094+0000 7f82d679b640 1 -- 172.21.15.96:0/2167634871 learned_addr learned my addr 172.21.15.96:0/2167634871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.094+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e213a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.094+0000 7f82d5799640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e21348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 -- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 msgr2=0x5642e213a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e213a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 -- 172.21.15.96:0/2167634871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 msgr2=0x5642e21348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e21348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 -- 172.21.15.96:0/2167634871 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e2135610 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82d679b640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82c8012a70 tx=0x7f82c8012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3cff1b9e2d0d366 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8020060 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f82c8010c00 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.095+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc053070 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82c8010e00 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 10 client.27601 fetch_fsmap learned FSMap version 32 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e21aef50 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc053070 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f82c8025070 con 0x5642e2132d90 2024-09-10T22:58:59.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 10 client.27601 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 10 client.27601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 20 client.27601 populate_metadata read hostname 'smithi096' 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e21af730 0x5642e21b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 4 client.27601 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.096+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642e20998a0 con 0x5642e21af730 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.097+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e21af730 0x5642e21b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.097+0000 7f82d5f9a640 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e21af730 0x5642e21b1b50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fda8aef0a44bbafe server_cookie=f99b5ed7a875c49c in_seq=0 out_seq=0 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.097+0000 7f82b97fa640 10 client.27601 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82b97fa640 1 -- 172.21.15.96:0/2167634871 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642e20998a0 con 0x5642e21af730 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82b97fa640 10 client.27601 handle_command_reply: tid=0 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 client.27601 shutdown 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 msgr2=0x5642e2134de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82c8012a70 tx=0x7f82c8012e70 comp rx=0 tx=0).stop 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 shutdown_connections 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5642e21af730 0x5642e21b1b50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642e2132d90 0x5642e2134de0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642e21319a0 0x5642e21348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82b97fa640 0 client.27601 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 --2- 172.21.15.96:0/2167634871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5642e2125a10 0x5642e213a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 >> 172.21.15.96:0/2167634871 conn(0x5642e2122f20 msgr2=0x5642e212cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 shutdown_connections 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 1 -- 172.21.15.96:0/2167634871 wait complete. 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.098+0000 7f82da5d3740 20 client.27601 trim_cache size 0 max 16384 2024-09-10T22:58:59.105 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:58:59.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 msgr2=0x7f82bc0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 0x7f82bc0b7760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0095a0 tx=0x7f82cc005a00 comp rx=0 tx=0).stop 2024-09-10T22:58:59.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 msgr2=0x7f82d0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:58:59.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d750 tx=0x7f82c000dc10 comp rx=0 tx=0).stop 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 shutdown_connections 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f82bc0b52a0 0x7f82bc0b7760 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d010e1b0 0x7f82d010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82d0106ed0 0x7f82d0108e10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 --2- 172.21.15.96:0/3770354550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d01023b0 0x7f82d01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 >> 172.21.15.96:0/3770354550 conn(0x7f82d00fd700 msgr2=0x7f82d0104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 shutdown_connections 2024-09-10T22:58:59.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:58:59.099+0000 7f82d81fe640 1 -- 172.21.15.96:0/3770354550 wait complete. 2024-09-10T22:58:59.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:59:00.291 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:00 smithi096 ceph-mon[30869]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 749 op/s 2024-09-10T22:59:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 685.0 s, 2171.4 tps, lat 46.072 ms stddev 51.953 2024-09-10T22:59:01.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:00 smithi121 ceph-mon[35119]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 749 op/s 2024-09-10T22:59:01.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:00 smithi117 ceph-mon[35081]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 749 op/s 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 -- 172.21.15.96:0/4109970898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f58108020 msgr2=0x7f1f5810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 --2- 172.21.15.96:0/4109970898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f58108020 0x7f1f5810e820 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009ca0 tx=0x7f1f5002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 -- 172.21.15.96:0/4109970898 shutdown_connections 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 --2- 172.21.15.96:0/4109970898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f58108020 0x7f1f5810e820 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 --2- 172.21.15.96:0/4109970898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f581076a0 0x7f1f58107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 --2- 172.21.15.96:0/4109970898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 0x7f1f58102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.886+0000 7f1f60008640 1 -- 172.21.15.96:0/4109970898 >> 172.21.15.96:0/4109970898 conn(0x7f1f580fd700 msgr2=0x7f1f580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.887+0000 7f1f60008640 1 -- 172.21.15.96:0/4109970898 shutdown_connections 2024-09-10T22:59:01.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.887+0000 7f1f60008640 1 -- 172.21.15.96:0/4109970898 wait complete. 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.887+0000 7f1f60008640 1 Processor -- start 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.887+0000 7f1f60008640 1 -- start start 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.887+0000 7f1f60008640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 0x7f1f58199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f58108020 0x7f1f5819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58111ca0 con 0x7f1f581076a0 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58111b20 con 0x7f1f58101c90 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58111e20 con 0x7f1f58108020 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50546/0 (socket says 172.21.15.96:50546) 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 -- 172.21.15.96:0/561395630 learned_addr learned my addr 172.21.15.96:0/561395630 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f58108020 0x7f1f5819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 0x7f1f58199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 -- 172.21.15.96:0/561395630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f58108020 msgr2=0x7f1f5819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f58108020 0x7f1f5819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 -- 172.21.15.96:0/561395630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 msgr2=0x7f1f58199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 0x7f1f58199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 -- 172.21.15.96:0/561395630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5819f320 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e9f0 tx=0x7f1f4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f4effd640 1 -- 172.21.15.96:0/561395630 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800cd00 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5819f610 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f4effd640 1 -- 172.21.15.96:0/561395630 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f480054d0 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f4effd640 1 -- 172.21.15.96:0/561395630 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48010430 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.888+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f581a5cd0 con 0x7f1f581076a0 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 Processor -- start 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 -- start start 2024-09-10T22:59:01.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a14330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a23cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a2dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5576d198cea0 con 0x5576d1a241f0 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576d197c2d0 con 0x5576d1a23870 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f623dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5576d19800e0 con 0x5576d1a13f50 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.892+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a23cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a23cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60338/0 (socket says 172.21.15.96:60338) 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a14330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5dda4640 1 -- 172.21.15.96:0/476655976 learned_addr learned my addr 172.21.15.96:0/476655976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/476655976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 msgr2=0x5576d1a23cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a23cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/476655976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 msgr2=0x5576d1a2dbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:01.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a2dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/476655976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d16da650 con 0x5576d1a13f50 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a14330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005e30 tx=0x7f1f500098b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4039d4742adeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f43fff640 1 -- 172.21.15.96:0/476655976 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50031cf0 con 0x5576d1a13f50 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.893+0000 7f1f43fff640 1 -- 172.21.15.96:0/476655976 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f50036030 con 0x5576d1a13f50 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f4effd640 1 -- 172.21.15.96:0/561395630 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1f480046c0 con 0x7f1f581076a0 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f43fff640 1 -- 172.21.15.96:0/476655976 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500042a0 con 0x5576d1a13f50 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f4effd640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 0x7f1f440b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 -- 172.21.15.96:0/476655976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 msgr2=0x5576d1a14330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a14330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005e30 tx=0x7f1f500098b0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 -- 172.21.15.96:0/476655976 shutdown_connections 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a23cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a2dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 --2- 172.21.15.96:0/476655976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a14330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 -- 172.21.15.96:0/476655976 >> 172.21.15.96:0/476655976 conn(0x5576d1a16470 msgr2=0x5576d1a16850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 0x7f1f440b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f4effd640 1 -- 172.21.15.96:0/561395630 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f480e5e60 con 0x7f1f581076a0 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 -- 172.21.15.96:0/476655976 shutdown_connections 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.894+0000 7f1f623dd740 1 -- 172.21.15.96:0/476655976 wait complete. 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 -- start start 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 0x7f1f440b7860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f5400b5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 Processor -- start 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a23870 0x5576d1a2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a241f0 0x5576d1a2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5576d17dd700 con 0x5576d1a23870 2024-09-10T22:59:01.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576d1917160 con 0x5576d1a13f50 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5576d1a2e1e0 con 0x5576d1a241f0 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.895+0000 7f1f623dd740 10 client.0 _ll_register_callbacks cb 0x5576d19309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5d5a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a23870 0x5576d1a2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5e5a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a241f0 0x5576d1a2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60346/0 (socket says 172.21.15.96:60346) 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 -- 172.21.15.96:0/2580950779 learned_addr learned my addr 172.21.15.96:0/2580950779 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 -- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a241f0 msgr2=0x5576d1a2b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a241f0 0x5576d1a2b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 -- 172.21.15.96:0/2580950779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a23870 msgr2=0x5576d1a2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a23870 0x5576d1a2aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f5dda4640 1 -- 172.21.15.96:0/2580950779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1944c90 con 0x5576d1a13f50 2024-09-10T22:59:01.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.896+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f50036030 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f540035f0 tx=0x7f1f540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347339e09fed557e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f417fa640 10 client.27554 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f417fa640 1 -- 172.21.15.96:0/2580950779 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5401d620 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f417fa640 1 -- 172.21.15.96:0/2580950779 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f5401d7c0 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f417fa640 1 -- 172.21.15.96:0/2580950779 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f5401d9c0 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 10 client.27554 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5576d1a39a90 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f417fa640 1 -- 172.21.15.96:0/2580950779 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1f5401db60 con 0x5576d1a13f50 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 10 client.27554 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 10 client.27554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 20 client.27554 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.897+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1a3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.898+0000 7f1f623dd740 4 client.27554 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:01.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.898+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d198edf0 con 0x5576d1a3ae50 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.898+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1a3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.898+0000 7f1f417fa640 10 client.27554 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.898+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1a3d240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67c7422b767715c5 server_cookie=44ac36cf79d55385 in_seq=0 out_seq=0 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f417fa640 1 -- 172.21.15.96:0/2580950779 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576d198edf0 con 0x5576d1a3ae50 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f417fa640 10 client.27554 handle_command_reply: tid=0 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 client.27554 shutdown 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 msgr2=0x5576d1a2da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f540035f0 tx=0x7f1f540046c0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 shutdown_connections 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a241f0 0x5576d1a2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a23870 0x5576d1a2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1a3d240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f417fa640 0 client.27554 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:01.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 --2- 172.21.15.96:0/2580950779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a13f50 0x5576d1a2da20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 >> 172.21.15.96:0/2580950779 conn(0x5576d1a16470 msgr2=0x5576d1a13420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 shutdown_connections 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 1 -- 172.21.15.96:0/2580950779 wait complete. 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.899+0000 7f1f623dd740 20 client.27554 trim_cache size 0 max 16384 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 Processor -- start 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 -- start start 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a0f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581abfe0 con 0x5576d1a241f0 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581adb20 con 0x5576d1a23870 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f623dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ac6b0 con 0x5576d1a13f50 2024-09-10T22:59:01.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f5e5a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f5e5a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50558/0 (socket says 172.21.15.96:50558) 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/3011169503 learned_addr learned my addr 172.21.15.96:0/3011169503 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.915+0000 7f1f5d5a3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a0f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/3011169503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 msgr2=0x5576d1a373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/3011169503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 msgr2=0x5576d1a0f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a0f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 -- 172.21.15.96:0/3011169503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d1a24e50 con 0x5576d1a241f0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a0f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.916+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f5002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e0147b0944153 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f437fe640 1 -- 172.21.15.96:0/3011169503 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50056070 con 0x5576d1a241f0 2024-09-10T22:59:01.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f437fe640 1 -- 172.21.15.96:0/3011169503 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f50003b70 con 0x5576d1a241f0 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f437fe640 1 -- 172.21.15.96:0/3011169503 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5004a040 con 0x5576d1a241f0 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 -- 172.21.15.96:0/3011169503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 msgr2=0x5576d1a26d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006010 tx=0x7f1f5002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 -- 172.21.15.96:0/3011169503 shutdown_connections 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a23870 0x5576d1a0f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a241f0 0x5576d1a26d50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 --2- 172.21.15.96:0/3011169503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a13f50 0x5576d1a373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 -- 172.21.15.96:0/3011169503 >> 172.21.15.96:0/3011169503 conn(0x5576d1a16470 msgr2=0x5576d1a15760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 -- 172.21.15.96:0/3011169503 shutdown_connections 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.917+0000 7f1f623dd740 1 -- 172.21.15.96:0/3011169503 wait complete. 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 -- start start 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 Processor -- start 2024-09-10T22:59:01.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a23870 0x5576d1a33970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a241f0 0x5576d1a41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581ad450 con 0x5576d1a13f50 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581acd80 con 0x5576d1a241f0 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f581aa000 con 0x5576d1a23870 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f623dd740 10 client.0 _ll_register_callbacks cb 0x7f1f54004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f5dda4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50572/0 (socket says 172.21.15.96:50572) 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f5dda4640 1 -- 172.21.15.96:0/448686263 learned_addr learned my addr 172.21.15.96:0/448686263 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a23870 0x5576d1a33970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.918+0000 7f1f5e5a5640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a241f0 0x5576d1a41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 -- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a23870 msgr2=0x5576d1a33970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a23870 0x5576d1a33970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 -- 172.21.15.96:0/448686263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a241f0 msgr2=0x5576d1a41b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a241f0 0x5576d1a41b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 -- 172.21.15.96:0/448686263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1a42090 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f5dda4640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004700 tx=0x7f1f54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c6a05b4f7c46a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.919+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54042060 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.920+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f54027420 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.920+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f50003b70 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f5401d420 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 10 client.17955 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576d1a43330 con 0x5576d1a13f50 2024-09-10T22:59:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f50036030 con 0x5576d1a13f50 2024-09-10T22:59:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1f5403d460 con 0x5576d1a13f50 2024-09-10T22:59:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 10 client.17955 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:01.144 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 10 client.17955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:59:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 20 client.17955 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1aaad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 4 client.17955 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1f50003b70 con 0x5576d1a3ae50 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.921+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1aaad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f417fa640 10 client.17955 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f5d5a3640 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1aaad80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3091e5c532c0ef10 server_cookie=9e756da0650d3ffa in_seq=0 out_seq=0 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f417fa640 1 -- 172.21.15.96:0/448686263 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1f50003b70 con 0x5576d1a3ae50 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f417fa640 10 client.17955 handle_command_reply: tid=0 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 client.17955 shutdown 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 msgr2=0x5576d1a33430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004700 tx=0x7f1f54017000 comp rx=0 tx=0).stop 2024-09-10T22:59:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 shutdown_connections 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x5576d1a3ae50 0x5576d1aaad80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576d1a241f0 0x5576d1a41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5576d1a23870 0x5576d1a33970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f417fa640 0 client.17955 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 --2- 172.21.15.96:0/448686263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576d1a13f50 0x5576d1a33430 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 >> 172.21.15.96:0/448686263 conn(0x5576d1a16470 msgr2=0x5576d1a29af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 shutdown_connections 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.922+0000 7f1f623dd740 1 -- 172.21.15.96:0/448686263 wait complete. 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f623dd740 20 client.17955 trim_cache size 0 max 16384 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 msgr2=0x7f1f440b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 0x7f1f440b7860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f5400b5a0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 msgr2=0x7f1f581997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e9f0 tx=0x7f1f4800edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 shutdown_connections 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f440b53a0 0x7f1f440b7860 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f58108020 0x7f1f5819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f581076a0 0x7f1f581997b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 --2- 172.21.15.96:0/561395630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f58101c90 0x7f1f58199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 >> 172.21.15.96:0/561395630 conn(0x7f1f580fd700 msgr2=0x7f1f5806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 shutdown_connections 2024-09-10T22:59:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:00.923+0000 7f1f60008640 1 -- 172.21.15.96:0/561395630 wait complete. 2024-09-10T22:59:02.049 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:59:02.388 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.737+0000 7fbd4faa9640 1 -- 172.21.15.96:0/3689930152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48102af0 msgr2=0x7fbd48109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.737+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/3689930152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48102af0 0x7fbd48109470 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009ca0 tx=0x7fbd4002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.742+0000 7fbd4faa9640 1 -- 172.21.15.96:0/3689930152 shutdown_connections 2024-09-10T22:59:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.742+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/3689930152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48102af0 0x7fbd48109470 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.742+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/3689930152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd48102130 0x7fbd481025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.742+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/3689930152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48100fe0 0x7fbd481013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.742+0000 7fbd4faa9640 1 -- 172.21.15.96:0/3689930152 >> 172.21.15.96:0/3689930152 conn(0x7fbd480fc880 msgr2=0x7fbd480fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.747+0000 7fbd4faa9640 1 -- 172.21.15.96:0/3689930152 shutdown_connections 2024-09-10T22:59:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.747+0000 7fbd4faa9640 1 -- 172.21.15.96:0/3689930152 wait complete. 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 Processor -- start 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 -- start start 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd48102130 0x7fbd48112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48102af0 0x7fbd48110870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60390/0 (socket says 172.21.15.96:60390) 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4d845640 1 -- 172.21.15.96:0/1815688832 learned_addr learned my addr 172.21.15.96:0/1815688832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810b130 con 0x7fbd48102af0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810afb0 con 0x7fbd48100fe0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.749+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810b2b0 con 0x7fbd48102130 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4e046640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48102af0 0x7fbd48110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 -- 172.21.15.96:0/1815688832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd48102130 msgr2=0x7fbd48112380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd48102130 0x7fbd48112380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 -- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48102af0 msgr2=0x7fbd48110870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48102af0 0x7fbd48110870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd48110eb0 con 0x7fbd48100fe0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd4d845640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b4d0 tx=0x7fbd3800b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.750+0000 7fbd3effd640 1 -- 172.21.15.96:0/1815688832 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38014070 con 0x7fbd48100fe0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.751+0000 7fbd3effd640 1 -- 172.21.15.96:0/1815688832 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd38010d20 con 0x7fbd48100fe0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.751+0000 7fbd3effd640 1 -- 172.21.15.96:0/1815688832 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3800fe10 con 0x7fbd48100fe0 2024-09-10T22:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.751+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481111a0 con 0x7fbd48100fe0 2024-09-10T22:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.751+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd48072ba0 con 0x7fbd48100fe0 2024-09-10T22:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.755+0000 7fbd3effd640 1 -- 172.21.15.96:0/1815688832 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbd38010870 con 0x7fbd48100fe0 2024-09-10T22:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.756+0000 7fbd3effd640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 0x7fbd340b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.757+0000 7fbd3effd640 1 -- 172.21.15.96:0/1815688832 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd380e5840 con 0x7fbd48100fe0 2024-09-10T22:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.757+0000 7fbd4d044640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 0x7fbd340b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.758+0000 7fbd4d044640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 0x7fbd340b7880 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a890 tx=0x7fbd44005b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.759+0000 7fbd52873740 1 Processor -- start 2024-09-10T22:59:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.759+0000 7fbd52873740 1 -- start start 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d9305f6f0 con 0x560d930f0190 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60406/0 (socket says 172.21.15.96:60406) 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 -- 172.21.15.96:0/3301178042 learned_addr learned my addr 172.21.15.96:0/3301178042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560d9304eb20 con 0x560d930ed860 2024-09-10T22:59:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560d93052930 con 0x560d930f2bd0 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 -- 172.21.15.96:0/3301178042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 msgr2=0x560d930f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 -- 172.21.15.96:0/3301178042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 msgr2=0x560d930f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 -- 172.21.15.96:0/3301178042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d93014530 con 0x560d930ed860 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.760+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd380a85d0 tx=0x7fbd380a8780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c2e6c7bc72788b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.761+0000 7fbd33fff640 1 -- 172.21.15.96:0/3301178042 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3801e030 con 0x560d930ed860 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.761+0000 7fbd33fff640 1 -- 172.21.15.96:0/3301178042 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd380abb10 con 0x560d930ed860 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.761+0000 7fbd33fff640 1 -- 172.21.15.96:0/3301178042 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b3e60 con 0x560d930ed860 2024-09-10T22:59:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 msgr2=0x560d930efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd380a85d0 tx=0x7fbd380a8780 comp rx=0 tx=0).stop 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 shutdown_connections 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 --2- 172.21.15.96:0/3301178042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930ed860 0x560d930efc50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 >> 172.21.15.96:0/3301178042 conn(0x560d930e89b0 msgr2=0x560d930e9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 shutdown_connections 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- 172.21.15.96:0/3301178042 wait complete. 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 -- start start 2024-09-10T22:59:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.762+0000 7fbd52873740 1 Processor -- start 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d92f86ab0 con 0x560d930ed860 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560d92ff4730 con 0x560d930f0190 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560d930f5bb0 con 0x560d930f2bd0 2024-09-10T22:59:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50602/0 (socket says 172.21.15.96:50602) 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd4d845640 1 -- 172.21.15.96:0/3778157490 learned_addr learned my addr 172.21.15.96:0/3778157490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.763+0000 7fbd52873740 10 client.0 _ll_register_callbacks cb 0x560d930486b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 -- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 msgr2=0x560d930f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 -- 172.21.15.96:0/3778157490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 msgr2=0x560d930f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 -- 172.21.15.96:0/3778157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d92db0790 con 0x560d930f2bd0 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f81c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd4002ade0 tx=0x7fbd40031680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436cf5763f1ba1de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003970 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd380abb10 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd40003b10 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd40003d50 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd52873740 10 client.27625 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.764+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d92e29620 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd380abb10 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbd4004e050 con 0x560d930f2bd0 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 10 client.27625 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 10 client.27625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 20 client.27625 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d93102790 0x560d93104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 4 client.27625 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.765+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d93061640 con 0x560d93102790 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.766+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d93102790 0x560d93104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.766+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d93102790 0x560d93104ba0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4995dad1f44c08c8 server_cookie=62f465c8aabcb334 in_seq=0 out_seq=0 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.766+0000 7fbd317fa640 10 client.27625 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.767+0000 7fbd317fa640 1 -- 172.21.15.96:0/3778157490 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d93061640 con 0x560d93102790 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.767+0000 7fbd317fa640 10 client.27625 handle_command_reply: tid=0 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.767+0000 7fbd52873740 1 client.27625 shutdown 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 msgr2=0x560d930f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f81c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd4002ade0 tx=0x7fbd40031680 comp rx=0 tx=0).stop 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 shutdown_connections 2024-09-10T22:59:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d93102790 0x560d93104ba0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930f81c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 --2- 172.21.15.96:0/3778157490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 >> 172.21.15.96:0/3778157490 conn(0x560d930e89b0 msgr2=0x560d930e7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd317fa640 0 client.27625 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 shutdown_connections 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 1 -- 172.21.15.96:0/3778157490 wait complete. 2024-09-10T22:59:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.768+0000 7fbd52873740 20 client.27625 trim_cache size 0 max 16384 2024-09-10T22:59:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.789+0000 7fbd52873740 1 Processor -- start 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.789+0000 7fbd52873740 1 -- start start 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930ed860 0x560d930f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f2bd0 0x560d930f6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b0450 con 0x560d930f0190 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b1f90 con 0x560d930f2bd0 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd52873740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b0b20 con 0x560d930ed860 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50618/0 (socket says 172.21.15.96:50618) 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 -- 172.21.15.96:0/3151999550 learned_addr learned my addr 172.21.15.96:0/3151999550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d845640 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930ed860 0x560d930f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4e046640 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f2bd0 0x560d930f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 -- 172.21.15.96:0/3151999550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930ed860 msgr2=0x560d930f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930ed860 0x560d930f81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 -- 172.21.15.96:0/3151999550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f2bd0 msgr2=0x560d930f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f2bd0 0x560d930f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.790+0000 7fbd4d044640 1 -- 172.21.15.96:0/3151999550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d930f5a90 con 0x560d930f0190 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.791+0000 7fbd4d044640 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400ce70 tx=0x7fbd4400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a6b9a96933cebe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.791+0000 7fbd337fe640 1 -- 172.21.15.96:0/3151999550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400fe60 con 0x560d930f0190 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.791+0000 7fbd337fe640 1 -- 172.21.15.96:0/3151999550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd44016040 con 0x560d930f0190 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.791+0000 7fbd337fe640 1 -- 172.21.15.96:0/3151999550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44026740 con 0x560d930f0190 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- 172.21.15.96:0/3151999550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 msgr2=0x560d930f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400ce70 tx=0x7fbd4400f000 comp rx=0 tx=0).stop 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- 172.21.15.96:0/3151999550 shutdown_connections 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f2bd0 0x560d930f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930f0190 0x560d930f5f50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 --2- 172.21.15.96:0/3151999550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930ed860 0x560d930f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- 172.21.15.96:0/3151999550 >> 172.21.15.96:0/3151999550 conn(0x560d930e89b0 msgr2=0x560d930ecda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- 172.21.15.96:0/3151999550 shutdown_connections 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- 172.21.15.96:0/3151999550 wait complete. 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.792+0000 7fbd52873740 1 -- start start 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 Processor -- start 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b18c0 con 0x560d930ed860 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b11f0 con 0x560d930f0190 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481ae430 con 0x560d930f2bd0 2024-09-10T22:59:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50622/0 (socket says 172.21.15.96:50622) 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 -- 172.21.15.96:0/434371333 learned_addr learned my addr 172.21.15.96:0/434371333 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd52873740 10 client.0 _ll_register_callbacks cb 0x7fbd40009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4e046640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d044640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 -- 172.21.15.96:0/434371333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 msgr2=0x560d930fd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930fd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 -- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 msgr2=0x560d930fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4d845640 1 -- 172.21.15.96:0/434371333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d930fdd00 con 0x560d930ed860 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.793+0000 7fbd4e046640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd4d845640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b4a0 tx=0x7fbd380a8650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69dae24d9b4c2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3801e040 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd44016040 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbd38097200 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd380b3e60 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd52873740 10 client.17970 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d930fcb50 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd44016040 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbd380b5100 con 0x560d930ed860 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.794+0000 7fbd52873740 10 client.17970 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd52873740 10 client.17970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd52873740 20 client.17970 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d9316f8c0 0x560d93171cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd52873740 4 client.17970 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d9305d0f0 con 0x560d9316f8c0 2024-09-10T22:59:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd4d044640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d9316f8c0 0x560d93171cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd317fa640 10 client.17970 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.795+0000 7fbd4d044640 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d9316f8c0 0x560d93171cb0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa493c3032fab1bc server_cookie=8470f87117cbf70f in_seq=0 out_seq=0 2024-09-10T22:59:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.808+0000 7fbd317fa640 1 -- 172.21.15.96:0/434371333 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d9305d0f0 con 0x560d9316f8c0 2024-09-10T22:59:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.808+0000 7fbd317fa640 10 client.17970 handle_command_reply: tid=0 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 client.17970 shutdown 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 msgr2=0x560d930fe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b4a0 tx=0x7fbd380a8650 comp rx=0 tx=0).stop 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 shutdown_connections 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x560d9316f8c0 0x560d93171cb0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd317fa640 0 client.17970 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d930f0190 0x560d930fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d930f2bd0 0x560d930fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 --2- 172.21.15.96:0/434371333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d930ed860 0x560d930fe670 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 >> 172.21.15.96:0/434371333 conn(0x560d930e89b0 msgr2=0x560d930f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 shutdown_connections 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 1 -- 172.21.15.96:0/434371333 wait complete. 2024-09-10T22:59:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.809+0000 7fbd52873740 20 client.17970 trim_cache size 0 max 16384 2024-09-10T22:59:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:59:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:59:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:59:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 msgr2=0x7fbd340b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 0x7fbd340b7880 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400a890 tx=0x7fbd44005b80 comp rx=0 tx=0).stop 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 msgr2=0x7fbd48111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b4d0 tx=0x7fbd3800b990 comp rx=0 tx=0).stop 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 shutdown_connections 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbd340b53c0 0x7fbd340b7880 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd48102af0 0x7fbd48110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd48102130 0x7fbd48112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 --2- 172.21.15.96:0/1815688832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd48100fe0 0x7fbd48111d50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 >> 172.21.15.96:0/1815688832 conn(0x7fbd480fc880 msgr2=0x7fbd48105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 shutdown_connections 2024-09-10T22:59:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:02.810+0000 7fbd4faa9640 1 -- 172.21.15.96:0/1815688832 wait complete. 2024-09-10T22:59:02.814 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:02 smithi096 ceph-mon[30869]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-09-10T22:59:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.950+0000 7f08ead78640 1 -- 172.21.15.96:0/1868195898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e410a150 msgr2=0x7f08e41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.950+0000 7f08ead78640 1 --2- 172.21.15.96:0/1868195898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e410a150 0x7f08e41111b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009ca0 tx=0x7f08e002a960 comp rx=0 tx=0).stop 2024-09-10T22:59:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.952+0000 7f08ead78640 1 -- 172.21.15.96:0/1868195898 shutdown_connections 2024-09-10T22:59:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.952+0000 7f08ead78640 1 --2- 172.21.15.96:0/1868195898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e411c0a0 0x7f08e411e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.952+0000 7f08ead78640 1 --2- 172.21.15.96:0/1868195898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a530 0x7f08e410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.952+0000 7f08ead78640 1 --2- 172.21.15.96:0/1868195898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e410a150 0x7f08e41111b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.952+0000 7f08ead78640 1 -- 172.21.15.96:0/1868195898 >> 172.21.15.96:0/1868195898 conn(0x7f08e406dfc0 msgr2=0x7f08e406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.953+0000 7f08ead78640 1 -- 172.21.15.96:0/1868195898 shutdown_connections 2024-09-10T22:59:02.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.953+0000 7f08ead78640 1 -- 172.21.15.96:0/1868195898 wait complete. 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.954+0000 7f08ead78640 1 Processor -- start 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.954+0000 7f08ead78640 1 -- start start 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a150 0x7f08e4119470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 0x7f08e41199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4121c10 con 0x7f08e410a530 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4121a90 con 0x7f08e411c0a0 2024-09-10T22:59:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08ead78640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4121d90 con 0x7f08e410a150 2024-09-10T22:59:02.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e9315640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e9315640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60440/0 (socket says 172.21.15.96:60440) 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e9315640 1 -- 172.21.15.96:0/4124282700 learned_addr learned my addr 172.21.15.96:0/4124282700 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08dbfff640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 0x7f08e41199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e9315640 1 -- 172.21.15.96:0/4124282700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a150 msgr2=0x7f08e4119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e8b14640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a150 0x7f08e4119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.955+0000 7f08e9315640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a150 0x7f08e4119470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08e9315640 1 -- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 msgr2=0x7f08e41199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08e9315640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 0x7f08e41199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08e9315640 1 -- 172.21.15.96:0/4124282700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e4113080 con 0x7f08e411c0a0 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08dbfff640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 0x7f08e41199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08e9315640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00d750 tx=0x7f08dc00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08d9ffb640 1 -- 172.21.15.96:0/4124282700 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc017070 con 0x7f08e411c0a0 2024-09-10T22:59:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.956+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e4113370 con 0x7f08e411c0a0 2024-09-10T22:59:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.957+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e4118300 con 0x7f08e411c0a0 2024-09-10T22:59:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.957+0000 7f08d9ffb640 1 -- 172.21.15.96:0/4124282700 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08dc0054d0 con 0x7f08e411c0a0 2024-09-10T22:59:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.958+0000 7f08d9ffb640 1 -- 172.21.15.96:0/4124282700 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc004c70 con 0x7f08e411c0a0 2024-09-10T22:59:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.960+0000 7f08d9ffb640 1 -- 172.21.15.96:0/4124282700 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f08dc004ed0 con 0x7f08e411c0a0 2024-09-10T22:59:02.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.961+0000 7f08d9ffb640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 0x7f08cc0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.961+0000 7f08d9ffb640 1 -- 172.21.15.96:0/4124282700 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f08dc0e54e0 con 0x7f08e411c0a0 2024-09-10T22:59:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.962+0000 7f08e8b14640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 0x7f08cc0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.962+0000 7f08ed14d740 1 Processor -- start 2024-09-10T22:59:02.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.962+0000 7f08e8b14640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 0x7f08cc0b76c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005e30 tx=0x7f08e0005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.962+0000 7f08ed14d740 1 -- start start 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 0x55d6438bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d643822ea0 con 0x55d6438acdf0 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6438122d0 con 0x55d6438b8d80 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08ed14d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d6438160e0 con 0x55d6438ac4d0 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50642/0 (socket says 172.21.15.96:50642) 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08dbfff640 1 -- 172.21.15.96:0/2454535180 learned_addr learned my addr 172.21.15.96:0/2454535180 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.963+0000 7f08e9315640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 0x55d6438bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.964+0000 7f08e8b14640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.965+0000 7f08e9315640 1 -- 172.21.15.96:0/2454535180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 msgr2=0x55d6438ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.965+0000 7f08e9315640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.965+0000 7f08e9315640 1 -- 172.21.15.96:0/2454535180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 msgr2=0x55d6438b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.965+0000 7f08e9315640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.966+0000 7f08e9315640 1 -- 172.21.15.96:0/2454535180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d643570650 con 0x55d6438b8d80 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.966+0000 7f08dbfff640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:02.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.967+0000 7f08e9315640 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 0x55d6438bb170 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0e30d0 tx=0x7f08dc0ab4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6c1db9f67a780 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.967+0000 7f08caffd640 1 -- 172.21.15.96:0/2454535180 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc020070 con 0x55d6438b8d80 2024-09-10T22:59:02.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.967+0000 7f08caffd640 1 -- 172.21.15.96:0/2454535180 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08dc0abd40 con 0x55d6438b8d80 2024-09-10T22:59:02.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.967+0000 7f08caffd640 1 -- 172.21.15.96:0/2454535180 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0a7fb0 con 0x55d6438b8d80 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 -- 172.21.15.96:0/2454535180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 msgr2=0x55d6438bb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 0x55d6438bb170 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0e30d0 tx=0x7f08dc0ab4b0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 -- 172.21.15.96:0/2454535180 shutdown_connections 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438b8d80 0x55d6438bb170 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438acdf0 0x55d6438b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 --2- 172.21.15.96:0/2454535180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.968+0000 7f08ed14d740 1 -- 172.21.15.96:0/2454535180 >> 172.21.15.96:0/2454535180 conn(0x55d6438aa300 msgr2=0x55d6438ab4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.969+0000 7f08ed14d740 1 -- 172.21.15.96:0/2454535180 shutdown_connections 2024-09-10T22:59:02.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.969+0000 7f08ed14d740 1 -- 172.21.15.96:0/2454535180 wait complete. 2024-09-10T22:59:02.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.969+0000 7f08ed14d740 1 -- start start 2024-09-10T22:59:02.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.970+0000 7f08ed14d740 1 Processor -- start 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.970+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d643673700 con 0x55d6438ac4d0 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60456/0 (socket says 172.21.15.96:60456) 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 -- 172.21.15.96:0/537323723 learned_addr learned my addr 172.21.15.96:0/537323723 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:02.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08e8b14640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6437ad160 con 0x55d6438acdf0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d6438bba00 con 0x55d6438b8d80 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08e9315640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08ed14d740 10 client.0 _ll_register_callbacks cb 0x55d6437c69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 -- 172.21.15.96:0/537323723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 msgr2=0x55d6438b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 -- 172.21.15.96:0/537323723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 msgr2=0x55d6438b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6437dac90 con 0x55d6438acdf0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08dbfff640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f08d40118b0 tx=0x7f08d4011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26fe9ded906ada73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.971+0000 7f08c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.972+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400fa30 con 0x55d6438acdf0 2024-09-10T22:59:02.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.972+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08dc0abd40 con 0x55d6438acdf0 2024-09-10T22:59:02.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.972+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f08d400fbd0 con 0x55d6438acdf0 2024-09-10T22:59:02.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.972+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08d400fdd0 con 0x55d6438acdf0 2024-09-10T22:59:02.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.972+0000 7f08ed14d740 10 client.27590 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:02.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.973+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6438c74a0 con 0x55d6438acdf0 2024-09-10T22:59:02.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.973+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d40148f0 con 0x55d6438acdf0 2024-09-10T22:59:02.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f08d40286a0 con 0x55d6438acdf0 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 10 client.27590 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 20 client.27590 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d6438cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 4 client.27590 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08dc0abd40 con 0x55d6438c8d00 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.974+0000 7f08e8b14640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d6438cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.975+0000 7f08c3fff640 10 client.27590 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.975+0000 7f08e8b14640 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d6438cb140 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1ee8ebbe222fcb server_cookie=5a0b27713c69de45 in_seq=0 out_seq=0 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.976+0000 7f08c3fff640 1 -- 172.21.15.96:0/537323723 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f08dc0abd40 con 0x55d6438c8d00 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.976+0000 7f08c3fff640 10 client.27590 handle_command_reply: tid=0 2024-09-10T22:59:02.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.976+0000 7f08ed14d740 1 client.27590 shutdown 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 msgr2=0x55d6438b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f08d40118b0 tx=0x7f08d4011cb0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 shutdown_connections 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d6438cb140 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5b10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08c3fff640 0 client.27590 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 --2- 172.21.15.96:0/537323723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:02.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.977+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 >> 172.21.15.96:0/537323723 conn(0x55d6438aa300 msgr2=0x55d6438afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:02.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.978+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 shutdown_connections 2024-09-10T22:59:02.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.978+0000 7f08ed14d740 1 -- 172.21.15.96:0/537323723 wait complete. 2024-09-10T22:59:02.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.978+0000 7f08ed14d740 20 client.27590 trim_cache size 0 max 16384 2024-09-10T22:59:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.996+0000 7f08ed14d740 1 Processor -- start 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.996+0000 7f08ed14d740 1 -- start start 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.996+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438b8d80 0x55d6438b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08ed14d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000e350 con 0x55d6438b8d80 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08ed14d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000fe90 con 0x55d6438acdf0 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08ed14d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000ea20 con 0x55d6438ac4d0 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49468/0 (socket says 172.21.15.96:49468) 2024-09-10T22:59:02.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 -- 172.21.15.96:0/562587633 learned_addr learned my addr 172.21.15.96:0/562587633 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08dbfff640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e9315640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438b8d80 0x55d6438b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 -- 172.21.15.96:0/562587633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 msgr2=0x55d6438b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 -- 172.21.15.96:0/562587633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438b8d80 msgr2=0x55d6438b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438b8d80 0x55d6438b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 -- 172.21.15.96:0/562587633 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6438c72d0 con 0x55d6438ac4d0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08dbfff640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.997+0000 7f08e8b14640 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009ca0 tx=0x7f08e0034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7259a9306f80770f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ca7fc640 1 -- 172.21.15.96:0/562587633 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0034070 con 0x55d6438ac4d0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ca7fc640 1 -- 172.21.15.96:0/562587633 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e0004040 con 0x55d6438ac4d0 2024-09-10T22:59:03.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ca7fc640 1 -- 172.21.15.96:0/562587633 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e003a650 con 0x55d6438ac4d0 2024-09-10T22:59:03.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 -- 172.21.15.96:0/562587633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 msgr2=0x55d6438c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009ca0 tx=0x7f08e0034000 comp rx=0 tx=0).stop 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 -- 172.21.15.96:0/562587633 shutdown_connections 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438b8d80 0x55d6438b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 --2- 172.21.15.96:0/562587633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438ac4d0 0x55d6438c4c90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 -- 172.21.15.96:0/562587633 >> 172.21.15.96:0/562587633 conn(0x55d6438aa300 msgr2=0x55d6438ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 -- 172.21.15.96:0/562587633 shutdown_connections 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.998+0000 7f08ed14d740 1 -- 172.21.15.96:0/562587633 wait complete. 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.999+0000 7f08ed14d740 1 -- start start 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.999+0000 7f08ed14d740 1 Processor -- start 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.999+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.999+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:02.999+0000 7f08ed14d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08ed14d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000f7c0 con 0x55d6438ac4d0 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08ed14d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000f0f0 con 0x55d6438acdf0 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08ed14d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08d000c330 con 0x55d6438b8d80 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08ed14d740 10 client.0 _ll_register_callbacks cb 0x7f08d4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60480/0 (socket says 172.21.15.96:60480) 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 -- 172.21.15.96:0/252182440 learned_addr learned my addr 172.21.15.96:0/252182440 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08e8b14640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 -- 172.21.15.96:0/252182440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 msgr2=0x55d6438bbcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438bbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 -- 172.21.15.96:0/252182440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 msgr2=0x55d6438c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:03.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.000+0000 7f08dbfff640 1 -- 172.21.15.96:0/252182440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6438aad80 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.001+0000 7f08dbfff640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f08d400f610 tx=0x7f08d40286b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce6bc4c6d100019 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.001+0000 7f08c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.001+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4004040 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.001+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f08d40041e0 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.001+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e0004040 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08d40043e0 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 10 client.27596 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d643936310 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e0004040 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f08d4038030 con 0x55d6438acdf0 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 10 client.27596 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:03.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:03.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 20 client.27596 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:03.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d643938b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:03.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 4 client.27596 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:03.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d643824df0 con 0x55d6438c8d00 2024-09-10T22:59:03.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.002+0000 7f08e8b14640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d643938b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:03.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08c3fff640 10 client.27596 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:03.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08e8b14640 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d643938b50 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=545d5a80284d0622 server_cookie=19901e07a3072230 in_seq=0 out_seq=0 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08c3fff640 1 -- 172.21.15.96:0/252182440 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d643824df0 con 0x55d6438c8d00 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08c3fff640 10 client.27596 handle_command_reply: tid=0 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 client.27596 shutdown 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 msgr2=0x55d6438c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f08d400f610 tx=0x7f08d40286b0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 shutdown_connections 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d6438c8d00 0x55d643938b50 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6438acdf0 0x55d6438c1240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6438b8d80 0x55d6438bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08c3fff640 0 client.27596 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 --2- 172.21.15.96:0/252182440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6438ac4d0 0x55d6438c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 >> 172.21.15.96:0/252182440 conn(0x55d6438aa300 msgr2=0x55d6438b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:03.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 shutdown_connections 2024-09-10T22:59:03.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 1 -- 172.21.15.96:0/252182440 wait complete. 2024-09-10T22:59:03.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.003+0000 7f08ed14d740 20 client.27596 trim_cache size 0 max 16384 2024-09-10T22:59:03.010 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 msgr2=0x7f08cc0b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 0x7f08cc0b76c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005e30 tx=0x7f08e0005d80 comp rx=0 tx=0).stop 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 msgr2=0x7f08e41127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00d750 tx=0x7f08dc00dc10 comp rx=0 tx=0).stop 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 shutdown_connections 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e411c0a0 0x7f08e41127c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f08cc0b5200 0x7f08cc0b76c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08e410a530 0x7f08e41199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 --2- 172.21.15.96:0/4124282700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08e410a150 0x7f08e4119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:03.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 >> 172.21.15.96:0/4124282700 conn(0x7f08e406dfc0 msgr2=0x7f08e411c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:03.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 shutdown_connections 2024-09-10T22:59:03.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:03.004+0000 7f08ead78640 1 -- 172.21.15.96:0/4124282700 wait complete. 2024-09-10T22:59:03.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:02 smithi121 ceph-mon[35119]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-09-10T22:59:03.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:02 smithi117 ceph-mon[35081]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 622 op/s 2024-09-10T22:59:05.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:04 smithi096 ceph-mon[30869]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-09-10T22:59:05.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:04 smithi121 ceph-mon[35119]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-09-10T22:59:05.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:04 smithi117 ceph-mon[35081]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-09-10T22:59:05.073 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:59:05.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:59:05.436 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 690.0 s, 1766.4 tps, lat 56.739 ms stddev 62.267 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.051+0000 7f89e6010640 1 -- 172.21.15.96:0/902691558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0101b20 msgr2=0x7f89e0101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.051+0000 7f89e6010640 1 --2- 172.21.15.96:0/902691558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0101b20 0x7f89e0101f00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009900 tx=0x7f89d002a420 comp rx=0 tx=0).stop 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.053+0000 7f89e6010640 1 -- 172.21.15.96:0/902691558 shutdown_connections 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.053+0000 7f89e6010640 1 --2- 172.21.15.96:0/902691558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0107530 0x7f89e0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.053+0000 7f89e6010640 1 --2- 172.21.15.96:0/902691558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107eb0 0x7f89e010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.053+0000 7f89e6010640 1 --2- 172.21.15.96:0/902691558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0101b20 0x7f89e0101f00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.053+0000 7f89e6010640 1 -- 172.21.15.96:0/902691558 >> 172.21.15.96:0/902691558 conn(0x7f89e00fd5d0 msgr2=0x7f89e00ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.058+0000 7f89e6010640 1 -- 172.21.15.96:0/902691558 shutdown_connections 2024-09-10T22:59:06.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.058+0000 7f89e6010640 1 -- 172.21.15.96:0/902691558 wait complete. 2024-09-10T22:59:06.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.059+0000 7f89e6010640 1 Processor -- start 2024-09-10T22:59:06.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.059+0000 7f89e6010640 1 -- start start 2024-09-10T22:59:06.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 0x7f89e0074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0107eb0 0x7f89e006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89e006d430 con 0x7f89e0107eb0 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89e006d2b0 con 0x7f89e0101b20 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.060+0000 7f89e6010640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89e006d5b0 con 0x7f89e0107530 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60490/0 (socket says 172.21.15.96:60490) 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 -- 172.21.15.96:0/1556665005 learned_addr learned my addr 172.21.15.96:0/1556665005 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0107eb0 0x7f89e006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89deffd640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 0x7f89e0074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 -- 172.21.15.96:0/1556665005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 msgr2=0x7f89e0074fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 0x7f89e0074fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 -- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0107eb0 msgr2=0x7f89e006ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.061+0000 7f89df7fe640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0107eb0 0x7f89e006ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89df7fe640 1 -- 172.21.15.96:0/1556665005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e006f6d0 con 0x7f89e0101b20 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89deffd640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 0x7f89e0074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89df7fe640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031440 tx=0x7f89d0003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89dcff9640 1 -- 172.21.15.96:0/1556665005 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0061070 con 0x7f89e0101b20 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e006f960 con 0x7f89e0101b20 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.062+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01a8140 con 0x7f89e0101b20 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.063+0000 7f89dcff9640 1 -- 172.21.15.96:0/1556665005 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d0031b50 con 0x7f89e0101b20 2024-09-10T22:59:06.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.063+0000 7f89dcff9640 1 -- 172.21.15.96:0/1556665005 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d006b060 con 0x7f89e0101b20 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.067+0000 7f89e83e5740 1 Processor -- start 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.067+0000 7f89e83e5740 1 -- start start 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf3a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e71cea4e60 con 0x55e71cf2ea10 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e71ce94290 con 0x55e71cf3a9a0 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89e83e5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e71ce980a0 con 0x55e71cf2e0f0 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89dffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89dffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:49500/0 (socket says 172.21.15.96:49500) 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.068+0000 7f89dffff640 1 -- 172.21.15.96:0/3566944499 learned_addr learned my addr 172.21.15.96:0/3566944499 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:06.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dffff640 1 -- 172.21.15.96:0/3566944499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 msgr2=0x55e71cf3cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dffff640 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dffff640 1 -- 172.21.15.96:0/3566944499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 msgr2=0x55e71cf3a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dffff640 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf3a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dffff640 1 -- 172.21.15.96:0/3566944499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71cbf2930 con 0x55e71cf2e0f0 2024-09-10T22:59:06.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.069+0000 7f89dcff9640 1 -- 172.21.15.96:0/1556665005 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f89d005dcd0 con 0x7f89e0101b20 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.070+0000 7f89dffff640 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f90 tx=0x7f89d800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c79edc7201266b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.070+0000 7f89c5ffb640 1 -- 172.21.15.96:0/3566944499 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d801a070 con 0x55e71cf2e0f0 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.070+0000 7f89c5ffb640 1 -- 172.21.15.96:0/3566944499 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d80158c0 con 0x55e71cf2e0f0 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.071+0000 7f89dcff9640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 0x7f89c80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.071+0000 7f89e83e5740 1 -- 172.21.15.96:0/3566944499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 msgr2=0x55e71cf2e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.071+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f90 tx=0x7f89d800c8b0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 -- 172.21.15.96:0/3566944499 shutdown_connections 2024-09-10T22:59:06.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf3cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf3a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3566944499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2e4d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 -- 172.21.15.96:0/3566944499 >> 172.21.15.96:0/3566944499 conn(0x55e71cf2bf20 msgr2=0x55e71cf2d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89dcff9640 1 -- 172.21.15.96:0/1556665005 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f89d0128e20 con 0x7f89e0101b20 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 -- 172.21.15.96:0/3566944499 shutdown_connections 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 -- 172.21.15.96:0/3566944499 wait complete. 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89df7fe640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 0x7f89c80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.072+0000 7f89e83e5740 1 -- start start 2024-09-10T22:59:06.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.073+0000 7f89e83e5740 1 Processor -- start 2024-09-10T22:59:06.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.073+0000 7f89df7fe640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 0x7f89c80b79a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89d002aa10 tx=0x7f89d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.073+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.073+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.073+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89dffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89dffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60504/0 (socket says 172.21.15.96:60504) 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89dffff640 1 -- 172.21.15.96:0/2854029872 learned_addr learned my addr 172.21.15.96:0/2854029872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e71ccf5890 con 0x55e71cf2ea10 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e71ce2ed40 con 0x55e71cf3a9a0 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e71cf3d6a0 con 0x55e71cf2e0f0 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.074+0000 7f89e83e5740 10 client.0 _ll_register_callbacks cb 0x55e71ce92c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.075+0000 7f89dffff640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.075+0000 7f89deffd640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.076+0000 7f89dffff640 1 -- 172.21.15.96:0/2854029872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 msgr2=0x55e71cf3a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.076+0000 7f89dffff640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf3a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.076+0000 7f89dffff640 1 -- 172.21.15.96:0/2854029872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 msgr2=0x55e71cf37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.076+0000 7f89dffff640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.076+0000 7f89dffff640 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71cf46ab0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.079+0000 7f89dffff640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f89d8018040 tx=0x7f89d800a4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262cdace7cef95a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.079+0000 7f89beffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:06.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.079+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d801a030 con 0x55e71cf3a9a0 2024-09-10T22:59:06.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.079+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f89d8012040 con 0x55e71cf3a9a0 2024-09-10T22:59:06.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.079+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d80158c0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89d80075e0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 10 client.27605 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:06.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e71cf493c0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d80158c0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f89d8024620 con 0x55e71cf3a9a0 2024-09-10T22:59:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 10 client.27605 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 10 client.27605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 20 client.27605 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cf4aba0 0x55e71cf4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 4 client.27605 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e71cea6db0 con 0x55e71cf4aba0 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.080+0000 7f89deffd640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cf4aba0 0x55e71cf4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.081+0000 7f89beffd640 10 client.27605 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.081+0000 7f89deffd640 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cf4aba0 0x55e71cf4cfe0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf21b34f7e8254b9 server_cookie=babb704564a4d152 in_seq=0 out_seq=0 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89beffd640 1 -- 172.21.15.96:0/2854029872 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e71cea6db0 con 0x55e71cf4aba0 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89beffd640 10 client.27605 handle_command_reply: tid=0 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 client.27605 shutdown 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 msgr2=0x55e71cf37f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f89d8018040 tx=0x7f89d800a4a0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 shutdown_connections 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cf4aba0 0x55e71cf4cfe0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf37f60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2ea10 0x55e71cf37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 --2- 172.21.15.96:0/2854029872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2e0f0 0x55e71cf3a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 >> 172.21.15.96:0/2854029872 conn(0x55e71cf2bf20 msgr2=0x55e71cf31ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89beffd640 0 client.27605 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.082+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 shutdown_connections 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.083+0000 7f89e83e5740 1 -- 172.21.15.96:0/2854029872 wait complete. 2024-09-10T22:59:06.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.083+0000 7f89e83e5740 20 client.27605 trim_cache size 0 max 16384 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 Processor -- start 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 -- start start 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00e440 con 0x55e71cf2e0f0 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00ff80 con 0x55e71cf3a9a0 2024-09-10T22:59:06.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.109+0000 7f89e83e5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00eb10 con 0x55e71cf2ea10 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47400/0 (socket says 172.21.15.96:47400) 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 -- 172.21.15.96:0/3597359167 learned_addr learned my addr 172.21.15.96:0/3597359167 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89dffff640 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89dffff640 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 -- 172.21.15.96:0/3597359167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 msgr2=0x55e71cf34fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 -- 172.21.15.96:0/3597359167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 msgr2=0x55e71cf2d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 -- 172.21.15.96:0/3597359167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71cf27840 con 0x55e71cf2ea10 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89deffd640 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e71cf2e0a0 tx=0x7f89d40187d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d7cfb8d5aec20b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89c57fa640 1 -- 172.21.15.96:0/3597359167 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4024070 con 0x55e71cf2ea10 2024-09-10T22:59:06.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.110+0000 7f89c57fa640 1 -- 172.21.15.96:0/3597359167 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d4004230 con 0x55e71cf2ea10 2024-09-10T22:59:06.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89c57fa640 1 -- 172.21.15.96:0/3597359167 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d401f6b0 con 0x55e71cf2ea10 2024-09-10T22:59:06.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 -- 172.21.15.96:0/3597359167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 msgr2=0x55e71cf34a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e71cf2e0a0 tx=0x7f89d40187d0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 -- 172.21.15.96:0/3597359167 shutdown_connections 2024-09-10T22:59:06.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf34a90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 --2- 172.21.15.96:0/3597359167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.111+0000 7f89e83e5740 1 -- 172.21.15.96:0/3597359167 >> 172.21.15.96:0/3597359167 conn(0x55e71cf2bf20 msgr2=0x55e71cf2cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.112+0000 7f89e83e5740 1 -- 172.21.15.96:0/3597359167 shutdown_connections 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.112+0000 7f89e83e5740 1 -- 172.21.15.96:0/3597359167 wait complete. 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.112+0000 7f89e83e5740 1 -- start start 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.112+0000 7f89e83e5740 1 Processor -- start 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf43050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf43050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf43050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48568/0 (socket says 172.21.15.96:48568) 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00f8b0 con 0x55e71cf2e0f0 2024-09-10T22:59:06.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00f1e0 con 0x55e71cf3a9a0 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89e83e5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc00c420 con 0x55e71cf2ea10 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.113+0000 7f89dffff640 1 -- 172.21.15.96:0/1119897037 learned_addr learned my addr 172.21.15.96:0/1119897037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89e83e5740 10 client.0 _ll_register_callbacks cb 0x7f89d8018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 -- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 msgr2=0x55e71cf34af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 -- 172.21.15.96:0/1119897037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 msgr2=0x55e71cf43050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf43050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 -- 172.21.15.96:0/1119897037 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71cf351b0 con 0x55e71cf2ea10 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89deffd640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf345b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55e71cf37a20 tx=0x7f89d4004450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f0ffe6eca0a70c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89beffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.114+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4024040 con 0x55e71cf2ea10 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.115+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f89d40052f0 con 0x55e71cf2ea10 2024-09-10T22:59:06.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.115+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d4004230 con 0x55e71cf2ea10 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89d4005530 con 0x55e71cf2ea10 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 10 client.27652 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e71cfb7ab0 con 0x55e71cf2ea10 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d4004230 con 0x55e71cf2ea10 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f89d4027070 con 0x55e71cf2ea10 2024-09-10T22:59:06.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 10 client.27652 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 20 client.27652 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cfb8000 0x55e71cfba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 4 client.27652 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.116+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e71cf27640 con 0x55e71cfb8000 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.117+0000 7f89dffff640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cfb8000 0x55e71cfba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.118+0000 7f89beffd640 10 client.27652 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:06.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.118+0000 7f89dffff640 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cfb8000 0x55e71cfba420 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9dfce19c12bad5e server_cookie=59f698a5b16674dd in_seq=0 out_seq=0 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.120+0000 7f89beffd640 1 -- 172.21.15.96:0/1119897037 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e71cf27640 con 0x55e71cfb8000 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.120+0000 7f89beffd640 10 client.27652 handle_command_reply: tid=0 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.120+0000 7f89e83e5740 1 client.27652 shutdown 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 msgr2=0x55e71cf345b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf345b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55e71cf37a20 tx=0x7f89d4004450 comp rx=0 tx=0).stop 2024-09-10T22:59:06.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 shutdown_connections 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e71cfb8000 0x55e71cfba420 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e71cf3a9a0 0x55e71cf34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e71cf2ea10 0x55e71cf345b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 --2- 172.21.15.96:0/1119897037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e71cf2e0f0 0x55e71cf43050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 >> 172.21.15.96:0/1119897037 conn(0x55e71cf2bf20 msgr2=0x55e71cf312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89beffd640 0 client.27652 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 shutdown_connections 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 1 -- 172.21.15.96:0/1119897037 wait complete. 2024-09-10T22:59:06.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.121+0000 7f89e83e5740 20 client.27652 trim_cache size 0 max 16384 2024-09-10T22:59:06.124 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:59:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:59:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "8e815330-f099-43c4-b9eb-e85915810f52", 2024-09-10T22:59:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:59:06.124 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 msgr2=0x7f89c80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 0x7f89c80b79a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89d002aa10 tx=0x7f89d0009260 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 msgr2=0x7f89e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031440 tx=0x7f89d0003c00 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 shutdown_connections 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f89c80b54e0 0x7f89c80b79a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89e0107eb0 0x7f89e006ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e0107530 0x7f89e0074fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 --2- 172.21.15.96:0/1556665005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89e0101b20 0x7f89e0074aa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 >> 172.21.15.96:0/1556665005 conn(0x7f89e00fd5d0 msgr2=0x7f89e00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:06.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.122+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 shutdown_connections 2024-09-10T22:59:06.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:06.123+0000 7f89e6010640 1 -- 172.21.15.96:0/1556665005 wait complete. 2024-09-10T22:59:06.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:06 smithi096 ceph-mon[30869]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 727 op/s 2024-09-10T22:59:06.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:06 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:59:06.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:06 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:06.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:06 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:59:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:06 smithi121 ceph-mon[35119]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 727 op/s 2024-09-10T22:59:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:06 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:59:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:06 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:06 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:59:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:06 smithi117 ceph-mon[35081]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 727 op/s 2024-09-10T22:59:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:06 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:59:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:06 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:06 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:59:07.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:07.402 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:07.857 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:07 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:59:07.857 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:07 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.857 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:07 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 -- 172.21.15.96:0/3508487724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c06f4a0 msgr2=0x7f734c11bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 --2- 172.21.15.96:0/3508487724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c06f4a0 0x7f734c11bcb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7334009900 tx=0x7f733402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 -- 172.21.15.96:0/3508487724 shutdown_connections 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 --2- 172.21.15.96:0/3508487724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c11c1f0 0x7f734c11e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 --2- 172.21.15.96:0/3508487724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c06f4a0 0x7f734c11bcb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 --2- 172.21.15.96:0/3508487724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06eb80 0x7f734c06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 -- 172.21.15.96:0/3508487724 >> 172.21.15.96:0/3508487724 conn(0x7f734c06dce0 msgr2=0x7f734c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 -- 172.21.15.96:0/3508487724 shutdown_connections 2024-09-10T22:59:07.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.971+0000 7f7352ab9640 1 -- 172.21.15.96:0/3508487724 wait complete. 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 Processor -- start 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 -- start start 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 0x7f734c1a8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c11c1f0 0x7f734c1ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f734c121bf0 con 0x7f734c11c1f0 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f734c121a70 con 0x7f734c06f4a0 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7352ab9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f734c121d70 con 0x7f734c06eb80 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47440/0 (socket says 172.21.15.96:47440) 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f7350855640 1 -- 172.21.15.96:0/140652104 learned_addr learned my addr 172.21.15.96:0/140652104 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.972+0000 7f734bfff640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 0x7f734c1a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7351056640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c11c1f0 0x7f734c1ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7350855640 1 -- 172.21.15.96:0/140652104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 msgr2=0x7f734c1a8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7350855640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 0x7f734c1a8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7350855640 1 -- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c11c1f0 msgr2=0x7f734c1ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7350855640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c11c1f0 0x7f734c1ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.973+0000 7f7350855640 1 -- 172.21.15.96:0/140652104 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c1ae0d0 con 0x7f734c06eb80 2024-09-10T22:59:07.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.974+0000 7f734bfff640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 0x7f734c1a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:07.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.974+0000 7f7350855640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f734400e9f0 tx=0x7f734400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.974+0000 7f7349ffb640 1 -- 172.21.15.96:0/140652104 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734400cd80 con 0x7f734c06eb80 2024-09-10T22:59:07.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.974+0000 7f7349ffb640 1 -- 172.21.15.96:0/140652104 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73440108e0 con 0x7f734c06eb80 2024-09-10T22:59:07.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.974+0000 7f7349ffb640 1 -- 172.21.15.96:0/140652104 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344018980 con 0x7f734c06eb80 2024-09-10T22:59:07.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.975+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c1ae3c0 con 0x7f734c06eb80 2024-09-10T22:59:07.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.975+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1b4a80 con 0x7f734c06eb80 2024-09-10T22:59:07.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.977+0000 7f7354e8e740 1 Processor -- start 2024-09-10T22:59:07.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7349ffb640 1 -- 172.21.15.96:0/140652104 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7344010430 con 0x7f734c06eb80 2024-09-10T22:59:07.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 -- start start 2024-09-10T22:59:07.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a14f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 0x5565a15017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a1504120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565a146be50 con 0x5565a1501d30 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565a145b280 con 0x5565a14f5480 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.978+0000 7f7354e8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565a145f090 con 0x5565a14f5da0 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.979+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 0x5565a15017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.979+0000 7f734bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a1504120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.979+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 0x5565a15017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47446/0 (socket says 172.21.15.96:47446) 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.979+0000 7f7350855640 1 -- 172.21.15.96:0/1411241987 learned_addr learned my addr 172.21.15.96:0/1411241987 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:07.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.979+0000 7f7351056640 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a14f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.980+0000 7f7349ffb640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 0x7f73380b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.980+0000 7f7349ffb640 1 -- 172.21.15.96:0/140652104 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f73440e5fa0 con 0x7f734c06eb80 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 -- 172.21.15.96:0/1411241987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 msgr2=0x5565a15017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 0x5565a15017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 -- 172.21.15.96:0/1411241987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 msgr2=0x5565a1504120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:07.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a1504120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 -- 172.21.15.96:0/1411241987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a11b9650 con 0x5565a14f5480 2024-09-10T22:59:07.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 0x7f73380b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a14f5860 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f733c010420 tx=0x7f733c0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccbb58950c7f3c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7342ffd640 1 -- 172.21.15.96:0/1411241987 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c024070 con 0x5565a14f5480 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7351056640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 0x7f73380b78f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f733c021000 tx=0x7f733c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7342ffd640 1 -- 172.21.15.96:0/1411241987 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c013580 con 0x5565a14f5480 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.981+0000 7f7342ffd640 1 -- 172.21.15.96:0/1411241987 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c01ec40 con 0x5565a14f5480 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 -- 172.21.15.96:0/1411241987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 msgr2=0x5565a14f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a14f5860 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f733c010420 tx=0x7f733c0108e0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 -- 172.21.15.96:0/1411241987 shutdown_connections 2024-09-10T22:59:07.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a1504120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5da0 0x5565a15017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 --2- 172.21.15.96:0/1411241987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a14f5860 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 -- 172.21.15.96:0/1411241987 >> 172.21.15.96:0/1411241987 conn(0x5565a14f32b0 msgr2=0x5565a14f4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 -- 172.21.15.96:0/1411241987 shutdown_connections 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.982+0000 7f7354e8e740 1 -- 172.21.15.96:0/1411241987 wait complete. 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 -- start start 2024-09-10T22:59:07.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 Processor -- start 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a1501620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a14fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a1501d30 0x5565a14ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565a12bc700 con 0x5565a14f5da0 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565a13f6110 con 0x5565a14f5480 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565a15049b0 con 0x5565a1501d30 2024-09-10T22:59:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f7354e8e740 10 client.0 _ll_register_callbacks cb 0x5565a140f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.983+0000 7f734bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a14fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f734bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a14fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48604/0 (socket says 172.21.15.96:48604) 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f734bfff640 1 -- 172.21.15.96:0/986607046 learned_addr learned my addr 172.21.15.96:0/986607046 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a1501620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7351056640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a1501d30 0x5565a14ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 -- 172.21.15.96:0/986607046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a1501d30 msgr2=0x5565a14ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a1501d30 0x5565a14ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 -- 172.21.15.96:0/986607046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 msgr2=0x5565a14fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a14fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 -- 172.21.15.96:0/986607046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a1423c40 con 0x5565a14f5480 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7350855640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a1501620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f73440097b0 tx=0x7f73440a8ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85c8f03c7f37f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:07.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f732bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440a6e60 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f7344018b20 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.984+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733c013580 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73440abed0 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 10 client.27620 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565a15102e0 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c013580 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f73440a80b0 con 0x5565a14f5480 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 10 client.27620 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 20 client.27620 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1513be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 4 client.27620 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:07.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a146dda0 con 0x5565a15117f0 2024-09-10T22:59:07.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.985+0000 7f734bfff640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1513be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:07.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.986+0000 7f734bfff640 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1513be0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f923f9716d0f0100 server_cookie=604174ec51c23b32 in_seq=0 out_seq=0 2024-09-10T22:59:07.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.986+0000 7f732bfff640 10 client.27620 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.986+0000 7f732bfff640 1 -- 172.21.15.96:0/986607046 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565a146dda0 con 0x5565a15117f0 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.986+0000 7f732bfff640 10 client.27620 handle_command_reply: tid=0 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.986+0000 7f7354e8e740 1 client.27620 shutdown 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.987+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 msgr2=0x5565a1501620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.987+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a1501620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f73440097b0 tx=0x7f73440a8ea0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 shutdown_connections 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1513be0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f732bfff640 0 client.27620 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a1501d30 0x5565a14ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a14fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 --2- 172.21.15.96:0/986607046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5480 0x5565a1501620 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:07.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 >> 172.21.15.96:0/986607046 conn(0x5565a14f32b0 msgr2=0x5565a14f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:07.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 shutdown_connections 2024-09-10T22:59:07.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 1 -- 172.21.15.96:0/986607046 wait complete. 2024-09-10T22:59:07.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:07.988+0000 7f7354e8e740 20 client.27620 trim_cache size 0 max 16384 2024-09-10T22:59:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:07 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:59:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:07 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:07 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:08.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 Processor -- start 2024-09-10T22:59:08.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 -- start start 2024-09-10T22:59:08.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a14fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1bd050 con 0x5565a1501d30 2024-09-10T22:59:08.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1beb90 con 0x5565a14f5da0 2024-09-10T22:59:08.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7354e8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1bd720 con 0x5565a14f5480 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47460/0 (socket says 172.21.15.96:47460) 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33642/0 (socket says 172.21.15.96:33642) 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7351056640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a14fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f7350855640 1 -- 172.21.15.96:0/2875669659 learned_addr learned my addr 172.21.15.96:0/2875669659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 -- 172.21.15.96:0/2875669659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 msgr2=0x5565a150dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 -- 172.21.15.96:0/2875669659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 msgr2=0x5565a14fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a14fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.015+0000 7f734bfff640 1 -- 172.21.15.96:0/2875669659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a14f5b90 con 0x5565a14f5da0 2024-09-10T22:59:08.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.016+0000 7f7350855640 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.016+0000 7f734bfff640 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7334031440 tx=0x7f7334005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e1710bd8063adb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.016+0000 7f73427fc640 1 -- 172.21.15.96:0/2875669659 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334049020 con 0x5565a14f5da0 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.016+0000 7f73427fc640 1 -- 172.21.15.96:0/2875669659 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73340046c0 con 0x5565a14f5da0 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.016+0000 7f73427fc640 1 -- 172.21.15.96:0/2875669659 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334035040 con 0x5565a14f5da0 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- 172.21.15.96:0/2875669659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 msgr2=0x5565a14fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7334031440 tx=0x7f7334005b50 comp rx=0 tx=0).stop 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- 172.21.15.96:0/2875669659 shutdown_connections 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a14f5da0 0x5565a14fe820 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a1501d30 0x5565a14fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 --2- 172.21.15.96:0/2875669659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- 172.21.15.96:0/2875669659 >> 172.21.15.96:0/2875669659 conn(0x5565a14f32b0 msgr2=0x5565a14f3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- 172.21.15.96:0/2875669659 shutdown_connections 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- 172.21.15.96:0/2875669659 wait complete. 2024-09-10T22:59:08.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.017+0000 7f7354e8e740 1 -- start start 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 Processor -- start 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a150a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a1501d30 0x5565a1504db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1be4c0 con 0x5565a14f5da0 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1bddf0 con 0x5565a1501d30 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1bb030 con 0x5565a14f5480 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7350855640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47468/0 (socket says 172.21.15.96:47468) 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7350855640 1 -- 172.21.15.96:0/4015558707 learned_addr learned my addr 172.21.15.96:0/4015558707 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f7354e8e740 10 client.0 _ll_register_callbacks cb 0x7f73440e95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:08.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.018+0000 7f734bfff640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a150a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 -- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a1501d30 msgr2=0x5565a1504db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7351056640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a1501d30 0x5565a1504db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a1501d30 0x5565a1504db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 -- 172.21.15.96:0/4015558707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 msgr2=0x5565a150a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:08.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a150a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 -- 172.21.15.96:0/4015558707 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a15055e0 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7350855640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f73440097b0 tx=0x7f73440ac3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f3c69123144117 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f732bfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440a6e60 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f73440e9280 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73340046c0 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73440b1240 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7354e8e740 10 client.27670 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.019+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565a157f370 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.020+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73340046c0 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.020+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f73440b23d0 con 0x5565a14f5480 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 10 client.27670 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:08.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 10 client.27670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 20 client.27670 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1581ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 4 client.27670 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565a1469850 con 0x5565a15117f0 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.021+0000 7f734bfff640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1581ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.022+0000 7f734bfff640 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1581ab0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac76156847192627 server_cookie=a145136976e909f6 in_seq=0 out_seq=0 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.022+0000 7f732bfff640 10 client.27670 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.022+0000 7f732bfff640 1 -- 172.21.15.96:0/4015558707 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x5565a1469850 con 0x5565a15117f0 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.022+0000 7f732bfff640 10 client.27670 handle_command_reply: tid=0 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.022+0000 7f7354e8e740 1 client.27670 shutdown 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 msgr2=0x5565a150a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f73440097b0 tx=0x7f73440ac3b0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 shutdown_connections 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5565a15117f0 0x5565a1581ab0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565a1501d30 0x5565a1504db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565a14f5da0 0x5565a150a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 --2- 172.21.15.96:0/4015558707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565a14f5480 0x5565a150a070 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f73440097b0 tx=0x7f73440ac3b0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 >> 172.21.15.96:0/4015558707 conn(0x5565a14f32b0 msgr2=0x5565a14fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f732bfff640 0 client.27670 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 shutdown_connections 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 1 -- 172.21.15.96:0/4015558707 wait complete. 2024-09-10T22:59:08.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.023+0000 7f7354e8e740 20 client.27670 trim_cache size 0 max 16384 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "8e815330-f099-43c4-b9eb-e85915810f52", 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T22:59:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.030 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:59:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:08.038 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:59:08.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:07 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:59:08.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:07 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:08.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:07 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 msgr2=0x7f73380b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 0x7f73380b78f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f733c021000 tx=0x7f733c01a000 comp rx=0 tx=0).stop 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 msgr2=0x7f734c1a8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f734400e9f0 tx=0x7f734400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 shutdown_connections 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734c06f4a0 0x7f734c1a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f73380b5430 0x7f73380b78f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f734c11c1f0 0x7f734c1ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 --2- 172.21.15.96:0/140652104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f734c06eb80 0x7f734c1a8020 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:08.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 >> 172.21.15.96:0/140652104 conn(0x7f734c06dce0 msgr2=0x7f734c10c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:08.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 shutdown_connections 2024-09-10T22:59:08.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:08.024+0000 7f7352ab9640 1 -- 172.21.15.96:0/140652104 wait complete. 2024-09-10T22:59:08.791 INFO:tasks.cephfs.filesystem:scrub status for tag:8e815330-f099-43c4-b9eb-e85915810f52 - {'path': '/', 'tag': '8e815330-f099-43c4-b9eb-e85915810f52', 'options': 'recursive,force'} 2024-09-10T22:59:09.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:08 smithi096 ceph-mon[30869]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-09-10T22:59:09.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:08 smithi121 ceph-mon[35119]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-09-10T22:59:09.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:08 smithi117 ceph-mon[35081]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-09-10T22:59:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 695.0 s, 1452.2 tps, lat 68.646 ms stddev 70.894 2024-09-10T22:59:11.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:10 smithi096 ceph-mon[30869]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-09-10T22:59:11.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:10 smithi121 ceph-mon[35119]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-09-10T22:59:11.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:10 smithi117 ceph-mon[35081]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-09-10T22:59:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:12 smithi096 ceph-mon[30869]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 693 op/s 2024-09-10T22:59:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:12 smithi121 ceph-mon[35119]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 693 op/s 2024-09-10T22:59:13.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:12 smithi117 ceph-mon[35081]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 693 op/s 2024-09-10T22:59:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:15 smithi096 ceph-mon[30869]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.8 MiB/s wr, 704 op/s 2024-09-10T22:59:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:15 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:59:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:15 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T22:59:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:15 smithi121 ceph-mon[35119]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.8 MiB/s wr, 704 op/s 2024-09-10T22:59:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:15 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:59:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:15 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T22:59:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:15 smithi117 ceph-mon[35081]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 5.8 MiB/s wr, 704 op/s 2024-09-10T22:59:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:15 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:59:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:15 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T22:59:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 -- 172.21.15.96:0/1155987703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 msgr2=0x7f1160075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 --2- 172.21.15.96:0/1155987703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f1160075fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1154009900 tx=0x7f115402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 -- 172.21.15.96:0/1155987703 shutdown_connections 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 --2- 172.21.15.96:0/1155987703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11601064a0 0x7f1160111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 --2- 172.21.15.96:0/1155987703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f1160075fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 --2- 172.21.15.96:0/1155987703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1160076f90 0x7f11600755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 -- 172.21.15.96:0/1155987703 >> 172.21.15.96:0/1155987703 conn(0x7f11600fe250 msgr2=0x7f1160100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 -- 172.21.15.96:0/1155987703 shutdown_connections 2024-09-10T22:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.934+0000 7f1167da0640 1 -- 172.21.15.96:0/1155987703 wait complete. 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 Processor -- start 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 -- start start 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f116019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 0x7f11601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114c30 con 0x7f11601a0610 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114ab0 con 0x7f1160076f90 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.935+0000 7f1167da0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1160114db0 con 0x7f1160075b20 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116533b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116533b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33656/0 (socket says 172.21.15.96:33656) 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116533b640 1 -- 172.21.15.96:0/179905609 learned_addr learned my addr 172.21.15.96:0/179905609 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f116019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116633d640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 0x7f11601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 -- 172.21.15.96:0/179905609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 msgr2=0x7f116019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 -- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 msgr2=0x7f11601a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 0x7f11601a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f1165b3c640 1 -- 172.21.15.96:0/179905609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11601a12c0 con 0x7f1160075b20 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116633d640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 0x7f11601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.936+0000 7f116533b640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1165b3c640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f116019f390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e9f0 tx=0x7f115c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1152ffd640 1 -- 172.21.15.96:0/179905609 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c00ccc0 con 0x7f1160075b20 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1152ffd640 1 -- 172.21.15.96:0/179905609 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115c0108e0 con 0x7f1160075b20 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1152ffd640 1 -- 172.21.15.96:0/179905609 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0188e0 con 0x7f1160075b20 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a76b0 con 0x7f1160075b20 2024-09-10T22:59:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.937+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a7b50 con 0x7f1160075b20 2024-09-10T22:59:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 Processor -- start 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 -- start start 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x55708259bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259c190 0x55708259e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55708250b6f0 con 0x55708259ebd0 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570824fab20 con 0x557082599860 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.940+0000 7f116ac53740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5570824fe930 con 0x55708259c190 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f1152ffd640 1 -- 172.21.15.96:0/179905609 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f115c010430 con 0x7f1160075b20 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48626/0 (socket says 172.21.15.96:48626) 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 -- 172.21.15.96:0/3377991039 learned_addr learned my addr 172.21.15.96:0/3377991039 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116633d640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x55708259bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f1165b3c640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259c190 0x55708259e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 -- 172.21.15.96:0/3377991039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259c190 msgr2=0x55708259e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259c190 0x55708259e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 -- 172.21.15.96:0/3377991039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 msgr2=0x55708259bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x55708259bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 -- 172.21.15.96:0/3377991039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570824c0530 con 0x55708259ebd0 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116633d640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x55708259bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f116533b640 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1154000c00 tx=0x7f1154032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189f38749812ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f114bfff640 1 -- 172.21.15.96:0/3377991039 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154035040 con 0x55708259ebd0 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f1152ffd640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 0x7f114c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f114bfff640 1 -- 172.21.15.96:0/3377991039 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11540318a0 con 0x55708259ebd0 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.941+0000 7f114bfff640 1 -- 172.21.15.96:0/3377991039 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154004a00 con 0x55708259ebd0 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116633d640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 0x7f114c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f1152ffd640 1 -- 172.21.15.96:0/179905609 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f115c0e5e00 con 0x7f1160075b20 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 -- 172.21.15.96:0/3377991039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 msgr2=0x5570825a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116633d640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 0x7f114c0b7820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1154000c00 tx=0x7f1154032ed0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 -- 172.21.15.96:0/3377991039 shutdown_connections 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a1090 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259c190 0x55708259e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 --2- 172.21.15.96:0/3377991039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x55708259bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 -- 172.21.15.96:0/3377991039 >> 172.21.15.96:0/3377991039 conn(0x5570825949b0 msgr2=0x557082595740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 -- 172.21.15.96:0/3377991039 shutdown_connections 2024-09-10T22:59:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.942+0000 7f116ac53740 1 -- 172.21.15.96:0/3377991039 wait complete. 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.943+0000 7f116ac53740 1 -- start start 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.944+0000 7f116ac53740 1 Processor -- start 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.944+0000 7f116ac53740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557082432ab0 con 0x55708259ebd0 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570824a0730 con 0x55708259c190 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5570825a1b50 con 0x557082599860 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116ac53740 10 client.0 _ll_register_callbacks cb 0x5570824f46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f1165b3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f1165b3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:47512/0 (socket says 172.21.15.96:47512) 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f1165b3c640 1 -- 172.21.15.96:0/801244973 learned_addr learned my addr 172.21.15.96:0/801244973 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:33670/0 (socket says 172.21.15.96:33670) 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116633d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 -- 172.21.15.96:0/801244973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 msgr2=0x5570825a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 -- 172.21.15.96:0/801244973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 msgr2=0x5570825a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 -- 172.21.15.96:0/801244973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708225c790 con 0x55708259c190 2024-09-10T22:59:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f1165b3c640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f116533b640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1154005940 tx=0x7f115402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39cb766fd26c4af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f11497fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.945+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115404e030 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11540318a0 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f11540316a0 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1154035040 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 10 client.27635 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5570822d5620 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11540318a0 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f115403cab0 con 0x55708259c190 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 10 client.27635 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 10 client.27635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 20 client.27635 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5570825ae2d0 0x5570825b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 4 client.27635 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55708250d640 con 0x5570825ae2d0 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.946+0000 7f1165b3c640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5570825ae2d0 0x5570825b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.947+0000 7f1165b3c640 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5570825ae2d0 0x5570825b06c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434f769e9eacb0f5 server_cookie=9960ec7aef09b56d in_seq=0 out_seq=0 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.947+0000 7f11497fa640 10 client.27635 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f11497fa640 1 -- 172.21.15.96:0/801244973 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55708250d640 con 0x5570825ae2d0 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f11497fa640 10 client.27635 handle_command_reply: tid=0 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 client.27635 shutdown 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 msgr2=0x5570825a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1154005940 tx=0x7f115402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 shutdown_connections 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5570825ae2d0 0x5570825b06c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259c190 0x5570825a34d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259ebd0 0x5570825a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f11497fa640 0 client.27635 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 --2- 172.21.15.96:0/801244973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 >> 172.21.15.96:0/801244973 conn(0x5570825949b0 msgr2=0x557082593bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 shutdown_connections 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 1 -- 172.21.15.96:0/801244973 wait complete. 2024-09-10T22:59:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.948+0000 7f116ac53740 20 client.27635 trim_cache size 0 max 16384 2024-09-10T22:59:15.954 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 700.0 s, 1861.4 tps, lat 53.790 ms stddev 60.766 2024-09-10T22:59:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 Processor -- start 2024-09-10T22:59:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 -- start start 2024-09-10T22:59:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x5570825a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 0x5570825a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b03d0 con 0x55708259c190 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b1f10 con 0x557082599860 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116ac53740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b0aa0 con 0x55708259ebd0 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.965+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48648/0 (socket says 172.21.15.96:48648) 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 -- 172.21.15.96:0/960319834 learned_addr learned my addr 172.21.15.96:0/960319834 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f1165b3c640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x5570825a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116633d640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 0x5570825a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 -- 172.21.15.96:0/960319834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 msgr2=0x5570825a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 0x5570825a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 -- 172.21.15.96:0/960319834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 msgr2=0x5570825a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x5570825a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 -- 172.21.15.96:0/960319834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570825b0cf0 con 0x55708259c190 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116633d640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 0x5570825a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f1165b3c640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x5570825a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f116533b640 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f115403ce50 tx=0x7f1154047710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591e5c804c1de0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f114b7fe640 1 -- 172.21.15.96:0/960319834 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115404e020 con 0x55708259c190 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f114b7fe640 1 -- 172.21.15.96:0/960319834 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1154004030 con 0x55708259c190 2024-09-10T22:59:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.966+0000 7f114b7fe640 1 -- 172.21.15.96:0/960319834 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115403c8d0 con 0x55708259c190 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 -- 172.21.15.96:0/960319834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 msgr2=0x5570825a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f115403ce50 tx=0x7f1154047710 comp rx=0 tx=0).stop 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 -- 172.21.15.96:0/960319834 shutdown_connections 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55708259ebd0 0x5570825a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a2e30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 --2- 172.21.15.96:0/960319834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557082599860 0x5570825a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.967+0000 7f116ac53740 1 -- 172.21.15.96:0/960319834 >> 172.21.15.96:0/960319834 conn(0x5570825949b0 msgr2=0x557082598cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- 172.21.15.96:0/960319834 shutdown_connections 2024-09-10T22:59:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- 172.21.15.96:0/960319834 wait complete. 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- start start 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 Processor -- start 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 0x5570825aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b1840 con 0x55708259c190 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11601b1170 con 0x55708259ebd0 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11601ae3b0 con 0x557082599860 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116ac53740 10 client.0 _ll_register_callbacks cb 0x7f1154040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.968+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116633d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 0x5570825aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48656/0 (socket says 172.21.15.96:48656) 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 -- 172.21.15.96:0/2198588612 learned_addr learned my addr 172.21.15.96:0/2198588612 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f1165b3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 -- 172.21.15.96:0/2198588612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 msgr2=0x5570825a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 -- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 msgr2=0x5570825aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 0x5570825aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 -- 172.21.15.96:0/2198588612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570825a2c00 con 0x55708259c190 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116633d640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 0x5570825aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f116533b640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11540534d0 tx=0x7f1154005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f11d0ee5911346f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.969+0000 7f11497fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115404e030 con 0x55708259c190 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1154031a50 con 0x55708259c190 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1154004030 con 0x55708259c190 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1154031c50 con 0x55708259c190 2024-09-10T22:59:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f116ac53740 10 client.18033 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.970+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55708261b3b0 con 0x55708259c190 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1154031df0 con 0x55708259c190 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1154003a10 con 0x55708259c190 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 10 client.18033 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 10 client.18033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 20 client.18033 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55708261bd60 0x55708261e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 4 client.18033 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1154004030 con 0x55708261bd60 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.971+0000 7f1165b3c640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55708261bd60 0x55708261e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.972+0000 7f1165b3c640 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55708261bd60 0x55708261e180 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0968794131e133 server_cookie=145fbe49e20d35f1 in_seq=0 out_seq=0 2024-09-10T22:59:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.972+0000 7f11497fa640 10 client.18033 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.988+0000 7f11497fa640 1 -- 172.21.15.96:0/2198588612 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1154004030 con 0x55708261bd60 2024-09-10T22:59:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.988+0000 7f11497fa640 10 client.18033 handle_command_reply: tid=0 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 client.18033 shutdown 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 msgr2=0x5570825a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11540534d0 tx=0x7f1154005c90 comp rx=0 tx=0).stop 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 shutdown_connections 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55708261bd60 0x55708261e180 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55708259ebd0 0x5570825aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708259c190 0x5570825a9f40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f11497fa640 0 client.18033 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 --2- 172.21.15.96:0/2198588612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557082599860 0x5570825a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 >> 172.21.15.96:0/2198588612 conn(0x5570825949b0 msgr2=0x5570825a2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 shutdown_connections 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 1 -- 172.21.15.96:0/2198588612 wait complete. 2024-09-10T22:59:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f116ac53740 20 client.18033 trim_cache size 0 max 16384 2024-09-10T22:59:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:59:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:59:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:59:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 msgr2=0x7f114c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 0x7f114c0b7820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1158005fd0 tx=0x7f1158009500 comp rx=0 tx=0).stop 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 msgr2=0x7f116019f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.989+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f116019f390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e9f0 tx=0x7f115c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 shutdown_connections 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1160076f90 0x7f116019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f114c0b5360 0x7f114c0b7820 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11601a0610 0x7f11601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 --2- 172.21.15.96:0/179905609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1160075b20 0x7f116019f390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 >> 172.21.15.96:0/179905609 conn(0x7f11600fe250 msgr2=0x7f11600fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 shutdown_connections 2024-09-10T22:59:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:15.990+0000 7f1167da0640 1 -- 172.21.15.96:0/179905609 wait complete. 2024-09-10T22:59:17.577 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:17 smithi117 ceph-mon[35081]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-09-10T22:59:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:17 smithi096 ceph-mon[30869]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-09-10T22:59:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:17 smithi121 ceph-mon[35119]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-09-10T22:59:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:19 smithi096 ceph-mon[30869]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.0 MiB/s wr, 625 op/s 2024-09-10T22:59:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:19 smithi121 ceph-mon[35119]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.0 MiB/s wr, 625 op/s 2024-09-10T22:59:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:19 smithi117 ceph-mon[35081]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.0 MiB/s wr, 625 op/s 2024-09-10T22:59:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 705.0 s, 2069.4 tps, lat 48.531 ms stddev 52.490 2024-09-10T22:59:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:21 smithi096 ceph-mon[30869]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-10T22:59:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:21 smithi121 ceph-mon[35119]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-10T22:59:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:21 smithi117 ceph-mon[35081]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-10T22:59:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:23 smithi096 ceph-mon[30869]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-09-10T22:59:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:23 smithi121 ceph-mon[35119]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-09-10T22:59:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:23 smithi117 ceph-mon[35081]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-09-10T22:59:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:25 smithi096 ceph-mon[30869]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-09-10T22:59:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:25 smithi121 ceph-mon[35119]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-09-10T22:59:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:25 smithi117 ceph-mon[35081]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-09-10T22:59:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 710.0 s, 2490.0 tps, lat 40.095 ms stddev 43.376 2024-09-10T22:59:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:27 smithi096 ceph-mon[30869]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 MiB/s wr, 716 op/s 2024-09-10T22:59:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:27 smithi121 ceph-mon[35119]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 MiB/s wr, 716 op/s 2024-09-10T22:59:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:27 smithi117 ceph-mon[35081]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 MiB/s wr, 716 op/s 2024-09-10T22:59:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.106+0000 7f85e6252640 1 -- 172.21.15.96:0/2234696437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0105ad0 msgr2=0x7f85e0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.106+0000 7f85e6252640 1 --2- 172.21.15.96:0/2234696437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0105ad0 0x7f85e0107f40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.107+0000 7f85e6252640 1 -- 172.21.15.96:0/2234696437 shutdown_connections 2024-09-10T22:59:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 --2- 172.21.15.96:0/2234696437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0108680 0x7f85e010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 --2- 172.21.15.96:0/2234696437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0105ad0 0x7f85e0107f40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 --2- 172.21.15.96:0/2234696437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 -- 172.21.15.96:0/2234696437 >> 172.21.15.96:0/2234696437 conn(0x7f85e00fd130 msgr2=0x7f85e00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 -- 172.21.15.96:0/2234696437 shutdown_connections 2024-09-10T22:59:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.108+0000 7f85e6252640 1 -- 172.21.15.96:0/2234696437 wait complete. 2024-09-10T22:59:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 Processor -- start 2024-09-10T22:59:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 -- start start 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0105ad0 0x7f85e01014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85e010e600 con 0x7f85e0108680 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85e010e480 con 0x7f85e0105ad0 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.109+0000 7f85e6252640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85e010e780 con 0x7f85e006bc90 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54934/0 (socket says 172.21.15.96:54934) 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 -- 172.21.15.96:0/44370902 learned_addr learned my addr 172.21.15.96:0/44370902 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85df7fe640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85deffd640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0105ad0 0x7f85e01014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 -- 172.21.15.96:0/44370902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 msgr2=0x7f85e0102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 -- 172.21.15.96:0/44370902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0105ad0 msgr2=0x7f85e01014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0105ad0 0x7f85e01014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 -- 172.21.15.96:0/44370902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e0102290 con 0x7f85e0108680 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85df7fe640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dffff640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009fd0 tx=0x7f85d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dcff9640 1 -- 172.21.15.96:0/44370902 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8017070 con 0x7f85e0108680 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dcff9640 1 -- 172.21.15.96:0/44370902 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d8003e20 con 0x7f85e0108680 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85dcff9640 1 -- 172.21.15.96:0/44370902 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8010460 con 0x7f85e0108680 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e01a2e70 con 0x7f85e0108680 2024-09-10T22:59:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.110+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e01a3220 con 0x7f85e0108680 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.113+0000 7f85dcff9640 1 -- 172.21.15.96:0/44370902 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f85d8003970 con 0x7f85e0108680 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.113+0000 7f85e9022740 1 Processor -- start 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 -- start start 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7789c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597d76f96f0 con 0x5597d778a190 2024-09-10T22:59:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597d76e8b20 con 0x5597d7787860 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85e9022740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5597d76ec930 con 0x5597d778cbd0 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54950/0 (socket says 172.21.15.96:54950) 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 -- 172.21.15.96:0/1371780460 learned_addr learned my addr 172.21.15.96:0/1371780460 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85dcff9640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 0x7f85c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85deffd640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7789c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85deffd640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85dffff640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 0x7f85c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 -- 172.21.15.96:0/1371780460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 msgr2=0x5597d778f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 -- 172.21.15.96:0/1371780460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 msgr2=0x5597d7789c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7789c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 -- 172.21.15.96:0/1371780460 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d76ae530 con 0x5597d778a190 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85deffd640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85dcff9640 1 -- 172.21.15.96:0/44370902 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f85d80e5970 con 0x7f85e0108680 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.114+0000 7f85df7fe640 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009d60 tx=0x7f85d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a85223dcb3a25d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85dffff640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 0x7f85c80b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f85d80a6800 tx=0x7f85d80dc640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85c5ffb640 1 -- 172.21.15.96:0/1371780460 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0012650 con 0x5597d778a190 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85c5ffb640 1 -- 172.21.15.96:0/1371780460 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0009e50 con 0x5597d778a190 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85c5ffb640 1 -- 172.21.15.96:0/1371780460 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d000fc00 con 0x5597d778a190 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85e9022740 1 -- 172.21.15.96:0/1371780460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 msgr2=0x5597d778c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.115+0000 7f85e9022740 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009d60 tx=0x7f85d00117b0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 -- 172.21.15.96:0/1371780460 shutdown_connections 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778c580 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 --2- 172.21.15.96:0/1371780460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7789c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 -- 172.21.15.96:0/1371780460 >> 172.21.15.96:0/1371780460 conn(0x5597d77829b0 msgr2=0x5597d7783740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 -- 172.21.15.96:0/1371780460 shutdown_connections 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 -- 172.21.15.96:0/1371780460 wait complete. 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 -- start start 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 Processor -- start 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7790f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.116+0000 7f85e9022740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 0x5597d7791620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85e9022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778cbd0 0x5597d77921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 0x5597d7791620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 0x5597d7791620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51088/0 (socket says 172.21.15.96:51088) 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85df7fe640 1 -- 172.21.15.96:0/1891238129 learned_addr learned my addr 172.21.15.96:0/1891238129 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597d7620ab0 con 0x5597d778cbd0 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597d768e730 con 0x5597d7787860 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5597d778fbb0 con 0x5597d778a190 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7790f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778cbd0 0x5597d77921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85e9022740 10 client.0 _ll_register_callbacks cb 0x5597d76e26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 -- 172.21.15.96:0/1891238129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 msgr2=0x5597d7791620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 0x5597d7791620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 -- 172.21.15.96:0/1891238129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778cbd0 msgr2=0x5597d77921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778cbd0 0x5597d77921f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.117+0000 7f85deffd640 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d744a790 con 0x5597d7787860 2024-09-10T22:59:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d0009e50 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85deffd640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7790f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85d402aa00 tx=0x7f85d402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b507382505deb5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85beffd640 10 client.27653 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85beffd640 1 -- 172.21.15.96:0/1891238129 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d404f050 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85beffd640 1 -- 172.21.15.96:0/1891238129 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f85d4004250 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85beffd640 1 -- 172.21.15.96:0/1891238129 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85d4035040 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85e9022740 10 client.27653 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.118+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5597d74c3620 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85beffd640 1 -- 172.21.15.96:0/1891238129 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f85d403a040 con 0x5597d7787860 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 10 client.27653 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 10 client.27653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 20 client.27653 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a4db0 0x5597d77a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 4 client.27653 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.119+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597d76fb640 con 0x5597d77a4db0 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.120+0000 7f85df7fe640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a4db0 0x5597d77a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.120+0000 7f85beffd640 10 client.27653 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.120+0000 7f85df7fe640 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a4db0 0x5597d77a71a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd60196cead7b53 server_cookie=9fa8e63e94a73a87 in_seq=0 out_seq=0 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85beffd640 1 -- 172.21.15.96:0/1891238129 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597d76fb640 con 0x5597d77a4db0 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85beffd640 10 client.27653 handle_command_reply: tid=0 2024-09-10T22:59:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 client.27653 shutdown 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 msgr2=0x5597d7790f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7790f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85d402aa00 tx=0x7f85d402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 shutdown_connections 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a4db0 0x5597d77a71a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778cbd0 0x5597d77921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778a190 0x5597d7791620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85beffd640 0 client.27653 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 --2- 172.21.15.96:0/1891238129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d7790f80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 >> 172.21.15.96:0/1891238129 conn(0x5597d77829b0 msgr2=0x5597d7781bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 shutdown_connections 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 1 -- 172.21.15.96:0/1891238129 wait complete. 2024-09-10T22:59:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.121+0000 7f85e9022740 20 client.27653 trim_cache size 0 max 16384 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 Processor -- start 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 -- start start 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d77901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01abf80 con 0x5597d778a190 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01adac0 con 0x5597d7787860 2024-09-10T22:59:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.153+0000 7f85e9022740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01ac650 con 0x5597d778cbd0 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51096/0 (socket says 172.21.15.96:51096) 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 -- 172.21.15.96:0/594734604 learned_addr learned my addr 172.21.15.96:0/594734604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 -- 172.21.15.96:0/594734604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 msgr2=0x5597d77901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d77901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 -- 172.21.15.96:0/594734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 msgr2=0x5597d778cfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778cfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.154+0000 7f85deffd640 1 -- 172.21.15.96:0/594734604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d777e0b0 con 0x5597d778cbd0 2024-09-10T22:59:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.155+0000 7f85deffd640 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000c00 tx=0x7f85d4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4a0026157880cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.155+0000 7f85c57fa640 1 -- 172.21.15.96:0/594734604 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d404f070 con 0x5597d778cbd0 2024-09-10T22:59:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.155+0000 7f85c57fa640 1 -- 172.21.15.96:0/594734604 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0009e50 con 0x5597d778cbd0 2024-09-10T22:59:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.156+0000 7f85e9022740 1 -- 172.21.15.96:0/594734604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 msgr2=0x5597d778d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.156+0000 7f85e9022740 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000c00 tx=0x7f85d4005b50 comp rx=0 tx=0).stop 2024-09-10T22:59:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.162+0000 7f85e9022740 1 -- 172.21.15.96:0/594734604 shutdown_connections 2024-09-10T22:59:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.162+0000 7f85e9022740 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d778d4f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.162+0000 7f85e9022740 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d778a190 0x5597d778cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.162+0000 7f85e9022740 1 --2- 172.21.15.96:0/594734604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d7787860 0x5597d77901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.162+0000 7f85e9022740 1 -- 172.21.15.96:0/594734604 >> 172.21.15.96:0/594734604 conn(0x5597d77829b0 msgr2=0x5597d7786d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.172+0000 7f85e9022740 1 -- 172.21.15.96:0/594734604 shutdown_connections 2024-09-10T22:59:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.176+0000 7f85e9022740 1 -- 172.21.15.96:0/594734604 wait complete. 2024-09-10T22:59:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.176+0000 7f85e9022740 1 -- start start 2024-09-10T22:59:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.176+0000 7f85e9022740 1 Processor -- start 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d7787860 0x5597d7797e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01ad3f0 con 0x5597d7787860 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01acd20 con 0x5597d778a190 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85e01a9f60 con 0x5597d778cbd0 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.179+0000 7f85e9022740 10 client.0 _ll_register_callbacks cb 0x7f85d0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.180+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51108/0 (socket says 172.21.15.96:51108) 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85deffd640 1 -- 172.21.15.96:0/3360263995 learned_addr learned my addr 172.21.15.96:0/3360263995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85deffd640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d7787860 0x5597d7797e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85deffd640 1 -- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 msgr2=0x5597d77983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.184+0000 7f85df7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35664/0 (socket says 172.21.15.96:35664) 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85deffd640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85deffd640 1 -- 172.21.15.96:0/3360263995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d7787860 msgr2=0x5597d7797e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85deffd640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d7787860 0x5597d7797e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85deffd640 1 -- 172.21.15.96:0/3360263995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d7789fc0 con 0x5597d778cbd0 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85df7fe640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.185+0000 7f85deffd640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009610 tx=0x7f85d40056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6db14a2d6f225f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.186+0000 7f85beffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.186+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d404f040 con 0x5597d778cbd0 2024-09-10T22:59:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.186+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f85d40548f0 con 0x5597d778cbd0 2024-09-10T22:59:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.187+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d0009e50 con 0x5597d778cbd0 2024-09-10T22:59:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.187+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85d4054af0 con 0x5597d778cbd0 2024-09-10T22:59:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.188+0000 7f85e9022740 10 client.27703 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.188+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5597d77a7510 con 0x5597d778cbd0 2024-09-10T22:59:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.188+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0009e50 con 0x5597d778cbd0 2024-09-10T22:59:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f85d40493f0 con 0x5597d778cbd0 2024-09-10T22:59:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 10 client.27703 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 10 client.27703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 20 client.27703 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a1e90 0x5597d77a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 4 client.27703 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.189+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597d76f70f0 con 0x5597d77a1e90 2024-09-10T22:59:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.192+0000 7f85df7fe640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a1e90 0x5597d77a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.192+0000 7f85df7fe640 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a1e90 0x5597d77a22a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d257d7c6c0d29cc6 server_cookie=130d24233b4af1f5 in_seq=0 out_seq=0 2024-09-10T22:59:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.193+0000 7f85beffd640 10 client.27703 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.208+0000 7f85beffd640 1 -- 172.21.15.96:0/3360263995 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597d76f70f0 con 0x5597d77a1e90 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.208+0000 7f85beffd640 10 client.27703 handle_command_reply: tid=0 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 client.27703 shutdown 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 msgr2=0x5597d7796960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009610 tx=0x7f85d40056d0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 shutdown_connections 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5597d77a1e90 0x5597d77a22a0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597d778a190 0x5597d77983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597d778cbd0 0x5597d7796960 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 --2- 172.21.15.96:0/3360263995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597d7787860 0x5597d7797e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 >> 172.21.15.96:0/3360263995 conn(0x5597d77829b0 msgr2=0x5597d7786950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85beffd640 0 client.27703 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 shutdown_connections 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 1 -- 172.21.15.96:0/3360263995 wait complete. 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.209+0000 7f85e9022740 20 client.27703 trim_cache size 0 max 16384 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:59:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 msgr2=0x7f85c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 0x7f85c80b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f85d80a6800 tx=0x7f85d80dc640 comp rx=0 tx=0).stop 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 msgr2=0x7f85e0101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009fd0 tx=0x7f85d800b260 comp rx=0 tx=0).stop 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 shutdown_connections 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e0105ad0 0x7f85e01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f85c80b52e0 0x7f85c80b77a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85e0108680 0x7f85e0101a20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 --2- 172.21.15.96:0/44370902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85e006bc90 0x7f85e0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 >> 172.21.15.96:0/44370902 conn(0x7f85e00fd130 msgr2=0x7f85e0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 shutdown_connections 2024-09-10T22:59:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:29.210+0000 7f85e6252640 1 -- 172.21.15.96:0/44370902 wait complete. 2024-09-10T22:59:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:29 smithi096 ceph-mon[30869]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-10T22:59:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:29 smithi121 ceph-mon[35119]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-10T22:59:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:29 smithi117 ceph-mon[35081]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-10T22:59:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 715.0 s, 2345.4 tps, lat 42.253 ms stddev 47.163 2024-09-10T22:59:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:31 smithi096 ceph-mon[30869]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-09-10T22:59:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:31 smithi121 ceph-mon[35119]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-09-10T22:59:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:31 smithi117 ceph-mon[35081]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-09-10T22:59:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:33 smithi096 ceph-mon[30869]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 640 op/s 2024-09-10T22:59:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:33 smithi121 ceph-mon[35119]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 640 op/s 2024-09-10T22:59:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:33 smithi117 ceph-mon[35081]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 640 op/s 2024-09-10T22:59:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:35 smithi096 ceph-mon[30869]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 732 op/s 2024-09-10T22:59:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:35 smithi121 ceph-mon[35119]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 732 op/s 2024-09-10T22:59:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:35 smithi117 ceph-mon[35081]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.5 MiB/s wr, 732 op/s 2024-09-10T22:59:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 720.0 s, 1699.6 tps, lat 58.850 ms stddev 67.994 2024-09-10T22:59:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:37 smithi096 ceph-mon[30869]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-09-10T22:59:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:37 smithi121 ceph-mon[35119]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-09-10T22:59:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:37 smithi117 ceph-mon[35081]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 718 op/s 2024-09-10T22:59:38.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:39.175 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:39.593 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:39 smithi096 ceph-mon[30869]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-09-10T22:59:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:39 smithi121 ceph-mon[35119]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-09-10T22:59:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:39 smithi117 ceph-mon[35081]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 573 op/s 2024-09-10T22:59:40.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.003+0000 7f6597878640 1 -- 172.21.15.96:0/1485916789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 msgr2=0x7f65900754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.003+0000 7f6597878640 1 --2- 172.21.15.96:0/1485916789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f65900754b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02a440 comp rx=0 tx=0).stop 2024-09-10T22:59:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.004+0000 7f6597878640 1 -- 172.21.15.96:0/1485916789 shutdown_connections 2024-09-10T22:59:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.004+0000 7f6597878640 1 --2- 172.21.15.96:0/1485916789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f65900754b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.004+0000 7f6597878640 1 --2- 172.21.15.96:0/1485916789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65900759f0 0x7f659007c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.004+0000 7f6597878640 1 --2- 172.21.15.96:0/1485916789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f659010a3b0 0x7f659010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.004+0000 7f6597878640 1 -- 172.21.15.96:0/1485916789 >> 172.21.15.96:0/1485916789 conn(0x7f659006dfc0 msgr2=0x7f659006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.006+0000 7f6597878640 1 -- 172.21.15.96:0/1485916789 shutdown_connections 2024-09-10T22:59:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.006+0000 7f6597878640 1 -- 172.21.15.96:0/1485916789 wait complete. 2024-09-10T22:59:40.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.007+0000 7f6597878640 1 Processor -- start 2024-09-10T22:59:40.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.011+0000 7f6597878640 1 -- start start 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.011+0000 7f6597878640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f6590085990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.011+0000 7f6597878640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 0x7f6590080bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.011+0000 7f6597878640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659010a3b0 0x7f6590081100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.011+0000 7f6597878640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f659007f5e0 con 0x7f65900759f0 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6597878640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f659007f460 con 0x7f6590075070 2024-09-10T22:59:40.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6597878640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f659007f760 con 0x7f659010a3b0 2024-09-10T22:59:40.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6594e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 0x7f6590080bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6594e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 0x7f6590080bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39590/0 (socket says 172.21.15.96:39590) 2024-09-10T22:59:40.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6594e13640 1 -- 172.21.15.96:0/3331393508 learned_addr learned my addr 172.21.15.96:0/3331393508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:40.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f6590085990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595e15640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659010a3b0 0x7f6590081100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 -- 172.21.15.96:0/3331393508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659010a3b0 msgr2=0x7f6590081100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659010a3b0 0x7f6590081100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 -- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 msgr2=0x7f6590080bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 0x7f6590080bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.012+0000 7f6595614640 1 -- 172.21.15.96:0/3331393508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65900817a0 con 0x7f6590075070 2024-09-10T22:59:40.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.013+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f6590085990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f658000b3f0 tx=0x7f658000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.014+0000 7f65867fc640 1 -- 172.21.15.96:0/3331393508 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580015020 con 0x7f6590075070 2024-09-10T22:59:40.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.015+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901c5710 con 0x7f6590075070 2024-09-10T22:59:40.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.015+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901c5b00 con 0x7f6590075070 2024-09-10T22:59:40.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.015+0000 7f65867fc640 1 -- 172.21.15.96:0/3331393508 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6580011ce0 con 0x7f6590075070 2024-09-10T22:59:40.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.015+0000 7f65867fc640 1 -- 172.21.15.96:0/3331393508 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580010d80 con 0x7f6590075070 2024-09-10T22:59:40.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.019+0000 7f6599c4d740 1 Processor -- start 2024-09-10T22:59:40.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.021+0000 7f65867fc640 1 -- 172.21.15.96:0/3331393508 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f658001a460 con 0x7f6590075070 2024-09-10T22:59:40.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 -- start start 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad07767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ad06e0e50 con 0x557ad0776d30 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ad06d0280 con 0x557ad076ada0 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6599c4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557ad06d4090 con 0x557ad076a480 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6595e15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6595e15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54452/0 (socket says 172.21.15.96:54452) 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6595e15640 1 -- 172.21.15.96:0/1421056189 learned_addr learned my addr 172.21.15.96:0/1421056189 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.022+0000 7f6595e15640 1 -- 172.21.15.96:0/1421056189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 msgr2=0x557ad07767f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f6595e15640 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad07767f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f6595e15640 1 -- 172.21.15.96:0/1421056189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 msgr2=0x557ad0779120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f6595e15640 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f6595e15640 1 -- 172.21.15.96:0/1421056189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad042e650 con 0x557ad076a480 2024-09-10T22:59:40.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f6595e15640 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6588010370 tx=0x7f6588010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c277bdd87cc6020c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f657b7fe640 1 -- 172.21.15.96:0/1421056189 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016070 con 0x557ad076a480 2024-09-10T22:59:40.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.023+0000 7f657b7fe640 1 -- 172.21.15.96:0/1421056189 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6588004d20 con 0x557ad076a480 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.024+0000 7f6599c4d740 1 -- 172.21.15.96:0/1421056189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 msgr2=0x557ad076a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.024+0000 7f65867fc640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 0x7f657c0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.024+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6588010370 tx=0x7f6588010830 comp rx=0 tx=0).stop 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f65867fc640 1 -- 172.21.15.96:0/3331393508 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f65800e52d0 con 0x7f6590075070 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f6599c4d740 1 -- 172.21.15.96:0/1421056189 shutdown_connections 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad07767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1421056189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076a480 0x557ad076a860 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.025+0000 7f6599c4d740 1 -- 172.21.15.96:0/1421056189 >> 172.21.15.96:0/1421056189 conn(0x557ad07682b0 msgr2=0x557ad0769450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.027+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 0x7f657c0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.028+0000 7f6595614640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 0x7f657c0b7970 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f65800aba20 tx=0x7f65800046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.028+0000 7f6599c4d740 1 -- 172.21.15.96:0/1421056189 shutdown_connections 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.028+0000 7f6599c4d740 1 -- 172.21.15.96:0/1421056189 wait complete. 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.029+0000 7f6599c4d740 1 -- start start 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.029+0000 7f6599c4d740 1 Processor -- start 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.029+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0776610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0773a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ad0531700 con 0x557ad076a480 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ad066b110 con 0x557ad076ada0 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557ad0779a00 con 0x557ad0776d30 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6599c4d740 10 client.0 _ll_register_callbacks cb 0x557ad0684990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:40.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54462/0 (socket says 172.21.15.96:54462) 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 -- 172.21.15.96:0/456364339 learned_addr learned my addr 172.21.15.96:0/456364339 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0776610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6594e13640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0773a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 -- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 msgr2=0x557ad0773a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0773a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 -- 172.21.15.96:0/456364339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 msgr2=0x557ad0776610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0776610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.030+0000 7f6595e15640 1 -- 172.21.15.96:0/456364339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad0698c40 con 0x557ad0776d30 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6595e15640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6588004bb0 tx=0x7f6588012f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd008f8deba2feb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:40.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016030 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6588004d20 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6588012510 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6588012710 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6599c4d740 10 client.27715 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ad07853e0 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.031+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658801a410 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f658800f040 con 0x557ad0776d30 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 10 client.27715 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 20 client.27715 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad0786c20 0x557ad0789040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 4 client.27715 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:40.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6588004d20 con 0x557ad0786c20 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.032+0000 7f6594e13640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad0786c20 0x557ad0789040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.033+0000 7f6578ff9640 10 client.27715 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.033+0000 7f6594e13640 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad0786c20 0x557ad0789040 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02a62637c9cd1b0 server_cookie=9e8d041dfde0f8d9 in_seq=0 out_seq=0 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.033+0000 7f6578ff9640 1 -- 172.21.15.96:0/456364339 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6588004d20 con 0x557ad0786c20 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.033+0000 7f6578ff9640 10 client.27715 handle_command_reply: tid=0 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.033+0000 7f6599c4d740 1 client.27715 shutdown 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 msgr2=0x557ad0774090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6588004bb0 tx=0x7f6588012f00 comp rx=0 tx=0).stop 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 shutdown_connections 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad0786c20 0x557ad0789040 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0773a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0774090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6578ff9640 0 client.27715 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:40.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 --2- 172.21.15.96:0/456364339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0776610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 >> 172.21.15.96:0/456364339 conn(0x557ad07682b0 msgr2=0x557ad076de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 shutdown_connections 2024-09-10T22:59:40.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 1 -- 172.21.15.96:0/456364339 wait complete. 2024-09-10T22:59:40.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.034+0000 7f6599c4d740 20 client.27715 trim_cache size 0 max 16384 2024-09-10T22:59:40.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.063+0000 7f6599c4d740 1 Processor -- start 2024-09-10T22:59:40.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 -- start start 2024-09-10T22:59:40.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0769370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0778120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f657400e3e0 con 0x557ad076a480 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f657400ff20 con 0x557ad076ada0 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6599c4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f657400eab0 con 0x557ad0776d30 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39596/0 (socket says 172.21.15.96:39596) 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 -- 172.21.15.96:0/3404940936 learned_addr learned my addr 172.21.15.96:0/3404940936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6594e13640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0769370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0778120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 -- 172.21.15.96:0/3404940936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 msgr2=0x557ad0778120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0778120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 -- 172.21.15.96:0/3404940936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 msgr2=0x557ad0769370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0769370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 -- 172.21.15.96:0/3404940936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad0768d30 con 0x557ad076a480 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6594e13640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0769370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.064+0000 7f6595e15640 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6588005900 tx=0x7f6588005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c8b43eee415e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f657affd640 1 -- 172.21.15.96:0/3404940936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588039070 con 0x557ad076a480 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f657affd640 1 -- 172.21.15.96:0/3404940936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6588012510 con 0x557ad076a480 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f657affd640 1 -- 172.21.15.96:0/3404940936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588016070 con 0x557ad076a480 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 -- 172.21.15.96:0/3404940936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 msgr2=0x557ad0782d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6588005900 tx=0x7f6588005890 comp rx=0 tx=0).stop 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 -- 172.21.15.96:0/3404940936 shutdown_connections 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076ada0 0x557ad0769370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad0776d30 0x557ad0778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 --2- 172.21.15.96:0/3404940936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad076a480 0x557ad0782d10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.065+0000 7f6599c4d740 1 -- 172.21.15.96:0/3404940936 >> 172.21.15.96:0/3404940936 conn(0x557ad07682b0 msgr2=0x557ad07722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.066+0000 7f6599c4d740 1 -- 172.21.15.96:0/3404940936 shutdown_connections 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.066+0000 7f6599c4d740 1 -- 172.21.15.96:0/3404940936 wait complete. 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.066+0000 7f6599c4d740 1 -- start start 2024-09-10T22:59:40.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.066+0000 7f6599c4d740 1 Processor -- start 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 0x557ad077f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6595e15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6595e15640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49554/0 (socket says 172.21.15.96:49554) 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6594e13640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 0x557ad077f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.067+0000 7f6594e13640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 0x557ad077f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54470/0 (socket says 172.21.15.96:54470) 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6599c4d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 -- 172.21.15.96:0/1052540519 learned_addr learned my addr 172.21.15.96:0/1052540519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f657400f850 con 0x557ad0776d30 2024-09-10T22:59:40.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f657400f180 con 0x557ad076a480 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f657400c3c0 con 0x557ad076ada0 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6599c4d740 10 client.0 _ll_register_callbacks cb 0x7f6588009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 -- 172.21.15.96:0/1052540519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 msgr2=0x557ad077f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 0x557ad077f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 -- 172.21.15.96:0/1052540519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 msgr2=0x557ad0779d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.068+0000 7f6595e15640 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad077f960 con 0x557ad076a480 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.069+0000 7f6595e15640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6588012ea0 tx=0x7f6588004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec75550e648843b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.069+0000 7f6578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:40.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.069+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588039040 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.069+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6588033660 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.069+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6588012510 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6588016040 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 10 client.27668 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ad07f3fa0 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658801e070 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f658803f060 con 0x557ad076a480 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 10 client.27668 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:40.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 20 client.27668 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad07f45b0 0x557ad07f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 4 client.27668 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.070+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6588012510 con 0x557ad07f45b0 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.072+0000 7f6594e13640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad07f45b0 0x557ad07f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.072+0000 7f6594e13640 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad07f45b0 0x557ad07f69a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa0ce2fce0088bf server_cookie=600ec932d53caadf in_seq=0 out_seq=0 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.072+0000 7f6578ff9640 10 client.27668 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6578ff9640 1 -- 172.21.15.96:0/1052540519 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x7f6588012510 con 0x557ad07f45b0 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6578ff9640 10 client.27668 handle_command_reply: tid=0 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 client.27668 shutdown 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 msgr2=0x557ad077ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6588012ea0 tx=0x7f6588004e30 comp rx=0 tx=0).stop 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 shutdown_connections 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x557ad07f45b0 0x557ad07f69a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6578ff9640 0 client.27668 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ad0776d30 0x557ad0779d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557ad076ada0 0x557ad077f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 --2- 172.21.15.96:0/1052540519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ad076a480 0x557ad077ed80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.073+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 >> 172.21.15.96:0/1052540519 conn(0x557ad07682b0 msgr2=0x557ad07722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.074+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 shutdown_connections 2024-09-10T22:59:40.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.074+0000 7f6599c4d740 1 -- 172.21.15.96:0/1052540519 wait complete. 2024-09-10T22:59:40.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6599c4d740 20 client.27668 trim_cache size 0 max 16384 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:59:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:59:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.087 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.088 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:40.089 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:59:40.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 msgr2=0x7f657c0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 0x7f657c0b7970 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f65800aba20 tx=0x7f65800046c0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 msgr2=0x7f6590085990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f6590085990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f658000b3f0 tx=0x7f658000b8b0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 shutdown_connections 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f657c0b54b0 0x7f657c0b7970 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659010a3b0 0x7f6590081100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65900759f0 0x7f6590080bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 --2- 172.21.15.96:0/3331393508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6590075070 0x7f6590085990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.075+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 >> 172.21.15.96:0/3331393508 conn(0x7f659006dfc0 msgr2=0x7f65900733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.077+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 shutdown_connections 2024-09-10T22:59:40.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:40.077+0000 7f6597878640 1 -- 172.21.15.96:0/3331393508 wait complete. 2024-09-10T22:59:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:40 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:40 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:40 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:59:40.953 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 725.0 s, 1911.0 tps, lat 52.665 ms stddev 69.404 2024-09-10T22:59:40.955 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T22:59:40.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:59:41.402 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:41.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:41 smithi096 ceph-mon[30869]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-09-10T22:59:41.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:59:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:41 smithi121 ceph-mon[35119]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-09-10T22:59:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:59:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:41 smithi117 ceph-mon[35081]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-09-10T22:59:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:59:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:59:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 -- 172.21.15.96:0/3988691951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a38102dd0 msgr2=0x7f5a381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 --2- 172.21.15.96:0/3988691951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a38102dd0 0x7f5a381031b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009900 tx=0x7f5a3402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 -- 172.21.15.96:0/3988691951 shutdown_connections 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 --2- 172.21.15.96:0/3988691951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 0x7f5a3810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 --2- 172.21.15.96:0/3988691951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a381087e0 0x7f5a38108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 --2- 172.21.15.96:0/3988691951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a38102dd0 0x7f5a381031b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 -- 172.21.15.96:0/3988691951 >> 172.21.15.96:0/3988691951 conn(0x7f5a380fe800 msgr2=0x7f5a38100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.975+0000 7f5a407c5640 1 -- 172.21.15.96:0/3988691951 shutdown_connections 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.977+0000 7f5a407c5640 1 -- 172.21.15.96:0/3988691951 wait complete. 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.978+0000 7f5a407c5640 1 Processor -- start 2024-09-10T22:59:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.978+0000 7f5a407c5640 1 -- start start 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.978+0000 7f5a407c5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 0x7f5a38072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.978+0000 7f5a407c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 0x7f5a380728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.979+0000 7f5a407c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 0x7f5a3806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.979+0000 7f5a407c5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380770c0 con 0x7f5a381087e0 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.979+0000 7f5a407c5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38076f40 con 0x7f5a38109160 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.979+0000 7f5a407c5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38077240 con 0x7f5a38102dd0 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.984+0000 7f5a3dd60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 0x7f5a380728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.984+0000 7f5a3dd60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 0x7f5a380728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39622/0 (socket says 172.21.15.96:39622) 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.984+0000 7f5a3dd60640 1 -- 172.21.15.96:0/1085069201 learned_addr learned my addr 172.21.15.96:0/1085069201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.986+0000 7f5a3e561640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 0x7f5a38072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.986+0000 7f5a3ed62640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 0x7f5a3806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.990+0000 7f5a3e561640 1 -- 172.21.15.96:0/1085069201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 msgr2=0x7f5a3806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:41.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.990+0000 7f5a3e561640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 0x7f5a3806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.990+0000 7f5a3e561640 1 -- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 msgr2=0x7f5a380728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.990+0000 7f5a3e561640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 0x7f5a380728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.990+0000 7f5a3e561640 1 -- 172.21.15.96:0/1085069201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3806df70 con 0x7f5a38102dd0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.994+0000 7f5a3e561640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 0x7f5a38072360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009850 tx=0x7f5a340305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.995+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3403f070 con 0x7f5a38102dd0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.995+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a34004460 con 0x7f5a38102dd0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.995+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34035030 con 0x7f5a38102dd0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.996+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3806e200 con 0x7f5a38102dd0 2024-09-10T22:59:42.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.996+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3806e5d0 con 0x7f5a38102dd0 2024-09-10T22:59:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:41.998+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3806eaa0 con 0x7f5a38102dd0 2024-09-10T22:59:42.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.000+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f5a34003cc0 con 0x7f5a38102dd0 2024-09-10T22:59:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.001+0000 7f5a2f7fe640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 0x7f5a240b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.002+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5a341066c0 con 0x7f5a38102dd0 2024-09-10T22:59:42.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.003+0000 7f5a3dd60640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 0x7f5a240b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.004+0000 7f5a3dd60640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 0x7f5a240b7970 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a38101000 tx=0x7f5a2800b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.009+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5a3806e5d0 con 0x7f5a38102dd0 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 -- 172.21.15.96:0/2613803972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 msgr2=0x7fbf741124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/2613803972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741124f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009cd0 tx=0x7fbf6c02a8e0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 -- 172.21.15.96:0/2613803972 shutdown_connections 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/2613803972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf7410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/2613803972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741124f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/2613803972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 0x7fbf74074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 -- 172.21.15.96:0/2613803972 >> 172.21.15.96:0/2613803972 conn(0x7fbf7406fa40 msgr2=0x7fbf74071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 -- 172.21.15.96:0/2613803972 shutdown_connections 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.320+0000 7fbf7ac47640 1 -- 172.21.15.96:0/2613803972 wait complete. 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 Processor -- start 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 -- start start 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 0x7fbf741a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf741a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74115cf0 con 0x7fbf7410bc20 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74115b70 con 0x7fbf7410b260 2024-09-10T22:59:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf7ac47640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74115e70 con 0x7fbf74074290 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39652/0 (socket says 172.21.15.96:39652) 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf791e4640 1 -- 172.21.15.96:0/1802260837 learned_addr learned my addr 172.21.15.96:0/1802260837 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf73fff640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf741a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.321+0000 7fbf789e3640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 0x7fbf741a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 -- 172.21.15.96:0/1802260837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 msgr2=0x7fbf741a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 0x7fbf741a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 -- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 msgr2=0x7fbf741a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 -- 172.21.15.96:0/1802260837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741a30b0 con 0x7fbf7410b260 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf791e4640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf73fff640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf741a1bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e9f0 tx=0x7fbf6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf71ffb640 1 -- 172.21.15.96:0/1802260837 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000ccc0 con 0x7fbf7410b260 2024-09-10T22:59:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf71ffb640 1 -- 172.21.15.96:0/1802260837 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf600108e0 con 0x7fbf7410b260 2024-09-10T22:59:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf71ffb640 1 -- 172.21.15.96:0/1802260837 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60018880 con 0x7fbf7410b260 2024-09-10T22:59:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a33a0 con 0x7fbf7410b260 2024-09-10T22:59:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.322+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a9a70 con 0x7fbf7410b260 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 Processor -- start 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 -- start start 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3d190 0x55dea8c3f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3fbd0 0x55dea8c42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dea8bac6f0 con 0x55dea8c3fbd0 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dea8b9bb20 con 0x55dea8c3d190 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf7da19740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dea8b9f930 con 0x55dea8c3a860 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54502/0 (socket says 172.21.15.96:54502) 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 -- 172.21.15.96:0/2772243437 learned_addr learned my addr 172.21.15.96:0/2772243437 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf789e3640 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3d190 0x55dea8c3f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 -- 172.21.15.96:0/2772243437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3d190 msgr2=0x55dea8c3f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3d190 0x55dea8c3f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.326+0000 7fbf791e4640 1 -- 172.21.15.96:0/2772243437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3fbd0 msgr2=0x55dea8c42090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf791e4640 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3fbd0 0x55dea8c42090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf791e4640 1 -- 172.21.15.96:0/2772243437 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea8b61530 con 0x55dea8c3a860 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf791e4640 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0093e0 tx=0x7fbf6c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc40bb6ecac7545 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf71ffb640 1 -- 172.21.15.96:0/1802260837 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fbf60010430 con 0x7fbf7410b260 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf5effd640 1 -- 172.21.15.96:0/2772243437 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c049070 con 0x55dea8c3a860 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf5effd640 1 -- 172.21.15.96:0/2772243437 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6c036040 con 0x55dea8c3a860 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.327+0000 7fbf5effd640 1 -- 172.21.15.96:0/2772243437 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c004540 con 0x55dea8c3a860 2024-09-10T22:59:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf71ffb640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 0x7fbf640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf789e3640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 0x7fbf640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 -- 172.21.15.96:0/2772243437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 msgr2=0x55dea8c3cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0093e0 tx=0x7fbf6c009a20 comp rx=0 tx=0).stop 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf71ffb640 1 -- 172.21.15.96:0/1802260837 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbf600e5de0 con 0x7fbf7410b260 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 -- 172.21.15.96:0/2772243437 shutdown_connections 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3d190 0x55dea8c3f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3fbd0 0x55dea8c42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2772243437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c3cc50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 -- 172.21.15.96:0/2772243437 >> 172.21.15.96:0/2772243437 conn(0x55dea8c359b0 msgr2=0x55dea8c36740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 -- 172.21.15.96:0/2772243437 shutdown_connections 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf7da19740 1 -- 172.21.15.96:0/2772243437 wait complete. 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.328+0000 7fbf789e3640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 0x7fbf640b7820 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e40 tx=0x7fbf68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 -- start start 2024-09-10T22:59:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 Processor -- start 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c49520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dea8ad3ab0 con 0x55dea8c3a860 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dea8b41730 con 0x55dea8c3fbd0 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dea8c42b50 con 0x55dea8c3d190 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf7da19740 10 client.0 _ll_register_callbacks cb 0x55dea8b956b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf789e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf789e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39656/0 (socket says 172.21.15.96:39656) 2024-09-10T22:59:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf789e3640 1 -- 172.21.15.96:0/3949172581 learned_addr learned my addr 172.21.15.96:0/3949172581 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c49520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.329+0000 7fbf73fff640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 -- 172.21.15.96:0/3949172581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 msgr2=0x55dea8c48950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c48950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 -- 172.21.15.96:0/3949172581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 msgr2=0x55dea8c482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 -- 172.21.15.96:0/3949172581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea88fd790 con 0x55dea8c3fbd0 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf789e3640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf73fff640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf791e4640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c49520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c005e30 tx=0x7fbf6c004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1fd13ff546eb02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf53fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c054060 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbf6c003970 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6c036040 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf6c049070 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf7da19740 10 client.27686 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.330+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dea8976620 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6c036040 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbf6c030070 con 0x55dea8c3fbd0 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 10 client.27686 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 10 client.27686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 20 client.27686 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8c55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 4 client.27686 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.331+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea8bae640 con 0x55dea8c53750 2024-09-10T22:59:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.332+0000 7fbf789e3640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8c55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.332+0000 7fbf789e3640 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8c55b40 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3dd34682fc523bf server_cookie=593017ace405c117 in_seq=0 out_seq=0 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.333+0000 7fbf53fff640 10 client.27686 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.333+0000 7fbf53fff640 1 -- 172.21.15.96:0/3949172581 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dea8bae640 con 0x55dea8c53750 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.333+0000 7fbf53fff640 10 client.27686 handle_command_reply: tid=0 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.333+0000 7fbf7da19740 1 client.27686 shutdown 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 msgr2=0x55dea8c49520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c49520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c005e30 tx=0x7fbf6c004990 comp rx=0 tx=0).stop 2024-09-10T22:59:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.334+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a38073880 con 0x7f5a38102dd0 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 shutdown_connections 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8c55b40 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf53fff640 0 client.27686 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c49520 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 --2- 172.21.15.96:0/3949172581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 >> 172.21.15.96:0/3949172581 conn(0x55dea8c359b0 msgr2=0x55dea8c34bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 shutdown_connections 2024-09-10T22:59:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 1 -- 172.21.15.96:0/3949172581 wait complete. 2024-09-10T22:59:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.334+0000 7fbf7da19740 20 client.27686 trim_cache size 0 max 16384 2024-09-10T22:59:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.334+0000 7f5a2f7fe640 1 -- 172.21.15.96:0/1085069201 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f5a34049030 con 0x7f5a38102dd0 2024-09-10T22:59:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T22:59:42.342 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T22:59:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T22:59:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 msgr2=0x7f5a240b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 0x7f5a240b7970 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a38101000 tx=0x7f5a2800b2c0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 msgr2=0x7f5a38072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 0x7f5a38072360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009850 tx=0x7f5a340305d0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 shutdown_connections 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a38109160 0x7f5a3806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f5a240b54b0 0x7f5a240b7970 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a381087e0 0x7f5a380728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 --2- 172.21.15.96:0/1085069201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a38102dd0 0x7f5a38072360 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 >> 172.21.15.96:0/1085069201 conn(0x7f5a380fe800 msgr2=0x7f5a380fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 shutdown_connections 2024-09-10T22:59:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:42.337+0000 7f5a407c5640 1 -- 172.21.15.96:0/1085069201 wait complete. 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.354+0000 7fbf7da19740 1 Processor -- start 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.354+0000 7fbf7da19740 1 -- start start 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.354+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c40560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf7da19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b0320 con 0x55dea8c3a860 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf7da19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b1e60 con 0x55dea8c3fbd0 2024-09-10T22:59:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf7da19740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b09f0 con 0x55dea8c3d190 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf789e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54516/0 (socket says 172.21.15.96:54516) 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf789e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c49500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39668/0 (socket says 172.21.15.96:39668) 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 -- 172.21.15.96:0/436969961 learned_addr learned my addr 172.21.15.96:0/436969961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf791e4640 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c40560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 -- 172.21.15.96:0/436969961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 msgr2=0x55dea8c40560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c40560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 -- 172.21.15.96:0/436969961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 msgr2=0x55dea8c49500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c49500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 -- 172.21.15.96:0/436969961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea8c48080 con 0x55dea8c3d190 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf73fff640 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbf600ac680 tx=0x7fbf60009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32758c82bce25ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf5e7fc640 1 -- 172.21.15.96:0/436969961 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600b1120 con 0x55dea8c3d190 2024-09-10T22:59:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf5e7fc640 1 -- 172.21.15.96:0/436969961 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf600b5bd0 con 0x55dea8c3d190 2024-09-10T22:59:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.355+0000 7fbf5e7fc640 1 -- 172.21.15.96:0/436969961 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600ab620 con 0x55dea8c3d190 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 -- 172.21.15.96:0/436969961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 msgr2=0x55dea8c40020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbf600ac680 tx=0x7fbf60009700 comp rx=0 tx=0).stop 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 -- 172.21.15.96:0/436969961 shutdown_connections 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c40560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3d190 0x55dea8c40020 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 --2- 172.21.15.96:0/436969961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3a860 0x55dea8c49500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 -- 172.21.15.96:0/436969961 >> 172.21.15.96:0/436969961 conn(0x55dea8c359b0 msgr2=0x55dea8c39d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 -- 172.21.15.96:0/436969961 shutdown_connections 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.357+0000 7fbf7da19740 1 -- 172.21.15.96:0/436969961 wait complete. 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 -- start start 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 Processor -- start 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c4eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b1790 con 0x55dea8c3d190 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b10c0 con 0x55dea8c3fbd0 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ae300 con 0x55dea8c3a860 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.358+0000 7fbf7da19740 10 client.0 _ll_register_callbacks cb 0x7fbf6c047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c4eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39680/0 (socket says 172.21.15.96:39680) 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf791e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c4eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49628/0 (socket says 172.21.15.96:49628) 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf73fff640 1 -- 172.21.15.96:0/2207826318 learned_addr learned my addr 172.21.15.96:0/2207826318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 -- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 msgr2=0x55dea8c4eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c4eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 -- 172.21.15.96:0/2207826318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 msgr2=0x55dea8c4e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 -- 172.21.15.96:0/2207826318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea8c4f620 con 0x55dea8c3a860 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf73fff640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf789e3640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c4fca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55dea8c56260 tx=0x7fbf680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b436e3d32e759a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf53fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68036060 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf600b5bd0 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.359+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fbf68003d80 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf68031440 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 10 client.27745 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dea8cc0580 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf600b5bd0 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbf6803b070 con 0x55dea8c3a860 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 10 client.27745 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 10 client.27745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 20 client.27745 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8cc2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 4 client.27745 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dea8baa0f0 con 0x55dea8c53750 2024-09-10T22:59:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf73fff640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8cc2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.360+0000 7fbf53fff640 10 client.27745 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.361+0000 7fbf73fff640 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8cc2fa0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee75399cea8670e server_cookie=bb2ab446f49b3eb3 in_seq=0 out_seq=0 2024-09-10T22:59:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.437+0000 7fbf53fff640 1 -- 172.21.15.96:0/2207826318 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dea8baa0f0 con 0x55dea8c53750 2024-09-10T22:59:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.437+0000 7fbf53fff640 10 client.27745 handle_command_reply: tid=0 2024-09-10T22:59:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.438+0000 7fbf7da19740 1 client.27745 shutdown 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 msgr2=0x55dea8c4fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c4fca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55dea8c56260 tx=0x7fbf680046c0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 shutdown_connections 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dea8c53750 0x55dea8cc2fa0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf53fff640 0 client.27745 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dea8c3fbd0 0x55dea8c4eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dea8c3d190 0x55dea8c4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 --2- 172.21.15.96:0/2207826318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dea8c3a860 0x55dea8c4fca0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55dea8c56260 tx=0x7fbf680046c0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.441+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 >> 172.21.15.96:0/2207826318 conn(0x55dea8c359b0 msgr2=0x55dea8c39d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.443+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 shutdown_connections 2024-09-10T22:59:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.443+0000 7fbf7da19740 1 -- 172.21.15.96:0/2207826318 wait complete. 2024-09-10T22:59:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.444+0000 7fbf7da19740 20 client.27745 trim_cache size 0 max 16384 2024-09-10T22:59:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:59:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:59:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:59:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 msgr2=0x7fbf640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 0x7fbf640b7820 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e40 tx=0x7fbf68005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 msgr2=0x7fbf741a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf741a1bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e9f0 tx=0x7fbf6000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 shutdown_connections 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf7410b260 0x7fbf741a1bc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fbf640b5360 0x7fbf640b7820 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf7410bc20 0x7fbf741a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 --2- 172.21.15.96:0/1802260837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74074290 0x7fbf741a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.446+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 >> 172.21.15.96:0/1802260837 conn(0x7fbf7406fa40 msgr2=0x7fbf7410a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.447+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 shutdown_connections 2024-09-10T22:59:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:42.447+0000 7fbf7ac47640 1 -- 172.21.15.96:0/1802260837 wait complete. 2024-09-10T22:59:42.675 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:42 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1085069201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:42 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1085069201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:42.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:42 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1085069201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:59:43.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T22:59:43.511 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:43.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:43 smithi096 ceph-mon[30869]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-10T22:59:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:43 smithi121 ceph-mon[35119]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-10T22:59:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:43 smithi117 ceph-mon[35081]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 593 op/s 2024-09-10T22:59:44.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.043+0000 7fa21effd640 1 -- 172.21.15.96:0/1706490228 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035020 con 0x7fa220101cd0 2024-09-10T22:59:44.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 -- 172.21.15.96:0/1706490228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa220101cd0 msgr2=0x7fa2201020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 --2- 172.21.15.96:0/1706490228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa220101cd0 0x7fa2201020b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa210009900 tx=0x7fa21002a410 comp rx=0 tx=0).stop 2024-09-10T22:59:44.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 -- 172.21.15.96:0/1706490228 shutdown_connections 2024-09-10T22:59:44.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 --2- 172.21.15.96:0/1706490228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 0x7fa22010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 --2- 172.21.15.96:0/1706490228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2201076e0 0x7fa220107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 --2- 172.21.15.96:0/1706490228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa220101cd0 0x7fa2201020b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 -- 172.21.15.96:0/1706490228 >> 172.21.15.96:0/1706490228 conn(0x7fa2200fd720 msgr2=0x7fa2200ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 -- 172.21.15.96:0/1706490228 shutdown_connections 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.044+0000 7fa226273640 1 -- 172.21.15.96:0/1706490228 wait complete. 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.045+0000 7fa226273640 1 Processor -- start 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.045+0000 7fa226273640 1 -- start start 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa220101cd0 0x7fa220199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 0x7fa22019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa220111d20 con 0x7fa2201076e0 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa220111ba0 con 0x7fa220108060 2024-09-10T22:59:44.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa226273640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa220111ea0 con 0x7fa220101cd0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39702/0 (socket says 172.21.15.96:39702) 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 -- 172.21.15.96:0/2612490917 learned_addr learned my addr 172.21.15.96:0/2612490917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21ffff640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa220101cd0 0x7fa220199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa224810640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 0x7fa22019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 -- 172.21.15.96:0/2612490917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa220101cd0 msgr2=0x7fa220199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa220101cd0 0x7fa220199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 -- 172.21.15.96:0/2612490917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 msgr2=0x7fa22019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 0x7fa22019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 -- 172.21.15.96:0/2612490917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22019f3a0 con 0x7fa2201076e0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.046+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9f0 tx=0x7fa21400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.047+0000 7fa21d7fa640 1 -- 172.21.15.96:0/2612490917 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21400cd80 con 0x7fa2201076e0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.047+0000 7fa21d7fa640 1 -- 172.21.15.96:0/2612490917 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2140054d0 con 0x7fa2201076e0 2024-09-10T22:59:44.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.047+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22019f690 con 0x7fa2201076e0 2024-09-10T22:59:44.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.047+0000 7fa21d7fa640 1 -- 172.21.15.96:0/2612490917 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214010690 con 0x7fa2201076e0 2024-09-10T22:59:44.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.047+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa220112480 con 0x7fa2201076e0 2024-09-10T22:59:44.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.050+0000 7fa228648740 1 Processor -- start 2024-09-10T22:59:44.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.050+0000 7fa228648740 1 -- start start 2024-09-10T22:59:44.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.050+0000 7fa21d7fa640 1 -- 172.21.15.96:0/2612490917 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa214004e90 con 0x7fa2201076e0 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a21291a220 0x55a212925c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a2129285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a212887ea0 con 0x55a2129261b0 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2128772d0 con 0x55a212919900 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa228648740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a21287b0e0 con 0x55a21291a220 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a2129285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa224810640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa224810640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49658/0 (socket says 172.21.15.96:49658) 2024-09-10T22:59:44.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa224810640 1 -- 172.21.15.96:0/3687150130 learned_addr learned my addr 172.21.15.96:0/3687150130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.051+0000 7fa21ffff640 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a21291a220 0x55a212925c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 -- 172.21.15.96:0/3687150130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a21291a220 msgr2=0x55a212925c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a21291a220 0x55a212925c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 -- 172.21.15.96:0/3687150130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 msgr2=0x55a212919ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 -- 172.21.15.96:0/3687150130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2125d5650 con 0x55a2129261b0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa224810640 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a2129285a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa2140e3b00 tx=0x7fa2140e9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2709f772e281c9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa2067fc640 1 -- 172.21.15.96:0/3687150130 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140f9070 con 0x55a2129261b0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa21d7fa640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 0x7fa2080b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa2067fc640 1 -- 172.21.15.96:0/3687150130 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2140ac330 con 0x55a2129261b0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.052+0000 7fa2067fc640 1 -- 172.21.15.96:0/3687150130 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140ac170 con 0x55a2129261b0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa224810640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 0x7fa2080b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa21d7fa640 1 -- 172.21.15.96:0/2612490917 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa2140e6060 con 0x7fa2201076e0 2024-09-10T22:59:44.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa224810640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 0x7fa2080b78a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa218005f40 tx=0x7fa2180096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 -- 172.21.15.96:0/3687150130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 msgr2=0x55a2129285a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a2129285a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa2140e3b00 tx=0x7fa2140e9f80 comp rx=0 tx=0).stop 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 -- 172.21.15.96:0/3687150130 shutdown_connections 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a2129285a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a21291a220 0x55a212925c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 --2- 172.21.15.96:0/3687150130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a212919900 0x55a212919ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.053+0000 7fa228648740 1 -- 172.21.15.96:0/3687150130 >> 172.21.15.96:0/3687150130 conn(0x55a212917730 msgr2=0x55a2129188d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.054+0000 7fa228648740 1 -- 172.21.15.96:0/3687150130 shutdown_connections 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.054+0000 7fa228648740 1 -- 172.21.15.96:0/3687150130 wait complete. 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.054+0000 7fa228648740 1 -- start start 2024-09-10T22:59:44.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.055+0000 7fa228648740 1 Processor -- start 2024-09-10T22:59:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.055+0000 7fa228648740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 0x55a212925a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 0x55a2129234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2126d8700 con 0x55a212919900 2024-09-10T22:59:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49664/0 (socket says 172.21.15.96:49664) 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa21f7fe640 1 -- 172.21.15.96:0/1965518924 learned_addr learned my addr 172.21.15.96:0/1965518924 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa224810640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 0x55a2129234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a212812160 con 0x55a21291a220 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a212928e30 con 0x55a2129261b0 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa228648740 10 client.0 _ll_register_callbacks cb 0x55a21282b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.056+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 0x55a212925a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 -- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 msgr2=0x55a212922ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 -- 172.21.15.96:0/1965518924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 msgr2=0x55a212925a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 0x55a212925a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a21283fc90 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 0x55a212925a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa224810640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 0x55a2129234b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2180048a0 tx=0x7fa218004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a3eb05cca70f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.057+0000 7fa1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21801fe40 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa218017030 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2140ac330 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa21801c640 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 10 client.27757 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2129347d0 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa21801cc90 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa218028b30 con 0x55a2129261b0 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 10 client.27757 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 10 client.27757 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:59:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 20 client.27757 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a212936060 0x55a2129384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 4 client.27757 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2140ac330 con 0x55a212936060 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.058+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a212936060 0x55a2129384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.059+0000 7fa1ff7fe640 10 client.27757 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:59:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.059+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a212936060 0x55a2129384c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4dc3962c10634315 server_cookie=3743eacbbac8fef5 in_seq=0 out_seq=0 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1965518924 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa2140ac330 con 0x55a212936060 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa1ff7fe640 10 client.27757 handle_command_reply: tid=0 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 client.27757 shutdown 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 msgr2=0x55a2129234b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 0x55a2129234b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2180048a0 tx=0x7fa218004ab0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 shutdown_connections 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a212936060 0x55a2129384c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a212922ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2129261b0 0x55a2129234b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa1ff7fe640 0 client.27757 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:59:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 --2- 172.21.15.96:0/1965518924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a212919900 0x55a212925a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 >> 172.21.15.96:0/1965518924 conn(0x55a212917730 msgr2=0x55a21291d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 shutdown_connections 2024-09-10T22:59:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 1 -- 172.21.15.96:0/1965518924 wait complete. 2024-09-10T22:59:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.060+0000 7fa228648740 20 client.27757 trim_cache size 0 max 16384 2024-09-10T22:59:44.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.079+0000 7fa228648740 1 Processor -- start 2024-09-10T22:59:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 -- start start 2024-09-10T22:59:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a212932090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 0x55a2129223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 0x55a212918780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ac0f0 con 0x55a21291a220 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201adc30 con 0x55a2129261b0 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.082+0000 7fa228648740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ac7c0 con 0x55a212919900 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.084+0000 7fa224810640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 0x55a212918780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.084+0000 7fa224810640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 0x55a212918780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49670/0 (socket says 172.21.15.96:49670) 2024-09-10T22:59:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.084+0000 7fa224810640 1 -- 172.21.15.96:0/1251207859 learned_addr learned my addr 172.21.15.96:0/1251207859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.088+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a212932090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.088+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 0x55a2129223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.090+0000 7fa21f7fe640 1 -- 172.21.15.96:0/1251207859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 msgr2=0x55a212932090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.090+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a212932090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.090+0000 7fa21f7fe640 1 -- 172.21.15.96:0/1251207859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 msgr2=0x55a212918780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.090+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 0x55a212918780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.090+0000 7fa21f7fe640 1 -- 172.21.15.96:0/1251207859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2129181b0 con 0x55a21291a220 2024-09-10T22:59:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.095+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 0x55a2129223c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa2140ac520 tx=0x7fa2140dcc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5631554f28ebb67e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.097+0000 7fa205ffb640 1 -- 172.21.15.96:0/1251207859 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140df3b0 con 0x55a21291a220 2024-09-10T22:59:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.097+0000 7fa205ffb640 1 -- 172.21.15.96:0/1251207859 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2140ac330 con 0x55a21291a220 2024-09-10T22:59:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.097+0000 7fa205ffb640 1 -- 172.21.15.96:0/1251207859 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140df580 con 0x55a21291a220 2024-09-10T22:59:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.098+0000 7fa228648740 1 -- 172.21.15.96:0/1251207859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 msgr2=0x55a2129223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.098+0000 7fa228648740 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 0x55a2129223c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa2140ac520 tx=0x7fa2140dcc90 comp rx=0 tx=0).stop 2024-09-10T22:59:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.099+0000 7fa228648740 1 -- 172.21.15.96:0/1251207859 shutdown_connections 2024-09-10T22:59:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.099+0000 7fa228648740 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2129261b0 0x55a212918780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.099+0000 7fa228648740 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a21291a220 0x55a2129223c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.099+0000 7fa228648740 1 --2- 172.21.15.96:0/1251207859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a212932090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.099+0000 7fa228648740 1 -- 172.21.15.96:0/1251207859 >> 172.21.15.96:0/1251207859 conn(0x55a212917730 msgr2=0x55a212921750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.102+0000 7fa228648740 1 -- 172.21.15.96:0/1251207859 shutdown_connections 2024-09-10T22:59:44.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.104+0000 7fa228648740 1 -- 172.21.15.96:0/1251207859 wait complete. 2024-09-10T22:59:44.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.104+0000 7fa228648740 1 -- start start 2024-09-10T22:59:44.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.104+0000 7fa228648740 1 Processor -- start 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a21299ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a21299cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ad560 con 0x55a2129261b0 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ace90 con 0x55a21291a220 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201aa0f0 con 0x55a212919900 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.106+0000 7fa228648740 10 client.0 _ll_register_callbacks cb 0x7fa218009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.107+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.107+0000 7fa21ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a21299ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.107+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:49672/0 (socket says 172.21.15.96:49672) 2024-09-10T22:59:44.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.107+0000 7fa21f7fe640 1 -- 172.21.15.96:0/1405798834 learned_addr learned my addr 172.21.15.96:0/1405798834 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:44.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 -- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 msgr2=0x55a2129a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 -- 172.21.15.96:0/1405798834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 msgr2=0x55a21299cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:44.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a21299cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 -- 172.21.15.96:0/1405798834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2129a0de0 con 0x55a212919900 2024-09-10T22:59:44.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:44.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa21ffff640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a21299ffd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa210009230 tx=0x7fa2100044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88472e83d16313d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:44.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.108+0000 7fa1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035040 con 0x55a212919900 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa210005510 con 0x55a212919900 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2140ac330 con 0x55a212919900 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa2100046c0 con 0x55a212919900 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa228648740 10 client.27763 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.110+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a21292d7a0 con 0x55a212919900 2024-09-10T22:59:44.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.111+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2140ac330 con 0x55a212919900 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.111+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa210031ad0 con 0x55a212919900 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.111+0000 7fa228648740 10 client.27763 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.111+0000 7fa228648740 10 client.27763 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.111+0000 7fa228648740 20 client.27763 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.112+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a21292de80 0x55a21292e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.112+0000 7fa228648740 4 client.27763 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:44.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.112+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a212889df0 con 0x55a21292de80 2024-09-10T22:59:44.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.112+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a21292de80 0x55a21292e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:44.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa21f7fe640 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a21292de80 0x55a21292e290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5fa76f20547a3ee server_cookie=1ca10198aa5c203f in_seq=0 out_seq=0 2024-09-10T22:59:44.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa1ff7fe640 10 client.27763 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa1ff7fe640 1 -- 172.21.15.96:0/1405798834 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a212889df0 con 0x55a21292de80 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa1ff7fe640 10 client.27763 handle_command_reply: tid=0 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 client.27763 shutdown 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 msgr2=0x55a21299ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a21299ffd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa210009230 tx=0x7fa2100044c0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 shutdown_connections 2024-09-10T22:59:44.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55a21292de80 0x55a21292e290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a21291a220 0x55a2129a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2129261b0 0x55a21299cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 --2- 172.21.15.96:0/1405798834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a212919900 0x55a21299ffd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 >> 172.21.15.96:0/1405798834 conn(0x55a212917730 msgr2=0x55a212924500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.113+0000 7fa1ff7fe640 0 client.27763 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.114+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 shutdown_connections 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.114+0000 7fa228648740 1 -- 172.21.15.96:0/1405798834 wait complete. 2024-09-10T22:59:44.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.114+0000 7fa228648740 20 client.27763 trim_cache size 0 max 16384 2024-09-10T22:59:44.117 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:44.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.114+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 msgr2=0x7fa2080b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.114+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 0x7fa2080b78a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa218005f40 tx=0x7fa2180096a0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 msgr2=0x7fa220199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9f0 tx=0x7fa21400edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 shutdown_connections 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa220108060 0x7fa22019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa2080b53e0 0x7fa2080b78a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2201076e0 0x7fa220199ab0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 --2- 172.21.15.96:0/2612490917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa220101cd0 0x7fa220199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 >> 172.21.15.96:0/2612490917 conn(0x7fa2200fd720 msgr2=0x7fa220105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 shutdown_connections 2024-09-10T22:59:44.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:44.115+0000 7fa226273640 1 -- 172.21.15.96:0/2612490917 wait complete. 2024-09-10T22:59:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:45 smithi096 ceph-mon[30869]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 639 op/s 2024-09-10T22:59:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:45 smithi121 ceph-mon[35119]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 639 op/s 2024-09-10T22:59:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:45 smithi117 ceph-mon[35081]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 639 op/s 2024-09-10T22:59:45.866 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T22:59:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 730.0 s, 1552.7 tps, lat 64.401 ms stddev 69.937 2024-09-10T22:59:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:46.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.792+0000 7ff797d00640 1 -- 172.21.15.96:0/1284436907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790101c90 msgr2=0x7ff790102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.792+0000 7ff797d00640 1 --2- 172.21.15.96:0/1284436907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790101c90 0x7ff790102070 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c02a410 comp rx=0 tx=0).stop 2024-09-10T22:59:46.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.793+0000 7ff797d00640 1 -- 172.21.15.96:0/1284436907 shutdown_connections 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.793+0000 7ff797d00640 1 --2- 172.21.15.96:0/1284436907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff790108020 0x7ff79010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.793+0000 7ff797d00640 1 --2- 172.21.15.96:0/1284436907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7901076a0 0x7ff790107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.793+0000 7ff797d00640 1 --2- 172.21.15.96:0/1284436907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790101c90 0x7ff790102070 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.793+0000 7ff797d00640 1 -- 172.21.15.96:0/1284436907 >> 172.21.15.96:0/1284436907 conn(0x7ff7900fd700 msgr2=0x7ff7900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.794+0000 7ff797d00640 1 -- 172.21.15.96:0/1284436907 shutdown_connections 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.794+0000 7ff797d00640 1 -- 172.21.15.96:0/1284436907 wait complete. 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.794+0000 7ff797d00640 1 Processor -- start 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.794+0000 7ff797d00640 1 -- start start 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff790101c90 0x7ff790108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff790111ce0 con 0x7ff790108020 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff790111b60 con 0x7ff7901076a0 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff797d00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff790111e60 con 0x7ff790101c90 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79629d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46636/0 (socket says 172.21.15.96:46636) 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79629d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42230/0 (socket says 172.21.15.96:42230) 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff795a9c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff790101c90 0x7ff790108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 -- 172.21.15.96:0/3096062156 learned_addr learned my addr 172.21.15.96:0/3096062156 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 -- 172.21.15.96:0/3096062156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff790101c90 msgr2=0x7ff790108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff790101c90 0x7ff790108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 -- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 msgr2=0x7ff790109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 -- 172.21.15.96:0/3096062156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7901a7580 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79629d640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.795+0000 7ff79529b640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e9f0 tx=0x7ff78000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.796+0000 7ff786ffd640 1 -- 172.21.15.96:0/3096062156 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000cd80 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.796+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7901a7870 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.796+0000 7ff786ffd640 1 -- 172.21.15.96:0/3096062156 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7800054d0 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.796+0000 7ff786ffd640 1 -- 172.21.15.96:0/3096062156 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780010690 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.796+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7901a7db0 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.799+0000 7ff79a0d5740 1 Processor -- start 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.799+0000 7ff786ffd640 1 -- 172.21.15.96:0/3096062156 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff780004e90 con 0x7ff7901076a0 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.799+0000 7ff79a0d5740 1 -- start start 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 0x55e91c2ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e91c214ea0 con 0x55e91c29edf0 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e91c2042d0 con 0x55e91c29e4d0 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e91c2080e0 con 0x55e91c2aad80 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46652/0 (socket says 172.21.15.96:46652) 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 -- 172.21.15.96:0/3684669337 learned_addr learned my addr 172.21.15.96:0/3684669337 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79529b640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 0x55e91c2ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 -- 172.21.15.96:0/3684669337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 msgr2=0x55e91c2ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 0x55e91c2ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 -- 172.21.15.96:0/3684669337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 msgr2=0x55e91c2aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff79629d640 1 -- 172.21.15.96:0/3684669337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91bf62650 con 0x55e91c29e4d0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.800+0000 7ff786ffd640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 0x7ff77c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.801+0000 7ff79529b640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 0x55e91c2ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.801+0000 7ff786ffd640 1 -- 172.21.15.96:0/3096062156 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff7800e5f40 con 0x7ff7901076a0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.801+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 0x7ff77c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.802+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 0x7ff77c0b77a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0096c0 tx=0x7ff78c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.803+0000 7ff79629d640 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff788011760 tx=0x7ff788011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcfe31a4b515510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.804+0000 7ff77bfff640 1 -- 172.21.15.96:0/3684669337 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801e070 con 0x55e91c29e4d0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.804+0000 7ff77bfff640 1 -- 172.21.15.96:0/3684669337 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78800fc60 con 0x55e91c29e4d0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.804+0000 7ff77bfff640 1 -- 172.21.15.96:0/3684669337 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78800e610 con 0x55e91c29e4d0 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3684669337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 msgr2=0x55e91c29e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff788011760 tx=0x7ff788011c20 comp rx=0 tx=0).stop 2024-09-10T22:59:46.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3684669337 shutdown_connections 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c2aad80 0x55e91c2ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3684669337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29e4d0 0x55e91c29e8b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3684669337 >> 172.21.15.96:0/3684669337 conn(0x55e91c29c300 msgr2=0x55e91c29d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3684669337 shutdown_connections 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3684669337 wait complete. 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 -- start start 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.805+0000 7ff79a0d5740 1 Processor -- start 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e91c065700 con 0x55e91c29edf0 2024-09-10T22:59:46.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e91c19f160 con 0x55e91c2aad80 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e91c2ada00 con 0x55e91c29e4d0 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79a0d5740 10 client.0 _ll_register_callbacks cb 0x55e91c1b89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79529b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79529b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2a7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42244/0 (socket says 172.21.15.96:42244) 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79529b640 1 -- 172.21.15.96:0/3218606836 learned_addr learned my addr 172.21.15.96:0/3218606836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff79629d640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 -- 172.21.15.96:0/3218606836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 msgr2=0x55e91c2a8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2a8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 -- 172.21.15.96:0/3218606836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 msgr2=0x55e91c2a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.806+0000 7ff795a9c640 1 -- 172.21.15.96:0/3218606836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91c1ccc90 con 0x55e91c29e4d0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff795a9c640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2aa740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005ab0 tx=0x7ff78c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46b3960a55e515c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c035040 con 0x55e91c29e4d0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff78c005500 con 0x55e91c29e4d0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78800fc60 con 0x55e91c29e4d0 2024-09-10T22:59:46.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.807+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff78c004780 con 0x55e91c29e4d0 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.813+0000 7ff79a0d5740 10 client.27772 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.814+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e91c2b93b0 con 0x55e91c29e4d0 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.814+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78800fc60 con 0x55e91c29e4d0 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.814+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff78c051990 con 0x55e91c29e4d0 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.814+0000 7ff79a0d5740 10 client.27772 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.814+0000 7ff79a0d5740 10 client.27772 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.815+0000 7ff79a0d5740 20 client.27772 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.815+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c2bac40 0x55e91c2bd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.815+0000 7ff79a0d5740 4 client.27772 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:46.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.815+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e91c216df0 con 0x55e91c2bac40 2024-09-10T22:59:46.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.820+0000 7ff79529b640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c2bac40 0x55e91c2bd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.830+0000 7ff79529b640 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c2bac40 0x55e91c2bd070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcbdd9233500bd8e server_cookie=965bdf2ef9c5f65c in_seq=0 out_seq=0 2024-09-10T22:59:46.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.830+0000 7ff7797fa640 10 client.27772 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:59:46.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.832+0000 7ff7797fa640 1 -- 172.21.15.96:0/3218606836 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e91c216df0 con 0x55e91c2bac40 2024-09-10T22:59:46.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.833+0000 7ff7797fa640 10 client.27772 handle_command_reply: tid=0 2024-09-10T22:59:46.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.833+0000 7ff79a0d5740 1 client.27772 shutdown 2024-09-10T22:59:46.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.835+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 msgr2=0x55e91c2aa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.835+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2aa740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff78c005ab0 tx=0x7ff78c031400 comp rx=0 tx=0).stop 2024-09-10T22:59:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 shutdown_connections 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c2bac40 0x55e91c2bd070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2a8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29edf0 0x55e91c2a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/3218606836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2aa740 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 >> 172.21.15.96:0/3218606836 conn(0x55e91c29c300 msgr2=0x55e91c2a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff7797fa640 0 client.27772 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:59:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.838+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 shutdown_connections 2024-09-10T22:59:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.839+0000 7ff79a0d5740 1 -- 172.21.15.96:0/3218606836 wait complete. 2024-09-10T22:59:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.839+0000 7ff79a0d5740 20 client.27772 trim_cache size 0 max 16384 2024-09-10T22:59:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.863+0000 7ff79a0d5740 1 Processor -- start 2024-09-10T22:59:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.863+0000 7ff79a0d5740 1 -- start start 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.863+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29e4d0 0x55e91c2b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29edf0 0x55e91c29bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901b0480 con 0x55e91c29e4d0 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901b1fc0 con 0x55e91c2aad80 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901b0b50 con 0x55e91c29edf0 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79629d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79629d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46658/0 (socket says 172.21.15.96:46658) 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79629d640 1 -- 172.21.15.96:0/2770412595 learned_addr learned my addr 172.21.15.96:0/2770412595 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff79529b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29edf0 0x55e91c29bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.864+0000 7ff795a9c640 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29e4d0 0x55e91c2b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 -- 172.21.15.96:0/2770412595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29edf0 msgr2=0x55e91c29bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29edf0 0x55e91c29bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 -- 172.21.15.96:0/2770412595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29e4d0 msgr2=0x55e91c2b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29e4d0 0x55e91c2b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 -- 172.21.15.96:0/2770412595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91c2a85d0 con 0x55e91c2aad80 2024-09-10T22:59:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.865+0000 7ff79629d640 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff78800f790 tx=0x7ff788019c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5547bd9b5cc4bf86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.868+0000 7ff77b7fe640 1 -- 172.21.15.96:0/2770412595 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801e040 con 0x55e91c2aad80 2024-09-10T22:59:46.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.868+0000 7ff77b7fe640 1 -- 172.21.15.96:0/2770412595 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78802a070 con 0x55e91c2aad80 2024-09-10T22:59:46.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.868+0000 7ff77b7fe640 1 -- 172.21.15.96:0/2770412595 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7880198d0 con 0x55e91c2aad80 2024-09-10T22:59:46.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2770412595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 msgr2=0x55e91c2ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff78800f790 tx=0x7ff788019c20 comp rx=0 tx=0).stop 2024-09-10T22:59:46.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2770412595 shutdown_connections 2024-09-10T22:59:46.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c2aad80 0x55e91c2ac170 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29edf0 0x55e91c29bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2770412595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c29e4d0 0x55e91c2b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.870+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2770412595 >> 172.21.15.96:0/2770412595 conn(0x55e91c29c300 msgr2=0x55e91c29cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.872+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2770412595 shutdown_connections 2024-09-10T22:59:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.872+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2770412595 wait complete. 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.872+0000 7ff79a0d5740 1 -- start start 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 Processor -- start 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29edf0 0x55e91c2abb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901b18f0 con 0x55e91c2aad80 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901b1220 con 0x55e91c29edf0 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901ae460 con 0x55e91c29e4d0 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.873+0000 7ff79a0d5740 10 client.0 _ll_register_callbacks cb 0x7ff78c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42254/0 (socket says 172.21.15.96:42254) 2024-09-10T22:59:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 -- 172.21.15.96:0/2510227674 learned_addr learned my addr 172.21.15.96:0/2510227674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff795a9c640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79529b640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29edf0 0x55e91c2abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 -- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 msgr2=0x55e91c2ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 -- 172.21.15.96:0/2510227674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29edf0 msgr2=0x55e91c2abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29edf0 0x55e91c2abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.874+0000 7ff79629d640 1 -- 172.21.15.96:0/2510227674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91c2b2200 con 0x55e91c2aad80 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.875+0000 7ff79629d640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff7880115a0 tx=0x7ff78800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e184a17a11c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.875+0000 7ff7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.875+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801e040 con 0x55e91c2aad80 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.875+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff788003d80 con 0x55e91c2aad80 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.876+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78802a070 con 0x55e91c2aad80 2024-09-10T22:59:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.876+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff788005bd0 con 0x55e91c2aad80 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 10 client.18114 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e91c2b3500 con 0x55e91c2aad80 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78802a070 con 0x55e91c2aad80 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff7880054d0 con 0x55e91c2aad80 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 10 client.18114 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:46.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 10 client.18114 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 20 client.18114 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c327b50 0x55e91c329f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 4 client.18114 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.877+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e91c2128a0 con 0x55e91c327b50 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.878+0000 7ff795a9c640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c327b50 0x55e91c329f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.879+0000 7ff7797fa640 10 client.18114 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.879+0000 7ff795a9c640 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c327b50 0x55e91c329f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec4338d931893e35 server_cookie=dc8c49920a366ceb in_seq=0 out_seq=0 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.880+0000 7ff7797fa640 1 -- 172.21.15.96:0/2510227674 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e91c2128a0 con 0x55e91c327b50 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.880+0000 7ff7797fa640 10 client.18114 handle_command_reply: tid=0 2024-09-10T22:59:46.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.880+0000 7ff79a0d5740 1 client.18114 shutdown 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.880+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 msgr2=0x55e91c2b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.880+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff7880115a0 tx=0x7ff78800fc90 comp rx=0 tx=0).stop 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 shutdown_connections 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55e91c327b50 0x55e91c329f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff7797fa640 0 client.18114 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e91c29edf0 0x55e91c2abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e91c2aad80 0x55e91c2b1cc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 --2- 172.21.15.96:0/2510227674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e91c29e4d0 0x55e91c2ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.881+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 >> 172.21.15.96:0/2510227674 conn(0x55e91c29c300 msgr2=0x55e91c2a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.882+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 shutdown_connections 2024-09-10T22:59:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.882+0000 7ff79a0d5740 1 -- 172.21.15.96:0/2510227674 wait complete. 2024-09-10T22:59:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.882+0000 7ff79a0d5740 20 client.18114 trim_cache size 0 max 16384 2024-09-10T22:59:46.885 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.884+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 msgr2=0x7ff77c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.884+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 0x7ff77c0b77a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0096c0 tx=0x7ff78c00b040 comp rx=0 tx=0).stop 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.884+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 msgr2=0x7ff790108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.884+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e9f0 tx=0x7ff78000edf0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 shutdown_connections 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7901076a0 0x7ff790108e50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff77c0b52e0 0x7ff77c0b77a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff790108020 0x7ff790109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 --2- 172.21.15.96:0/3096062156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff790101c90 0x7ff790108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:46.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.885+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 >> 172.21.15.96:0/3096062156 conn(0x7ff7900fd700 msgr2=0x7ff79006a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.887+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 shutdown_connections 2024-09-10T22:59:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:46.887+0000 7ff797d00640 1 -- 172.21.15.96:0/3096062156 wait complete. 2024-09-10T22:59:47.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:47 smithi096 ceph-mon[30869]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 643 op/s 2024-09-10T22:59:47.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T22:59:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:47 smithi121 ceph-mon[35119]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 643 op/s 2024-09-10T22:59:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:47 smithi117 ceph-mon[35081]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 643 op/s 2024-09-10T22:59:48.061 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:48.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 -- 172.21.15.96:0/2224611209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3520103ce0 msgr2=0x7f35201040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 --2- 172.21.15.96:0/2224611209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3520103ce0 0x7f35201040c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3510009ca0 tx=0x7f351002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 -- 172.21.15.96:0/2224611209 shutdown_connections 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 --2- 172.21.15.96:0/2224611209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520104600 0x7f352010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 --2- 172.21.15.96:0/2224611209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f352010d140 0x7f352010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 --2- 172.21.15.96:0/2224611209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3520103ce0 0x7f35201040c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 -- 172.21.15.96:0/2224611209 >> 172.21.15.96:0/2224611209 conn(0x7f35200fd720 msgr2=0x7f35200ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 -- 172.21.15.96:0/2224611209 shutdown_connections 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.669+0000 7f3524f72640 1 -- 172.21.15.96:0/2224611209 wait complete. 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.670+0000 7f3524f72640 1 Processor -- start 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.670+0000 7f3524f72640 1 -- start start 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3520104600 0x7f3520101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f352010d140 0x7f35201022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112db0 con 0x7f352010d140 2024-09-10T22:59:48.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112c30 con 0x7f3520103ce0 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3524f72640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112f30 con 0x7f3520104600 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46688/0 (socket says 172.21.15.96:46688) 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 -- 172.21.15.96:0/2294181342 learned_addr learned my addr 172.21.15.96:0/2294181342 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351ed9d640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f352010d140 0x7f35201022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f3517fff640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3520104600 0x7f3520101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 -- 172.21.15.96:0/2294181342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3520104600 msgr2=0x7f3520101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3520104600 0x7f3520101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 -- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f352010d140 msgr2=0x7f35201022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f352010d140 0x7f35201022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.671+0000 7f351e59c640 1 -- 172.21.15.96:0/2294181342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3520073bc0 con 0x7f3520103ce0 2024-09-10T22:59:48.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f351e59c640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3510033040 tx=0x7f35100316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f35177fe640 1 -- 172.21.15.96:0/2294181342 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510003b70 con 0x7f3520103ce0 2024-09-10T22:59:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f35177fe640 1 -- 172.21.15.96:0/2294181342 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3510003d10 con 0x7f3520103ce0 2024-09-10T22:59:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f35177fe640 1 -- 172.21.15.96:0/2294181342 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351002f9e0 con 0x7f3520103ce0 2024-09-10T22:59:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3520073e50 con 0x7f3520103ce0 2024-09-10T22:59:48.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.672+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3520074340 con 0x7f3520103ce0 2024-09-10T22:59:48.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.676+0000 7f35177fe640 1 -- 172.21.15.96:0/2294181342 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f351002fb80 con 0x7f3520103ce0 2024-09-10T22:59:48.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.676+0000 7f35177fe640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 0x7f350c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.676+0000 7f3527347740 1 Processor -- start 2024-09-10T22:59:48.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f35177fe640 1 -- 172.21.15.96:0/2294181342 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3510106cd0 con 0x7f3520103ce0 2024-09-10T22:59:48.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3517fff640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 0x7f350c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 -- start start 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c46df0 0x55df50c52840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 0x55df50c55170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df50bbcea0 con 0x55df50c464d0 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df50bac2d0 con 0x55df50c52d80 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3527347740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df50bb00e0 con 0x55df50c46df0 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3517fff640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 0x7f350c0b78f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3520103360 tx=0x7f3508005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42284/0 (socket says 172.21.15.96:42284) 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 -- 172.21.15.96:0/3779045504 learned_addr learned my addr 172.21.15.96:0/3779045504 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:48.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f3517fff640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c46df0 0x55df50c52840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351ed9d640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 0x55df50c55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 -- 172.21.15.96:0/3779045504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c46df0 msgr2=0x55df50c52840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c46df0 0x55df50c52840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 -- 172.21.15.96:0/3779045504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 msgr2=0x55df50c55170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 0x55df50c55170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351e59c640 1 -- 172.21.15.96:0/3779045504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5090a650 con 0x55df50c464d0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.677+0000 7f351ed9d640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 0x55df50c55170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.678+0000 7f351e59c640 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f35100cd4c0 tx=0x7f35100ccf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab69e95a3f093f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.678+0000 7f3514ff9640 1 -- 172.21.15.96:0/3779045504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351002fea0 con 0x55df50c464d0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.678+0000 7f3514ff9640 1 -- 172.21.15.96:0/3779045504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3510036040 con 0x55df50c464d0 2024-09-10T22:59:48.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.678+0000 7f3514ff9640 1 -- 172.21.15.96:0/3779045504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100cd9f0 con 0x55df50c464d0 2024-09-10T22:59:48.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 -- 172.21.15.96:0/3779045504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 msgr2=0x55df50c468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f35100cd4c0 tx=0x7f35100ccf40 comp rx=0 tx=0).stop 2024-09-10T22:59:48.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 -- 172.21.15.96:0/3779045504 shutdown_connections 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c52d80 0x55df50c55170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c46df0 0x55df50c52840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 --2- 172.21.15.96:0/3779045504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c468b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 -- 172.21.15.96:0/3779045504 >> 172.21.15.96:0/3779045504 conn(0x55df50c44300 msgr2=0x55df50c454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.679+0000 7f3527347740 1 -- 172.21.15.96:0/3779045504 shutdown_connections 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 -- 172.21.15.96:0/3779045504 wait complete. 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 -- start start 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 Processor -- start 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c4ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df50a0d700 con 0x55df50c464d0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df50b47160 con 0x55df50c46df0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3527347740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df50bbcea0 con 0x55df50c52d80 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42290/0 (socket says 172.21.15.96:42290) 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f351e59c640 1 -- 172.21.15.96:0/1229107681 learned_addr learned my addr 172.21.15.96:0/1229107681 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.680+0000 7f3517fff640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f351ed9d640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c4ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3527347740 10 client.0 _ll_register_callbacks cb 0x55df50b609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 -- 172.21.15.96:0/1229107681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 msgr2=0x55df50c4ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c4ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 -- 172.21.15.96:0/1229107681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 msgr2=0x55df50c525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 -- 172.21.15.96:0/1229107681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df50b74c90 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3510036040 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3517fff640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c4fa30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3508009fd0 tx=0x7f3508004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9904f531c57edd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3505ffb640 10 client.27737 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.681+0000 7f3505ffb640 1 -- 172.21.15.96:0/1229107681 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508004220 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3505ffb640 1 -- 172.21.15.96:0/1229107681 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35080043c0 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3505ffb640 1 -- 172.21.15.96:0/1229107681 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f350801f910 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 10 client.27737 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55df50c60d70 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3505ffb640 1 -- 172.21.15.96:0/1229107681 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f350801fab0 con 0x55df50c46df0 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 10 client.27737 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 10 client.27737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:59:48.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 20 client.27737 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50c62110 0x55df50c64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 4 client.27737 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.682+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df50bbedf0 con 0x55df50c62110 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.683+0000 7f351e59c640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50c62110 0x55df50c64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.683+0000 7f351e59c640 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50c62110 0x55df50c64560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5bdbae65f7f65dd server_cookie=d51780af1fa82aa7 in_seq=0 out_seq=0 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.683+0000 7f3505ffb640 10 client.27737 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3505ffb640 1 -- 172.21.15.96:0/1229107681 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df50bbedf0 con 0x55df50c62110 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3505ffb640 10 client.27737 handle_command_reply: tid=0 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 client.27737 shutdown 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 msgr2=0x55df50c4fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c4fa30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3508009fd0 tx=0x7f3508004730 comp rx=0 tx=0).stop 2024-09-10T22:59:48.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 shutdown_connections 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50c62110 0x55df50c64560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c4fa30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c4ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 --2- 172.21.15.96:0/1229107681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 >> 172.21.15.96:0/1229107681 conn(0x55df50c44300 msgr2=0x55df50c49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3505ffb640 0 client.27737 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 shutdown_connections 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 1 -- 172.21.15.96:0/1229107681 wait complete. 2024-09-10T22:59:48.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.684+0000 7f3527347740 20 client.27737 trim_cache size 0 max 16384 2024-09-10T22:59:48.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.707+0000 7f3527347740 1 Processor -- start 2024-09-10T22:59:48.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 -- start start 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c4ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35201bcfe0 con 0x55df50c464d0 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35201beb20 con 0x55df50c46df0 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f3527347740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35201bd6b0 con 0x55df50c52d80 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c4ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f351e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c4ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42294/0 (socket says 172.21.15.96:42294) 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f351e59c640 1 -- 172.21.15.96:0/2201509763 learned_addr learned my addr 172.21.15.96:0/2201509763 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:48.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.708+0000 7f351ed9d640 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 -- 172.21.15.96:0/2201509763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 msgr2=0x55df50c64120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c64120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 -- 172.21.15.96:0/2201509763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 msgr2=0x55df50c4ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c4ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 -- 172.21.15.96:0/2201509763 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df50c44d80 con 0x55df50c52d80 2024-09-10T22:59:48.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.709+0000 7f351ed9d640 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c51770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55df50c52d50 tx=0x7f3518013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474a819dbbde5fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.710+0000 7f3507fff640 1 -- 172.21.15.96:0/2201509763 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518009d30 con 0x55df50c52d80 2024-09-10T22:59:48.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.710+0000 7f3507fff640 1 -- 172.21.15.96:0/2201509763 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35180035f0 con 0x55df50c52d80 2024-09-10T22:59:48.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.710+0000 7f3507fff640 1 -- 172.21.15.96:0/2201509763 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518010690 con 0x55df50c52d80 2024-09-10T22:59:48.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.711+0000 7f3527347740 1 -- 172.21.15.96:0/2201509763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 msgr2=0x55df50c51770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.711+0000 7f3527347740 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c51770 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55df50c52d50 tx=0x7f3518013830 comp rx=0 tx=0).stop 2024-09-10T22:59:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.715+0000 7f3527347740 1 -- 172.21.15.96:0/2201509763 shutdown_connections 2024-09-10T22:59:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.715+0000 7f3527347740 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c46df0 0x55df50c64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.715+0000 7f3527347740 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c51770 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.715+0000 7f3527347740 1 --2- 172.21.15.96:0/2201509763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c464d0 0x55df50c4ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.715+0000 7f3527347740 1 -- 172.21.15.96:0/2201509763 >> 172.21.15.96:0/2201509763 conn(0x55df50c44300 msgr2=0x55df50c51060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.719+0000 7f3527347740 1 -- 172.21.15.96:0/2201509763 shutdown_connections 2024-09-10T22:59:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.720+0000 7f3527347740 1 -- 172.21.15.96:0/2201509763 wait complete. 2024-09-10T22:59:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.720+0000 7f3527347740 1 -- start start 2024-09-10T22:59:48.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.720+0000 7f3527347740 1 Processor -- start 2024-09-10T22:59:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 0x55df50c5aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c46df0 0x55df50c5b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c57860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35201be450 con 0x55df50c46df0 2024-09-10T22:59:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35201bdd80 con 0x55df50c464d0 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.721+0000 7f3527347740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35201bafc0 con 0x55df50c52d80 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f3517fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c46df0 0x55df50c5b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f351e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 0x55df50c5aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f351ed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c57860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f351ed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c57860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53794/0 (socket says 172.21.15.96:53794) 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f351ed9d640 1 -- 172.21.15.96:0/2595559838 learned_addr learned my addr 172.21.15.96:0/2595559838 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.722+0000 7f3527347740 10 client.0 _ll_register_callbacks cb 0x7f35080128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.727+0000 7f351e59c640 1 -- 172.21.15.96:0/2595559838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 msgr2=0x55df50c57860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.727+0000 7f351e59c640 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c57860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.727+0000 7f351e59c640 1 -- 172.21.15.96:0/2595559838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c46df0 msgr2=0x55df50c5b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.727+0000 7f351e59c640 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c46df0 0x55df50c5b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.727+0000 7f351e59c640 1 -- 172.21.15.96:0/2595559838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df50c58070 con 0x55df50c464d0 2024-09-10T22:59:48.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.730+0000 7f351e59c640 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 0x55df50c5aae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3510005fa0 tx=0x7f35100d64d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb8861297f5f077 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:48.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.730+0000 7f3505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:48.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.731+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351002fea0 con 0x55df50c464d0 2024-09-10T22:59:48.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.731+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f35100c97a0 con 0x55df50c464d0 2024-09-10T22:59:48.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.731+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35180035f0 con 0x55df50c464d0 2024-09-10T22:59:48.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.735+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35100cdc30 con 0x55df50c464d0 2024-09-10T22:59:48.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.735+0000 7f3527347740 10 client.27740 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:48.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.735+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df50ccfb10 con 0x55df50c464d0 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35100d6ac0 con 0x55df50c464d0 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f35100d6630 con 0x55df50c464d0 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3527347740 10 client.27740 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3527347740 10 client.27740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3527347740 20 client.27740 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:48.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50cd0050 0x55df50cd2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:48.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.738+0000 7f3527347740 4 client.27740 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:48.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.739+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f35180035f0 con 0x55df50cd0050 2024-09-10T22:59:48.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.740+0000 7f3517fff640 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50cd0050 0x55df50cd2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:48.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.742+0000 7f3517fff640 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50cd0050 0x55df50cd2440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca973f367514898d server_cookie=8d147072bd7574da in_seq=0 out_seq=0 2024-09-10T22:59:48.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.745+0000 7f3505ffb640 10 client.27740 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T22:59:48.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.745+0000 7f3505ffb640 1 -- 172.21.15.96:0/2595559838 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f35180035f0 con 0x55df50cd0050 2024-09-10T22:59:48.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.745+0000 7f3505ffb640 10 client.27740 handle_command_reply: tid=0 2024-09-10T22:59:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.745+0000 7f3527347740 1 client.27740 shutdown 2024-09-10T22:59:48.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.748+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 msgr2=0x55df50c5aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.748+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 0x55df50c5aae0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3510005fa0 tx=0x7f35100d64d0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 shutdown_connections 2024-09-10T22:59:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55df50cd0050 0x55df50cd2440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df50c52d80 0x55df50c57860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df50c46df0 0x55df50c5b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 --2- 172.21.15.96:0/2595559838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df50c464d0 0x55df50c5aae0 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3510005fa0 tx=0x7f35100d64d0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.749+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 >> 172.21.15.96:0/2595559838 conn(0x55df50c44300 msgr2=0x55df50c51880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.750+0000 7f3505ffb640 0 client.27740 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T22:59:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.750+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 shutdown_connections 2024-09-10T22:59:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.753+0000 7f3527347740 1 -- 172.21.15.96:0/2595559838 wait complete. 2024-09-10T22:59:48.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.753+0000 7f3527347740 20 client.27740 trim_cache size 0 max 16384 2024-09-10T22:59:48.755 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 msgr2=0x7f350c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 0x7f350c0b78f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3520103360 tx=0x7f3508005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 msgr2=0x7f3520101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3510033040 tx=0x7f35100316f0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 shutdown_connections 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f350c0b5430 0x7f350c0b78f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f352010d140 0x7f35201022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3520104600 0x7f3520101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 --2- 172.21.15.96:0/2294181342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3520103ce0 0x7f3520101870 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3510033040 tx=0x7f35100316f0 comp rx=0 tx=0).stop 2024-09-10T22:59:48.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.763+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 >> 172.21.15.96:0/2294181342 conn(0x7f35200fd720 msgr2=0x7f35200fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:48.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.764+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 shutdown_connections 2024-09-10T22:59:48.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:48.764+0000 7f3524f72640 1 -- 172.21.15.96:0/2294181342 wait complete. 2024-09-10T22:59:49.712 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:49 smithi096 ceph-mon[30869]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 555 op/s 2024-09-10T22:59:49.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T22:59:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:49 smithi121 ceph-mon[35119]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 555 op/s 2024-09-10T22:59:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:49 smithi117 ceph-mon[35081]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 555 op/s 2024-09-10T22:59:50.127 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 -- 172.21.15.96:0/1962786100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc101cd0 msgr2=0x7ff2bc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/1962786100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc101cd0 0x7ff2bc1020b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009ca0 tx=0x7ff2b402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 -- 172.21.15.96:0/1962786100 shutdown_connections 2024-09-10T22:59:50.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/1962786100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc108060 0x7ff2bc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/1962786100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/1962786100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc101cd0 0x7ff2bc1020b0 secure :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009ca0 tx=0x7ff2b402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 -- 172.21.15.96:0/1962786100 >> 172.21.15.96:0/1962786100 conn(0x7ff2bc0fd700 msgr2=0x7ff2bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 -- 172.21.15.96:0/1962786100 shutdown_connections 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.727+0000 7ff2c1c15640 1 -- 172.21.15.96:0/1962786100 wait complete. 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.729+0000 7ff2c1c15640 1 Processor -- start 2024-09-10T22:59:50.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.729+0000 7ff2c1c15640 1 -- start start 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.729+0000 7ff2c1c15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2c1c15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 0x7ff2bc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53822/0 (socket says 172.21.15.96:53822) 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/4250347029 learned_addr learned my addr 172.21.15.96:0/4250347029 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 0x7ff2bc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2baffd640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 0x7ff2bc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111d20 con 0x7ff2bc108060 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111ba0 con 0x7ff2bc19bb60 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111ea0 con 0x7ff2bc1076e0 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.730+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 0x7ff2bc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 -- 172.21.15.96:0/4250347029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 msgr2=0x7ff2bc199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 -- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 msgr2=0x7ff2bc19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 0x7ff2bc19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19c6f0 con 0x7ff2bc108060 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 0x7ff2bc19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2baffd640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 0x7ff2bc199ab0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9f0 tx=0x7ff2b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2b8ff9640 1 -- 172.21.15.96:0/4250347029 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000ccc0 con 0x7ff2bc108060 2024-09-10T22:59:50.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc1a5f20 con 0x7ff2bc108060 2024-09-10T22:59:50.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2b8ff9640 1 -- 172.21.15.96:0/4250347029 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b0005020 con 0x7ff2bc108060 2024-09-10T22:59:50.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc1a62c0 con 0x7ff2bc108060 2024-09-10T22:59:50.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.731+0000 7ff2b8ff9640 1 -- 172.21.15.96:0/4250347029 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004a00 con 0x7ff2bc108060 2024-09-10T22:59:50.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 Processor -- start 2024-09-10T22:59:50.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 -- start start 2024-09-10T22:59:50.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc19954320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc1994d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc198c6ea0 con 0x55dc1994e040 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc198b62d0 con 0x55dc19953f40 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dc198ba0e0 con 0x55dc19954860 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42338/0 (socket says 172.21.15.96:42338) 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2baffd640 1 -- 172.21.15.96:0/2491256332 learned_addr learned my addr 172.21.15.96:0/2491256332 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.735+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc19954320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc1994d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2baffd640 1 -- 172.21.15.96:0/2491256332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 msgr2=0x55dc1994d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2baffd640 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc1994d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2baffd640 1 -- 172.21.15.96:0/2491256332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 msgr2=0x55dc19954320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2baffd640 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc19954320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.736+0000 7ff2baffd640 1 -- 172.21.15.96:0/2491256332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc19614650 con 0x55dc1994e040 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.737+0000 7ff2b8ff9640 1 -- 172.21.15.96:0/4250347029 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff2b0011580 con 0x7ff2bc108060 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.737+0000 7ff2baffd640 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00054a0 tx=0x7ff2b0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec57e2134a1ad70e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.738+0000 7ff2a1ffb640 1 -- 172.21.15.96:0/2491256332 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b010c070 con 0x55dc1994e040 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.738+0000 7ff2a1ffb640 1 -- 172.21.15.96:0/2491256332 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b0049040 con 0x55dc1994e040 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.738+0000 7ff2a1ffb640 1 -- 172.21.15.96:0/2491256332 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b01115d0 con 0x55dc1994e040 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.738+0000 7ff2b8ff9640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 0x7ff2a40b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.738+0000 7ff2b8ff9640 1 -- 172.21.15.96:0/4250347029 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff2b00047a0 con 0x7ff2bc108060 2024-09-10T22:59:50.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 0x7ff2a40b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2491256332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 msgr2=0x55dc19958f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00054a0 tx=0x7ff2b0005700 comp rx=0 tx=0).stop 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2491256332 shutdown_connections 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19958f40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc1994d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2491256332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc19954320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2491256332 >> 172.21.15.96:0/2491256332 conn(0x55dc19950090 msgr2=0x55dc19950470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2491256332 shutdown_connections 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 0x7ff2a40b79a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012560 tx=0x7ff2ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.739+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2491256332 wait complete. 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 -- start start 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 Processor -- start 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc1995cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc19961ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc19717700 con 0x55dc1994e040 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc19851160 con 0x55dc19953f40 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dc19959630 con 0x55dc19954860 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.740+0000 7ff2c3fea740 10 client.0 _ll_register_callbacks cb 0x55dc1986a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc1995cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42348/0 (socket says 172.21.15.96:42348) 2024-09-10T22:59:50.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/165778942 learned_addr learned my addr 172.21.15.96:0/165778942 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc19961ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 -- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 msgr2=0x55dc19961ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc19961ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 -- 172.21.15.96:0/165778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 msgr2=0x55dc1995c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 -- 172.21.15.96:0/165778942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1987ec90 con 0x55dc19953f40 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc19961ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2baffd640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc1995cdb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00cbce0 tx=0x7ff2b010a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98461aa3e40f436 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff29affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001c070 con 0x55dc19953f40 2024-09-10T22:59:50.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff2b00cf970 con 0x55dc19953f40 2024-09-10T22:59:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b0049040 con 0x55dc19953f40 2024-09-10T22:59:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2b00cb5b0 con 0x55dc19953f40 2024-09-10T22:59:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2c3fea740 10 client.27752 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.741+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc1996af60 con 0x55dc19953f40 2024-09-10T22:59:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.742+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b0049040 con 0x55dc19953f40 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff2b00d8780 con 0x55dc19953f40 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 10 client.27752 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 10 client.27752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 20 client.27752 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc1996c2b0 0x55dc1996e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 4 client.27752 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc198c8df0 con 0x55dc1996c2b0 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc1996c2b0 0x55dc1996e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff29affd640 10 client.27752 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.743+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc1996c2b0 0x55dc1996e6a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9d94710882f5bce server_cookie=f2b941caf63b6f3 in_seq=0 out_seq=0 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff29affd640 1 -- 172.21.15.96:0/165778942 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc198c8df0 con 0x55dc1996c2b0 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff29affd640 10 client.27752 handle_command_reply: tid=0 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 client.27752 shutdown 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 msgr2=0x55dc1995cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc1995cdb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00cbce0 tx=0x7ff2b010a000 comp rx=0 tx=0).stop 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 shutdown_connections 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc1996c2b0 0x55dc1996e6a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc1995cdb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19954860 0x55dc19961ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/165778942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc1995c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.744+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 >> 172.21.15.96:0/165778942 conn(0x55dc19950090 msgr2=0x55dc1994f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.745+0000 7ff29affd640 0 client.27752 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.745+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 shutdown_connections 2024-09-10T22:59:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.746+0000 7ff2c3fea740 1 -- 172.21.15.96:0/165778942 wait complete. 2024-09-10T22:59:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.746+0000 7ff2c3fea740 20 client.27752 trim_cache size 0 max 16384 2024-09-10T22:59:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 Processor -- start 2024-09-10T22:59:50.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 -- start start 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 0x55dc19968e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 0x55dc1994d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ac770 con 0x55dc1994e040 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ae2b0 con 0x55dc19954860 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ace40 con 0x55dc19953f40 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42358/0 (socket says 172.21.15.96:42358) 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/2373044845 learned_addr learned my addr 172.21.15.96:0/2373044845 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2baffd640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 0x55dc19968e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.763+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 0x55dc1994d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/2373044845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 msgr2=0x55dc19968e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 0x55dc19968e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/2373044845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 msgr2=0x55dc1994d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 0x55dc1994d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/2373044845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1994e920 con 0x55dc1994e040 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bbfff640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 0x55dc1994d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2baffd640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 0x55dc19968e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:50.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4031a90 tx=0x7ff2b4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fb2e86616bfa4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2a17fa640 1 -- 172.21.15.96:0/2373044845 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4003b70 con 0x55dc1994e040 2024-09-10T22:59:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2a17fa640 1 -- 172.21.15.96:0/2373044845 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b4036040 con 0x55dc1994e040 2024-09-10T22:59:50.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.764+0000 7ff2a17fa640 1 -- 172.21.15.96:0/2373044845 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4030820 con 0x55dc1994e040 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2373044845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 msgr2=0x55dc19956a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4031a90 tx=0x7ff2b4005d80 comp rx=0 tx=0).stop 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2373044845 shutdown_connections 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19954860 0x55dc1994d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc19953f40 0x55dc19968e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/2373044845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc1994e040 0x55dc19956a90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2373044845 >> 172.21.15.96:0/2373044845 conn(0x55dc19950090 msgr2=0x55dc19954c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2373044845 shutdown_connections 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- 172.21.15.96:0/2373044845 wait complete. 2024-09-10T22:59:50.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.765+0000 7ff2c3fea740 1 -- start start 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 Processor -- start 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 0x55dc19964c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc19954860 0x55dc199656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1adbe0 con 0x55dc19954860 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ad510 con 0x55dc19953f40 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1aa790 con 0x55dc1994e040 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2c3fea740 10 client.0 _ll_register_callbacks cb 0x7ff2b000c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc19954860 0x55dc199656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 0x55dc19964c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46756/0 (socket says 172.21.15.96:46756) 2024-09-10T22:59:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.766+0000 7ff2baffd640 1 -- 172.21.15.96:0/334697686 learned_addr learned my addr 172.21.15.96:0/334697686 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/334697686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 msgr2=0x55dc199651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/334697686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc19954860 msgr2=0x55dc199656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc19954860 0x55dc199656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 -- 172.21.15.96:0/334697686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc199d7fc0 con 0x55dc1994e040 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2baffd640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2bb7fe640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 0x55dc19964c70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4031a90 tx=0x7ff2b40099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940f9667467ad3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff29affd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4057020 con 0x55dc1994e040 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b4036040 con 0x55dc1994e040 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff2b4004d40 con 0x55dc1994e040 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2b40039f0 con 0x55dc1994e040 2024-09-10T22:59:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2c3fea740 10 client.27814 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.767+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc199d92a0 con 0x55dc1994e040 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b4036040 con 0x55dc1994e040 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff2b404e030 con 0x55dc1994e040 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 10 client.27814 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 10 client.27814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 20 client.27814 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc199d9960 0x55dc199dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 4 client.27814 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc198c48a0 con 0x55dc199d9960 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2baffd640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc199d9960 0x55dc199dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff29affd640 10 client.27814 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.768+0000 7ff2baffd640 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc199d9960 0x55dc199dbd80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58568ac9520a9bf5 server_cookie=2fa57c737485c80f in_seq=0 out_seq=0 2024-09-10T22:59:50.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff29affd640 1 -- 172.21.15.96:0/334697686 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc198c48a0 con 0x55dc199d9960 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff29affd640 10 client.27814 handle_command_reply: tid=0 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 client.27814 shutdown 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 msgr2=0x55dc19964c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 0x55dc19964c70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4031a90 tx=0x7ff2b40099c0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 shutdown_connections 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55dc199d9960 0x55dc199dbd80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc19953f40 0x55dc199651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff29affd640 0 client.27814 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc19954860 0x55dc199656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 --2- 172.21.15.96:0/334697686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc1994e040 0x55dc19964c70 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4031a90 tx=0x7ff2b40099c0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 >> 172.21.15.96:0/334697686 conn(0x55dc19950090 msgr2=0x55dc199598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 shutdown_connections 2024-09-10T22:59:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.769+0000 7ff2c3fea740 1 -- 172.21.15.96:0/334697686 wait complete. 2024-09-10T22:59:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.770+0000 7ff2c3fea740 20 client.27814 trim_cache size 0 max 16384 2024-09-10T22:59:50.778 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 msgr2=0x7ff2a40b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 0x7ff2a40b79a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012560 tx=0x7ff2ac005f90 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 msgr2=0x7ff2bc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 0x7ff2bc199ab0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9f0 tx=0x7ff2b000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 shutdown_connections 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2bc19bb60 0x7ff2bc19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff2a40b54e0 0x7ff2a40b79a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2bc108060 0x7ff2bc199ab0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 --2- 172.21.15.96:0/4250347029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2bc1076e0 0x7ff2bc199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 >> 172.21.15.96:0/4250347029 conn(0x7ff2bc0fd700 msgr2=0x7ff2bc06bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:50.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 shutdown_connections 2024-09-10T22:59:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:50.773+0000 7ff2c1c15640 1 -- 172.21.15.96:0/4250347029 wait complete. 2024-09-10T22:59:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 735.0 s, 1683.8 tps, lat 59.524 ms stddev 69.289 2024-09-10T22:59:51.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:51 smithi096 ceph-mon[30869]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-09-10T22:59:51.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T22:59:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:51 smithi121 ceph-mon[35119]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-09-10T22:59:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:51 smithi117 ceph-mon[35081]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-09-10T22:59:52.026 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:52.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.596+0000 7f440ffff640 1 -- 172.21.15.96:0/252765094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080318a0 con 0x7f4418106ca0 2024-09-10T22:59:52.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 -- 172.21.15.96:0/252765094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418106ca0 msgr2=0x7f4418107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 --2- 172.21.15.96:0/252765094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418106ca0 0x7f4418107100 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4408009900 tx=0x7f440802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 -- 172.21.15.96:0/252765094 shutdown_connections 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 --2- 172.21.15.96:0/252765094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418107640 0x7f441810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 --2- 172.21.15.96:0/252765094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4418106ca0 0x7f4418107100 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 --2- 172.21.15.96:0/252765094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44181012a0 0x7f4418101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 -- 172.21.15.96:0/252765094 >> 172.21.15.96:0/252765094 conn(0x7f44180fd2f0 msgr2=0x7f44180ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 -- 172.21.15.96:0/252765094 shutdown_connections 2024-09-10T22:59:52.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.597+0000 7f441e1a8640 1 -- 172.21.15.96:0/252765094 wait complete. 2024-09-10T22:59:52.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.598+0000 7f441e1a8640 1 Processor -- start 2024-09-10T22:59:52.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.598+0000 7f441e1a8640 1 -- start start 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.598+0000 7f441e1a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 0x7f441810f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.598+0000 7f441e1a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418106ca0 0x7f441810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.598+0000 7f441e1a8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 0x7f4418110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441e1a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112c10 con 0x7f44181012a0 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 0x7f441810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441e1a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112a90 con 0x7f4418106ca0 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 0x7f441810f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42396/0 (socket says 172.21.15.96:42396) 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d1a6640 1 -- 172.21.15.96:0/2152402689 learned_addr learned my addr 172.21.15.96:0/2152402689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 0x7f4418110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112d90 con 0x7f4418107640 2024-09-10T22:59:52.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441c9a5640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418106ca0 0x7f441810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 -- 172.21.15.96:0/2152402689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418106ca0 msgr2=0x7f441810fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418106ca0 0x7f441810fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 -- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 msgr2=0x7f441810f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 0x7f441810f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.599+0000 7f441d9a7640 1 -- 172.21.15.96:0/2152402689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181a7520 con 0x7f4418107640 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 0x7f4418110010 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f441000da10 tx=0x7f441000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f440e7fc640 1 -- 172.21.15.96:0/2152402689 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000bd70 con 0x7f4418107640 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f440e7fc640 1 -- 172.21.15.96:0/2152402689 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4410003e20 con 0x7f4418107640 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f440e7fc640 1 -- 172.21.15.96:0/2152402689 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100106c0 con 0x7f4418107640 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44181a7810 con 0x7f4418107640 2024-09-10T22:59:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.600+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181a7c70 con 0x7f4418107640 2024-09-10T22:59:52.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.603+0000 7f44217df740 1 Processor -- start 2024-09-10T22:59:52.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f440e7fc640 1 -- 172.21.15.96:0/2152402689 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4410024060 con 0x7f4418107640 2024-09-10T22:59:52.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 -- start start 2024-09-10T22:59:52.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acf3a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8acea1ea0 con 0x55e8acf37f50 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e8ace912d0 con 0x55e8acf2bfc0 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.604+0000 7f44217df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e8ace950e0 con 0x55e8acf2b6a0 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46776/0 (socket says 172.21.15.96:46776) 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 -- 172.21.15.96:0/2030231809 learned_addr learned my addr 172.21.15.96:0/2030231809 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 -- 172.21.15.96:0/2030231809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 msgr2=0x55e8acf2ba80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf2ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 -- 172.21.15.96:0/2030231809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 msgr2=0x55e8acf3a340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acf3a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 -- 172.21.15.96:0/2030231809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8acbef650 con 0x55e8acf2bfc0 2024-09-10T22:59:52.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.605+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4414009dc0 tx=0x7f4414011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd9482ce4c46df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.606+0000 7f440e7fc640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 0x7f44040b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.606+0000 7f44037fe640 1 -- 172.21.15.96:0/2030231809 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414005220 con 0x55e8acf2bfc0 2024-09-10T22:59:52.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.606+0000 7f44037fe640 1 -- 172.21.15.96:0/2030231809 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4414004c50 con 0x55e8acf2bfc0 2024-09-10T22:59:52.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.606+0000 7f440e7fc640 1 -- 172.21.15.96:0/2152402689 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44100e5b50 con 0x7f4418107640 2024-09-10T22:59:52.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44037fe640 1 -- 172.21.15.96:0/2030231809 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44140197f0 con 0x55e8acf2bfc0 2024-09-10T22:59:52.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f441c9a5640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 0x7f44040b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 -- 172.21.15.96:0/2030231809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 msgr2=0x55e8acf37a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4414009dc0 tx=0x7f4414011810 comp rx=0 tx=0).stop 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 -- 172.21.15.96:0/2030231809 shutdown_connections 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf37a10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acf3a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 --2- 172.21.15.96:0/2030231809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf2ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.607+0000 7f44217df740 1 -- 172.21.15.96:0/2030231809 >> 172.21.15.96:0/2030231809 conn(0x55e8acf294d0 msgr2=0x55e8acf2a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.608+0000 7f44217df740 1 -- 172.21.15.96:0/2030231809 shutdown_connections 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.608+0000 7f441c9a5640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 0x7f44040b7950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4408031440 tx=0x7f4408005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.608+0000 7f44217df740 1 -- 172.21.15.96:0/2030231809 wait complete. 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.608+0000 7f44217df740 1 -- start start 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.608+0000 7f44217df740 1 Processor -- start 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8accf2700 con 0x55e8acf2b6a0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e8ace2c160 con 0x55e8acf2bfc0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f44217df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e8acf3ac20 con 0x55e8acf37f50 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42406/0 (socket says 172.21.15.96:42406) 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f441d1a6640 1 -- 172.21.15.96:0/1270399033 learned_addr learned my addr 172.21.15.96:0/1270399033 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f441c9a5640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.609+0000 7f441d9a7640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f44217df740 10 client.0 _ll_register_callbacks cb 0x55e8ace459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 -- 172.21.15.96:0/1270399033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 msgr2=0x55e8acf35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf35240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 -- 172.21.15.96:0/1270399033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 msgr2=0x55e8acf37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 -- 172.21.15.96:0/1270399033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ace59c90 con 0x55e8acf2bfc0 2024-09-10T22:59:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441d1a6640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4414004c50 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f441c9a5640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf34c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f4408009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8837b16ddfe6eb11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f4400ff9640 10 client.27770 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f4400ff9640 1 -- 172.21.15.96:0/1270399033 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440804c020 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f4400ff9640 1 -- 172.21.15.96:0/1270399033 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f44080043d0 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f4400ff9640 1 -- 172.21.15.96:0/1270399033 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4408035040 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f44217df740 10 client.27770 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.610+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e8acf46470 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f4400ff9640 1 -- 172.21.15.96:0/1270399033 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4408005030 con 0x55e8acf2bfc0 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 10 client.27770 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 10 client.27770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 20 client.27770 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acf49ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 4 client.27770 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:52.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8acea3df0 con 0x55e8acf47830 2024-09-10T22:59:52.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.611+0000 7f441d1a6640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acf49ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.612+0000 7f4400ff9640 10 client.27770 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:52.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.612+0000 7f441d1a6640 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acf49ca0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf2e27dde1f6486 server_cookie=a75ed293672d1202 in_seq=0 out_seq=0 2024-09-10T22:59:52.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.612+0000 7f4400ff9640 1 -- 172.21.15.96:0/1270399033 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8acea3df0 con 0x55e8acf47830 2024-09-10T22:59:52.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.612+0000 7f4400ff9640 10 client.27770 handle_command_reply: tid=0 2024-09-10T22:59:52.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.612+0000 7f44217df740 1 client.27770 shutdown 2024-09-10T22:59:52.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 msgr2=0x55e8acf34c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf34c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f4408009260 comp rx=0 tx=0).stop 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 shutdown_connections 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acf49ca0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f4400ff9640 0 client.27770 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.613+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf34c60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.614+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.614+0000 7f44217df740 1 --2- 172.21.15.96:0/1270399033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.614+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 >> 172.21.15.96:0/1270399033 conn(0x55e8acf294d0 msgr2=0x55e8acf2f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.614+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 shutdown_connections 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.614+0000 7f44217df740 1 -- 172.21.15.96:0/1270399033 wait complete. 2024-09-10T22:59:52.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.615+0000 7f44217df740 20 client.27770 trim_cache size 0 max 16384 2024-09-10T22:59:52.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 Processor -- start 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 -- start start 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf28e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf31e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b0160 con 0x55e8acf2b6a0 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b1ca0 con 0x55e8acf2bfc0 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f44217df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b0830 con 0x55e8acf37f50 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f441d1a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42414/0 (socket says 172.21.15.96:42414) 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.631+0000 7f441d1a6640 1 -- 172.21.15.96:0/4060071944 learned_addr learned my addr 172.21.15.96:0/4060071944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d9a7640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf31e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441c9a5640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf28e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 -- 172.21.15.96:0/4060071944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 msgr2=0x55e8acf31e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf31e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 -- 172.21.15.96:0/4060071944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 msgr2=0x55e8acf28e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf28e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 -- 172.21.15.96:0/4060071944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8acf29f50 con 0x55e8acf2b6a0 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441c9a5640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf28e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f441d1a6640 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4414009ce0 tx=0x7f4414009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b731adc1606601d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f4402ffd640 1 -- 172.21.15.96:0/4060071944 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414004240 con 0x55e8acf2b6a0 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f4402ffd640 1 -- 172.21.15.96:0/4060071944 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44140052b0 con 0x55e8acf2b6a0 2024-09-10T22:59:52.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.632+0000 7f4402ffd640 1 -- 172.21.15.96:0/4060071944 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414003a90 con 0x55e8acf2b6a0 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 -- 172.21.15.96:0/4060071944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 msgr2=0x55e8acf43ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4414009ce0 tx=0x7f4414009310 comp rx=0 tx=0).stop 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 -- 172.21.15.96:0/4060071944 shutdown_connections 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf28e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf37f50 0x55e8acf31e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 --2- 172.21.15.96:0/4060071944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf2b6a0 0x55e8acf43ea0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 -- 172.21.15.96:0/4060071944 >> 172.21.15.96:0/4060071944 conn(0x55e8acf294d0 msgr2=0x55e8acf2a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 -- 172.21.15.96:0/4060071944 shutdown_connections 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.633+0000 7f44217df740 1 -- 172.21.15.96:0/4060071944 wait complete. 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 -- start start 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 Processor -- start 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b15d0 con 0x55e8acf37f50 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44181b0f00 con 0x55e8acf2bfc0 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f44217df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44181ae180 con 0x55e8acf2b6a0 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f441d9a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f441d9a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42418/0 (socket says 172.21.15.96:42418) 2024-09-10T22:59:52.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f441d9a7640 1 -- 172.21.15.96:0/2917815269 learned_addr learned my addr 172.21.15.96:0/2917815269 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.634+0000 7f441c9a5640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f44217df740 10 client.0 _ll_register_callbacks cb 0x7f4408040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 -- 172.21.15.96:0/2917815269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 msgr2=0x55e8acf3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 -- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 msgr2=0x55e8acf3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 -- 172.21.15.96:0/2917815269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8acf3cf30 con 0x55e8acf37f50 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44140052b0 con 0x55e8acf37f50 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f441d9a7640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f441000b630 tx=0x7f44100b52a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8a413b736114b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f4400ff9640 10 client.18162 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f4400ff9640 1 -- 172.21.15.96:0/2917815269 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410097020 con 0x55e8acf37f50 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f4400ff9640 1 -- 172.21.15.96:0/2917815269 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4410004270 con 0x55e8acf37f50 2024-09-10T22:59:52.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f4400ff9640 1 -- 172.21.15.96:0/2917815269 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44100b5040 con 0x55e8acf37f50 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f44217df740 10 client.18162 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.635+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e8acfb4c50 con 0x55e8acf37f50 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f4400ff9640 1 -- 172.21.15.96:0/2917815269 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f44100b5410 con 0x55e8acf37f50 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 10 client.18162 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 20 client.18162 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acfb7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 4 client.18162 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8ace9f8a0 con 0x55e8acf47830 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acfb7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f441d1a6640 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acfb7270 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9710ae7a341d8569 server_cookie=ada215ed22155ccd in_seq=0 out_seq=0 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.636+0000 7f4400ff9640 10 client.18162 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.637+0000 7f4400ff9640 1 -- 172.21.15.96:0/2917815269 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8ace9f8a0 con 0x55e8acf47830 2024-09-10T22:59:52.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.637+0000 7f4400ff9640 10 client.18162 handle_command_reply: tid=0 2024-09-10T22:59:52.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.638+0000 7f44217df740 1 client.18162 shutdown 2024-09-10T22:59:52.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.638+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 msgr2=0x55e8acfb37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.638+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f441000b630 tx=0x7f44100b52a0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 shutdown_connections 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e8acf47830 0x55e8acfb7270 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8acf2bfc0 0x55e8acf3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f4400ff9640 0 client.18162 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8acf37f50 0x55e8acfb37c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 --2- 172.21.15.96:0/2917815269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8acf2b6a0 0x55e8acf3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 >> 172.21.15.96:0/2917815269 conn(0x55e8acf294d0 msgr2=0x55e8acf2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 shutdown_connections 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 1 -- 172.21.15.96:0/2917815269 wait complete. 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.639+0000 7f44217df740 20 client.18162 trim_cache size 0 max 16384 2024-09-10T22:59:52.644 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 msgr2=0x7f44040b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 0x7f44040b7950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4408031440 tx=0x7f4408005b50 comp rx=0 tx=0).stop 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 msgr2=0x7f4418110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 0x7f4418110010 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f441000da10 tx=0x7f441000ded0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 shutdown_connections 2024-09-10T22:59:52.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4418106ca0 0x7f441810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f44040b5490 0x7f44040b7950 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4418107640 0x7f4418110010 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 --2- 172.21.15.96:0/2152402689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44181012a0 0x7f441810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 >> 172.21.15.96:0/2152402689 conn(0x7f44180fd2f0 msgr2=0x7f4418104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 shutdown_connections 2024-09-10T22:59:52.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:52.640+0000 7f441e1a8640 1 -- 172.21.15.96:0/2152402689 wait complete. 2024-09-10T22:59:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:53 smithi096 ceph-mon[30869]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-09-10T22:59:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:53 smithi121 ceph-mon[35119]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-09-10T22:59:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:53 smithi117 ceph-mon[35081]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 MiB/s wr, 611 op/s 2024-09-10T22:59:54.905 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T22:59:54.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T22:59:55.251 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 -- 172.21.15.96:0/297372705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd101066b0 msgr2=0x7fdd1010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/297372705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd101066b0 0x7fdd1010d030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009ca0 tx=0x7fdd0802a8e0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 -- 172.21.15.96:0/297372705 shutdown_connections 2024-09-10T22:59:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/297372705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd101066b0 0x7fdd1010d030 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/297372705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd10105cf0 0x7fdd10106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/297372705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd10104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.593+0000 7fdd15ea2640 1 -- 172.21.15.96:0/297372705 >> 172.21.15.96:0/297372705 conn(0x7fdd10100440 msgr2=0x7fdd10102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.594+0000 7fdd15ea2640 1 -- 172.21.15.96:0/297372705 shutdown_connections 2024-09-10T22:59:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.594+0000 7fdd15ea2640 1 -- 172.21.15.96:0/297372705 wait complete. 2024-09-10T22:59:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.595+0000 7fdd15ea2640 1 Processor -- start 2024-09-10T22:59:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.595+0000 7fdd15ea2640 1 -- start start 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd1019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 0x7fdd1019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd101066b0 0x7fdd1019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd1019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd1019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53938/0 (socket says 172.21.15.96:53938) 2024-09-10T22:59:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/133837462 learned_addr learned my addr 172.21.15.96:0/133837462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd0effd640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 0x7fdd1019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd101066b0 0x7fdd1019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10110840 con 0x7fdd101066b0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101106c0 con 0x7fdd10105cf0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.596+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101109c0 con 0x7fdd10104ba0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 -- 172.21.15.96:0/133837462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 msgr2=0x7fdd1019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd1019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 -- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd101066b0 msgr2=0x7fdd1019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd101066b0 0x7fdd1019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1019c800 con 0x7fdd10105cf0 2024-09-10T22:59:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.597+0000 7fdd0effd640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 0x7fdd1019b310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d6d0 tx=0x7fdd0400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.598+0000 7fdd0cff9640 1 -- 172.21.15.96:0/133837462 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04017070 con 0x7fdd10105cf0 2024-09-10T22:59:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.598+0000 7fdd0cff9640 1 -- 172.21.15.96:0/133837462 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040047a0 con 0x7fdd10105cf0 2024-09-10T22:59:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.598+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1019caf0 con 0x7fdd10105cf0 2024-09-10T22:59:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.598+0000 7fdd0cff9640 1 -- 172.21.15.96:0/133837462 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04009b20 con 0x7fdd10105cf0 2024-09-10T22:59:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.598+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101a31c0 con 0x7fdd10105cf0 2024-09-10T22:59:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.602+0000 7fdd18c70740 1 Processor -- start 2024-09-10T22:59:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.602+0000 7fdd18c70740 1 -- start start 2024-09-10T22:59:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.602+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 0x558c379bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd18c70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c3792e6f0 con 0x558c379bf190 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd18c70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558c3791db20 con 0x558c379c1bd0 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd18c70740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558c37921930 con 0x558c379bc860 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42446/0 (socket says 172.21.15.96:42446) 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1105307060 learned_addr learned my addr 172.21.15.96:0/1105307060 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 0x558c379bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.603+0000 7fdd0cff9640 1 -- 172.21.15.96:0/133837462 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdd04005220 con 0x7fdd10105cf0 2024-09-10T22:59:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1105307060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 msgr2=0x558c379bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 0x558c379bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1105307060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 msgr2=0x558c379c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1105307060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c378e3530 con 0x558c379bf190 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 0x558c379bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000bd90 tx=0x7fdd0000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317385d9e8e3f50a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdcf5ffb640 1 -- 172.21.15.96:0/1105307060 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00015070 con 0x558c379bf190 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdcf5ffb640 1 -- 172.21.15.96:0/1105307060 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd000035f0 con 0x558c379bf190 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.604+0000 7fdcf5ffb640 1 -- 172.21.15.96:0/1105307060 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000116a0 con 0x558c379bf190 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 -- 172.21.15.96:0/1105307060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 msgr2=0x558c379c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000bd90 tx=0x7fdd0000f730 comp rx=0 tx=0).stop 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd0cff9640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 0x7fdcf80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 -- 172.21.15.96:0/1105307060 shutdown_connections 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379c1580 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1105307060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bc860 0x558c379bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 -- 172.21.15.96:0/1105307060 >> 172.21.15.96:0/1105307060 conn(0x558c379b79b0 msgr2=0x558c379b8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 -- 172.21.15.96:0/1105307060 shutdown_connections 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.605+0000 7fdd18c70740 1 -- 172.21.15.96:0/1105307060 wait complete. 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.606+0000 7fdd0cff9640 1 -- 172.21.15.96:0/133837462 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdd040e56c0 con 0x7fdd10105cf0 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.606+0000 7fdd18c70740 1 -- start start 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.606+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 0x7fdcf80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.606+0000 7fdd18c70740 1 Processor -- start 2024-09-10T22:59:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bc860 0x558c379c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558c37855ab0 con 0x558c379bc860 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558c378c3730 con 0x558c379c1bd0 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558c379c4c00 con 0x558c379bf190 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.607+0000 7fdd18c70740 10 client.0 _ll_register_callbacks cb 0x558c379176b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46882/0 (socket says 172.21.15.96:46882) 2024-09-10T22:59:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 -- 172.21.15.96:0/1325578122 learned_addr learned my addr 172.21.15.96:0/1325578122 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 -- 172.21.15.96:0/1325578122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 msgr2=0x558c379c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 -- 172.21.15.96:0/1325578122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bc860 msgr2=0x558c379c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bc860 0x558c379c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.608+0000 7fdd0ffff640 1 -- 172.21.15.96:0/1325578122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3767f790 con 0x558c379c1bd0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009a60 tx=0x7fdd08030e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efdd2a7d82b3e01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08051060 con 0x558c379c1bd0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd000035f0 con 0x558c379c1bd0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdd0803d830 con 0x558c379c1bd0 2024-09-10T22:59:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdd08046050 con 0x558c379c1bd0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdd18c70740 10 client.27782 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c376f8620 con 0x558c379c1bd0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd000035f0 con 0x558c379c1bd0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.609+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdd08056070 con 0x558c379c1bd0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 10 client.27782 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 10 client.27782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 20 client.27782 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c379e27f0 0x558c379e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 4 client.27782 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c37930640 con 0x558c379e27f0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.610+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c379e27f0 0x558c379e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.611+0000 7fdceeffd640 10 client.27782 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.611+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 0x7fdcf80b78a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd0804e000 tx=0x7fdd0802f6b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.611+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c379e27f0 0x558c379e4c60 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc634f44866356f server_cookie=e79c3deae6dce6a4 in_seq=0 out_seq=0 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.612+0000 7fdceeffd640 1 -- 172.21.15.96:0/1325578122 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c37930640 con 0x558c379e27f0 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.612+0000 7fdceeffd640 10 client.27782 handle_command_reply: tid=0 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.612+0000 7fdd18c70740 1 client.27782 shutdown 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 msgr2=0x558c379c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009a60 tx=0x7fdd08030e00 comp rx=0 tx=0).stop 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 shutdown_connections 2024-09-10T22:59:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c379e27f0 0x558c379e4c60 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379c1bd0 0x558c379c7130 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1325578122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bc860 0x558c379c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 >> 172.21.15.96:0/1325578122 conn(0x558c379b79b0 msgr2=0x558c379b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdceeffd640 0 client.27782 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 shutdown_connections 2024-09-10T22:59:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 1 -- 172.21.15.96:0/1325578122 wait complete. 2024-09-10T22:59:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.613+0000 7fdd18c70740 20 client.27782 trim_cache size 0 max 16384 2024-09-10T22:59:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.636+0000 7fdd18c70740 1 Processor -- start 2024-09-10T22:59:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.636+0000 7fdd18c70740 1 -- start start 2024-09-10T22:59:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.636+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.636+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.636+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd18c70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101a3810 con 0x558c379c1bd0 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd18c70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101a5350 con 0x558c379bc860 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd18c70740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101a3ee0 con 0x558c379bf190 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42460/0 (socket says 172.21.15.96:42460) 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 -- 172.21.15.96:0/3081349412 learned_addr learned my addr 172.21.15.96:0/3081349412 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0effd640 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 -- 172.21.15.96:0/3081349412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 msgr2=0x558c379c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 -- 172.21.15.96:0/3081349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 msgr2=0x558c379c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 -- 172.21.15.96:0/3081349412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c379c68d0 con 0x558c379c1bd0 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdd08031ba0 tx=0x7fdd0805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406c3f9503d4e8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdcf57fa640 1 -- 172.21.15.96:0/3081349412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0805c030 con 0x558c379c1bd0 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdcf57fa640 1 -- 172.21.15.96:0/3081349412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08067070 con 0x558c379c1bd0 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.637+0000 7fdcf57fa640 1 -- 172.21.15.96:0/3081349412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0802f330 con 0x558c379c1bd0 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 -- 172.21.15.96:0/3081349412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 msgr2=0x558c379c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdd08031ba0 tx=0x7fdd0805e000 comp rx=0 tx=0).stop 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 -- 172.21.15.96:0/3081349412 shutdown_connections 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379c1bd0 0x558c379c5660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379bf190 0x558c379c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 --2- 172.21.15.96:0/3081349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 -- 172.21.15.96:0/3081349412 >> 172.21.15.96:0/3081349412 conn(0x558c379b79b0 msgr2=0x558c379bbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 -- 172.21.15.96:0/3081349412 shutdown_connections 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.638+0000 7fdd18c70740 1 -- 172.21.15.96:0/3081349412 wait complete. 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 -- start start 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 Processor -- start 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379c1bd0 0x558c379cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101a4c80 con 0x558c379bf190 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101a45b0 con 0x558c379bc860 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd18c70740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100737d0 con 0x558c379c1bd0 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.639+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46900/0 (socket says 172.21.15.96:46900) 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd18c70740 10 client.0 _ll_register_callbacks cb 0x7fdd08031690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1489672831 learned_addr learned my addr 172.21.15.96:0/1489672831 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0ffff640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379c1bd0 0x558c379cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1489672831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379c1bd0 msgr2=0x558c379cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379c1bd0 0x558c379cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1489672831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 msgr2=0x558c379cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 -- 172.21.15.96:0/1489672831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c379ccf90 con 0x558c379bc860 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd0f7fe640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd000058c0 tx=0x7fdd0000ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47e55edb2811baa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00015040 con 0x558c379bc860 2024-09-10T22:59:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08067070 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.640+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdd00005590 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdd00033050 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 10 client.27788 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c37a4f770 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08067070 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdd00011810 con 0x558c379bc860 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 10 client.27788 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 10 client.27788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 20 client.27788 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c37a4fea0 0x558c37a522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 4 client.27788 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c3792c0f0 con 0x558c37a4fea0 2024-09-10T22:59:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c37a4fea0 0x558c37a522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.641+0000 7fdceeffd640 10 client.27788 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.642+0000 7fdd0effd640 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c37a4fea0 0x558c37a522c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1b85cd936d791c server_cookie=b2fb1f6fbdb02b28 in_seq=0 out_seq=0 2024-09-10T22:59:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.664+0000 7fdceeffd640 1 -- 172.21.15.96:0/1489672831 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c3792c0f0 con 0x558c37a4fea0 2024-09-10T22:59:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.664+0000 7fdceeffd640 10 client.27788 handle_command_reply: tid=0 2024-09-10T22:59:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.664+0000 7fdd18c70740 1 client.27788 shutdown 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.664+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 msgr2=0x558c379cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.664+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd000058c0 tx=0x7fdd0000ba00 comp rx=0 tx=0).stop 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 shutdown_connections 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558c37a4fea0 0x558c37a522c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdceeffd640 0 client.27788 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558c379c1bd0 0x558c379cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558c379bf190 0x558c379cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 --2- 172.21.15.96:0/1489672831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558c379bc860 0x558c379cd740 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 >> 172.21.15.96:0/1489672831 conn(0x558c379b79b0 msgr2=0x558c379bb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 shutdown_connections 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.665+0000 7fdd18c70740 1 -- 172.21.15.96:0/1489672831 wait complete. 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.666+0000 7fdd18c70740 20 client.27788 trim_cache size 0 max 16384 2024-09-10T22:59:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T22:59:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T22:59:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T22:59:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T22:59:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 msgr2=0x7fdcf80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 0x7fdcf80b78a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdd0804e000 tx=0x7fdd0802f6b0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 msgr2=0x7fdd1019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 0x7fdd1019b310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d6d0 tx=0x7fdd0400db90 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 shutdown_connections 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd10105cf0 0x7fdd1019b310 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdcf80b53e0 0x7fdcf80b78a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd101066b0 0x7fdd1019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 --2- 172.21.15.96:0/133837462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd10104ba0 0x7fdd1019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 >> 172.21.15.96:0/133837462 conn(0x7fdd10100440 msgr2=0x7fdd10102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 shutdown_connections 2024-09-10T22:59:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T22:59:55.667+0000 7fdd15ea2640 1 -- 172.21.15.96:0/133837462 wait complete. 2024-09-10T22:59:55.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:55 smithi096 ceph-mon[30869]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-10T22:59:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:55 smithi121 ceph-mon[35119]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-10T22:59:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:55 smithi117 ceph-mon[35081]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 635 op/s 2024-09-10T22:59:55.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 -- 172.21.15.96:0/3317459593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 msgr2=0x7f32e8102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 --2- 172.21.15.96:0/3317459593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8102080 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009900 tx=0x7f32e402a410 comp rx=0 tx=0).stop 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 -- 172.21.15.96:0/3317459593 shutdown_connections 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 --2- 172.21.15.96:0/3317459593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e8108030 0x7f32e810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 --2- 172.21.15.96:0/3317459593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e81076b0 0x7f32e8107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 --2- 172.21.15.96:0/3317459593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8102080 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 -- 172.21.15.96:0/3317459593 >> 172.21.15.96:0/3317459593 conn(0x7f32e80fd6d0 msgr2=0x7f32e80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 -- 172.21.15.96:0/3317459593 shutdown_connections 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.876+0000 7f32f071d640 1 -- 172.21.15.96:0/3317459593 wait complete. 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 Processor -- start 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 -- start start 2024-09-10T22:59:55.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 0x7f32e81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e8108030 0x7f32e819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8111cb0 con 0x7f32e8101ca0 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8111b30 con 0x7f32e81076b0 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32f071d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8111e30 con 0x7f32e8108030 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32ee4b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32ee4b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42498/0 (socket says 172.21.15.96:42498) 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.877+0000 7f32ee4b9640 1 -- 172.21.15.96:0/1124368337 learned_addr learned my addr 172.21.15.96:0/1124368337 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32eecba640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e8108030 0x7f32e819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 0x7f32e81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 -- 172.21.15.96:0/1124368337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e8108030 msgr2=0x7f32e819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e8108030 0x7f32e819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 -- 172.21.15.96:0/1124368337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 msgr2=0x7f32e81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 0x7f32e81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 -- 172.21.15.96:0/1124368337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e819f330 con 0x7f32e8101ca0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 0x7f32e81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005940 tx=0x7f32e40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32df7fe640 1 -- 172.21.15.96:0/1124368337 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e403f030 con 0x7f32e8101ca0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32df7fe640 1 -- 172.21.15.96:0/1124368337 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e40044c0 con 0x7f32e8101ca0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32df7fe640 1 -- 172.21.15.96:0/1124368337 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4035070 con 0x7f32e8101ca0 2024-09-10T22:59:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e819f5c0 con 0x7f32e8101ca0 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.878+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e81a5ce0 con 0x7f32e8101ca0 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32df7fe640 1 -- 172.21.15.96:0/1124368337 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f32e403ba20 con 0x7f32e8101ca0 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 Processor -- start 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 -- start start 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fca1f7e60 con 0x563fca281800 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563fca1e7290 con 0x563fca28d790 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.881+0000 7f32f2af2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563fca1eb0a0 con 0x563fca280ee0 2024-09-10T22:59:55.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.882+0000 7f32eecba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.882+0000 7f32eecba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53994/0 (socket says 172.21.15.96:53994) 2024-09-10T22:59:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.882+0000 7f32eecba640 1 -- 172.21.15.96:0/1242490819 learned_addr learned my addr 172.21.15.96:0/1242490819 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.882+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.882+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32eecba640 1 -- 172.21.15.96:0/1242490819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 msgr2=0x563fca28fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32eecba640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32eecba640 1 -- 172.21.15.96:0/1242490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 msgr2=0x563fca28d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32eecba640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32eecba640 1 -- 172.21.15.96:0/1242490819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc9f45930 con 0x563fca280ee0 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.883+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.884+0000 7f32df7fe640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 0x7f32d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.884+0000 7f32eecba640 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011440 tx=0x7f32e0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f903e192fadd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.885+0000 7f32df7fe640 1 -- 172.21.15.96:0/1124368337 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f32e41061d0 con 0x7f32e8101ca0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.885+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 0x7f32d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.886+0000 7f32dcff9640 1 -- 172.21.15.96:0/1242490819 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0004d20 con 0x563fca280ee0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.886+0000 7f32dcff9640 1 -- 172.21.15.96:0/1242490819 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0004050 con 0x563fca280ee0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.886+0000 7f32dcff9640 1 -- 172.21.15.96:0/1242490819 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000f7a0 con 0x563fca280ee0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- 172.21.15.96:0/1242490819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 msgr2=0x563fca2812c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32ee4b9640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 0x7f32d40b77a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32e40cc9a0 tx=0x7f32e4109cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011440 tx=0x7f32e0011900 comp rx=0 tx=0).stop 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- 172.21.15.96:0/1242490819 shutdown_connections 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1242490819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca2812c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- 172.21.15.96:0/1242490819 >> 172.21.15.96:0/1242490819 conn(0x563fca27ed10 msgr2=0x563fca27feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- 172.21.15.96:0/1242490819 shutdown_connections 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- 172.21.15.96:0/1242490819 wait complete. 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.887+0000 7f32f2af2740 1 -- start start 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 Processor -- start 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca28a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fca048890 con 0x563fca281800 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563fca181d40 con 0x563fca28d790 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563fca290440 con 0x563fca280ee0 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32f2af2740 10 client.0 _ll_register_callbacks cb 0x563fca1e5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42520/0 (socket says 172.21.15.96:42520) 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 -- 172.21.15.96:0/3848283241 learned_addr learned my addr 172.21.15.96:0/3848283241 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32eecba640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca28a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32eecba640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 -- 172.21.15.96:0/3848283241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 msgr2=0x563fca28a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca28a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 -- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 msgr2=0x563fca28b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32edcb8640 1 -- 172.21.15.96:0/3848283241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fca299920 con 0x563fca281800 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.888+0000 7f32eecba640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32edcb8640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009970 tx=0x7f32d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0ccb011d984696 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80122e0 con 0x563fca281800 2024-09-10T22:59:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f32d8012480 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e0004050 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32d8004020 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 10 client.18189 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563fca29c2d0 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0004050 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f32d8025030 con 0x563fca281800 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 10 client.18189 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:55.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 20 client.18189 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca29d880 0x563fca29fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 4 client.18189 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.889+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fca1f9db0 con 0x563fca29d880 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.890+0000 7f32eecba640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca29d880 0x563fca29fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.890+0000 7f32eecba640 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca29d880 0x563fca29fc70 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02d4c8d7b46760 server_cookie=3328c5a2a6ec003f in_seq=0 out_seq=0 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.890+0000 7f32d1ffb640 10 client.18189 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32d1ffb640 1 -- 172.21.15.96:0/3848283241 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563fca1f9db0 con 0x563fca29d880 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32d1ffb640 10 client.18189 handle_command_reply: tid=0 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 client.18189 shutdown 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 msgr2=0x563fca28ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009970 tx=0x7f32d8009e30 comp rx=0 tx=0).stop 2024-09-10T22:59:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 shutdown_connections 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca29d880 0x563fca29fc70 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca28d790 0x563fca28b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca281800 0x563fca28ad90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32d1ffb640 0 client.18189 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 --2- 172.21.15.96:0/3848283241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca280ee0 0x563fca28a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 >> 172.21.15.96:0/3848283241 conn(0x563fca27ed10 msgr2=0x563fca27df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 shutdown_connections 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 1 -- 172.21.15.96:0/3848283241 wait complete. 2024-09-10T22:59:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.891+0000 7f32f2af2740 20 client.18189 trim_cache size 0 max 16384 2024-09-10T22:59:55.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 Processor -- start 2024-09-10T22:59:55.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 -- start start 2024-09-10T22:59:55.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca287a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca287fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81ac050 con 0x563fca280ee0 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81adb90 con 0x563fca281800 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.918+0000 7f32f2af2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81ac720 con 0x563fca27ed10 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca287a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:46956/0 (socket says 172.21.15.96:46956) 2024-09-10T22:59:55.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 -- 172.21.15.96:0/1936800822 learned_addr learned my addr 172.21.15.96:0/1936800822 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 -- 172.21.15.96:0/1936800822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 msgr2=0x563fca287a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca287a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 -- 172.21.15.96:0/1936800822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 msgr2=0x563fca287fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca287fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.919+0000 7f32eecba640 1 -- 172.21.15.96:0/1936800822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fca2803f0 con 0x563fca281800 2024-09-10T22:59:55.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.923+0000 7f32eecba640 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e00099b0 tx=0x7f32e002ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66228935b233baf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.924+0000 7f32d3fff640 1 -- 172.21.15.96:0/1936800822 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e002b030 con 0x563fca281800 2024-09-10T22:59:55.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.924+0000 7f32d3fff640 1 -- 172.21.15.96:0/1936800822 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0004500 con 0x563fca281800 2024-09-10T22:59:55.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.924+0000 7f32d3fff640 1 -- 172.21.15.96:0/1936800822 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e002e050 con 0x563fca281800 2024-09-10T22:59:55.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.925+0000 7f32f2af2740 1 -- 172.21.15.96:0/1936800822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 msgr2=0x563fca2a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.925+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32e00099b0 tx=0x7f32e002ef90 comp rx=0 tx=0).stop 2024-09-10T22:59:55.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.926+0000 7f32f2af2740 1 -- 172.21.15.96:0/1936800822 shutdown_connections 2024-09-10T22:59:55.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.926+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca2a5530 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.926+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca287fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.926+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1936800822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca287a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.926+0000 7f32f2af2740 1 -- 172.21.15.96:0/1936800822 >> 172.21.15.96:0/1936800822 conn(0x563fca2861f0 msgr2=0x563fca28c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.930+0000 7f32f2af2740 1 -- 172.21.15.96:0/1936800822 shutdown_connections 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.932+0000 7f32f2af2740 1 -- 172.21.15.96:0/1936800822 wait complete. 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 -- start start 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 Processor -- start 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca293ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca294470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca311f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81ad4c0 con 0x563fca280ee0 2024-09-10T22:59:55.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81acdf0 con 0x563fca281800 2024-09-10T22:59:55.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81aa030 con 0x563fca27ed10 2024-09-10T22:59:55.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.933+0000 7f32f2af2740 10 client.0 _ll_register_callbacks cb 0x7f32d8012d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:55.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.934+0000 7f32eecba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca293ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.934+0000 7f32eecba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca311f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.934+0000 7f32eecba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca293ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:54008/0 (socket says 172.21.15.96:54008) 2024-09-10T22:59:55.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.934+0000 7f32eecba640 1 -- 172.21.15.96:0/1719780859 learned_addr learned my addr 172.21.15.96:0/1719780859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:55.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.934+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca294470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 -- 172.21.15.96:0/1719780859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 msgr2=0x563fca293ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca293ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 -- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 msgr2=0x563fca311f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca311f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 -- 172.21.15.96:0/1719780859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fca323670 con 0x563fca280ee0 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32edcb8640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca294470 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f32d8010a00 tx=0x7f32d8012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8404d19120a419e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.935+0000 7f32d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.936+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8000e60 con 0x563fca280ee0 2024-09-10T22:59:55.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.936+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f32d8005b90 con 0x563fca280ee0 2024-09-10T22:59:55.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.941+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e0004500 con 0x563fca280ee0 2024-09-10T22:59:55.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.941+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32d8034050 con 0x563fca280ee0 2024-09-10T22:59:55.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.942+0000 7f32f2af2740 10 client.18192 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:55.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.942+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563fca3249b0 con 0x563fca280ee0 2024-09-10T22:59:55.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.945+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0004500 con 0x563fca280ee0 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f32d800b820 con 0x563fca280ee0 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 10 client.18192 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 20 client.18192 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca28d790 0x563fca3270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 4 client.18192 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T22:59:55.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.946+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563fca1f5860 con 0x563fca28d790 2024-09-10T22:59:55.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.949+0000 7f32eecba640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca28d790 0x563fca3270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 740.0 s, 1520.0 tps, lat 65.006 ms stddev 68.641 2024-09-10T22:59:55.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.952+0000 7f32eecba640 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca28d790 0x563fca3270e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff5f976fbc35a04 server_cookie=80e1df2e56815d0a in_seq=0 out_seq=0 2024-09-10T22:59:55.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.952+0000 7f32d1ffb640 10 client.18192 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.956+0000 7f32d1ffb640 1 -- 172.21.15.96:0/1719780859 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563fca1f5860 con 0x563fca28d790 2024-09-10T22:59:55.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.956+0000 7f32d1ffb640 10 client.18192 handle_command_reply: tid=0 2024-09-10T22:59:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.956+0000 7f32f2af2740 1 client.18192 shutdown 2024-09-10T22:59:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.957+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 msgr2=0x563fca294470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.957+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca294470 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f32d8010a00 tx=0x7f32d8012590 comp rx=0 tx=0).stop 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 shutdown_connections 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563fca28d790 0x563fca3270e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563fca281800 0x563fca311f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fca280ee0 0x563fca294470 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 --2- 172.21.15.96:0/1719780859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563fca27ed10 0x563fca293ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 >> 172.21.15.96:0/1719780859 conn(0x563fca2861f0 msgr2=0x563fca28be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32d1ffb640 0 client.18192 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 shutdown_connections 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 1 -- 172.21.15.96:0/1719780859 wait complete. 2024-09-10T22:59:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.967+0000 7f32f2af2740 20 client.18192 trim_cache size 0 max 16384 2024-09-10T22:59:55.970 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:59:55.970 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T22:59:55.970 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "9e936084-729f-4a85-93c6-78633e5cb370", 2024-09-10T22:59:55.970 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T22:59:55.970 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 msgr2=0x7f32d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 0x7f32d40b77a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f32e40cc9a0 tx=0x7f32e4109cc0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 msgr2=0x7f32e8199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005940 tx=0x7f32e40040c0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 shutdown_connections 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32e81076b0 0x7f32e81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f32d40b52e0 0x7f32d40b77a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32e8108030 0x7f32e819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 --2- 172.21.15.96:0/1124368337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32e8101ca0 0x7f32e8199280 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 >> 172.21.15.96:0/1124368337 conn(0x7f32e80fd6d0 msgr2=0x7f32e806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 shutdown_connections 2024-09-10T22:59:55.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:55.972+0000 7f32f071d640 1 -- 172.21.15.96:0/1124368337 wait complete. 2024-09-10T22:59:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:56 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T22:59:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:56 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:56 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:59:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:56 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T22:59:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:56 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:56 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T22:59:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:56 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:56 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:59:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:56 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T22:59:56.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:56 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:56 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T22:59:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:56 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:56 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:59:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:56 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T22:59:56.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:56 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:57.166 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T22:59:57.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:57 smithi096 ceph-mon[30869]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-09-10T22:59:57.551 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:59:57.690 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:57 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:57 smithi121 ceph-mon[35119]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-09-10T22:59:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:57 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:57 smithi117 ceph-mon[35081]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-09-10T22:59:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:57 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T22:59:58.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 -- 172.21.15.96:0/2166508239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a0101cb0 msgr2=0x7f98a0102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 --2- 172.21.15.96:0/2166508239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a0101cb0 0x7f98a0102090 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9890009900 tx=0x7f989002a440 comp rx=0 tx=0).stop 2024-09-10T22:59:58.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 -- 172.21.15.96:0/2166508239 shutdown_connections 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 --2- 172.21.15.96:0/2166508239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 0x7f98a006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 --2- 172.21.15.96:0/2166508239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a01076c0 0x7f98a0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 --2- 172.21.15.96:0/2166508239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a0101cb0 0x7f98a0102090 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 -- 172.21.15.96:0/2166508239 >> 172.21.15.96:0/2166508239 conn(0x7f98a00fd700 msgr2=0x7f98a00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 -- 172.21.15.96:0/2166508239 shutdown_connections 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.058+0000 7f98a669c640 1 -- 172.21.15.96:0/2166508239 wait complete. 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 Processor -- start 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 -- start start 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0101cb0 0x7f98a01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 0x7f98a010c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01150e0 con 0x7f98a01076c0 2024-09-10T22:59:58.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0114f60 con 0x7f98a0108040 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f98a669c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0115260 con 0x7f98a0101cb0 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.059+0000 7f989f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58762/0 (socket says 172.21.15.96:58762) 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 -- 172.21.15.96:0/3127978318 learned_addr learned my addr 172.21.15.96:0/3127978318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f98a4c39640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 0x7f98a010c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989ffff640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0101cb0 0x7f98a01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 -- 172.21.15.96:0/3127978318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0101cb0 msgr2=0x7f98a01a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0101cb0 0x7f98a01a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 -- 172.21.15.96:0/3127978318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 msgr2=0x7f98a010c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 0x7f98a010c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 -- 172.21.15.96:0/3127978318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a010d140 con 0x7f98a01076c0 2024-09-10T22:59:58.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.060+0000 7f989f7fe640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f989400b450 tx=0x7f989400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.064+0000 7f989d7fa640 1 -- 172.21.15.96:0/3127978318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894015020 con 0x7f98a01076c0 2024-09-10T22:59:58.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.064+0000 7f989d7fa640 1 -- 172.21.15.96:0/3127978318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894011d40 con 0x7f98a01076c0 2024-09-10T22:59:58.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.064+0000 7f989d7fa640 1 -- 172.21.15.96:0/3127978318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400f340 con 0x7f98a01076c0 2024-09-10T22:59:58.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.065+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a010d430 con 0x7f98a01076c0 2024-09-10T22:59:58.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.065+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a0113d90 con 0x7f98a01076c0 2024-09-10T22:59:58.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.068+0000 7f98a8a71740 1 Processor -- start 2024-09-10T22:59:58.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.070+0000 7f989d7fa640 1 -- 172.21.15.96:0/3127978318 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9894011890 con 0x7f98a01076c0 2024-09-10T22:59:58.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 -- start start 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 0x5641a2623860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2632120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641a2599e50 con 0x5641a2623480 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641a2589280 con 0x5641a2623da0 2024-09-10T22:59:58.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f98a8a71740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641a258d090 con 0x5641a262fd30 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.071+0000 7f989ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35596/0 (socket says 172.21.15.96:35596) 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 -- 172.21.15.96:0/462357082 learned_addr learned my addr 172.21.15.96:0/462357082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989f7fe640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2632120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f98a4c39640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 0x5641a2623860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989d7fa640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 0x7f98880b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 -- 172.21.15.96:0/462357082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 msgr2=0x5641a2632120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2632120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 -- 172.21.15.96:0/462357082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 msgr2=0x5641a2623860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 0x5641a2623860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f989ffff640 1 -- 172.21.15.96:0/462357082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a22e7650 con 0x5641a2623da0 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.072+0000 7f98a4c39640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 0x5641a2623860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.073+0000 7f989d7fa640 1 -- 172.21.15.96:0/3127978318 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98940e5840 con 0x7f98a01076c0 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.073+0000 7f989f7fe640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2632120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.073+0000 7f98a4c39640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 0x7f98880b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.073+0000 7f989ffff640 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f989002a960 tx=0x7f9890004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c774493e2934649 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.074+0000 7f98867fc640 1 -- 172.21.15.96:0/462357082 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989003f070 con 0x5641a2623da0 2024-09-10T22:59:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.074+0000 7f98a4c39640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 0x7f98880b78a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f989800b6d0 tx=0x7f9898004af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.074+0000 7f98867fc640 1 -- 172.21.15.96:0/462357082 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890003aa0 con 0x5641a2623da0 2024-09-10T22:59:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.074+0000 7f98867fc640 1 -- 172.21.15.96:0/462357082 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890035070 con 0x5641a2623da0 2024-09-10T22:59:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- 172.21.15.96:0/462357082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 msgr2=0x5641a262f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f989002a960 tx=0x7f9890004200 comp rx=0 tx=0).stop 2024-09-10T22:59:58.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- 172.21.15.96:0/462357082 shutdown_connections 2024-09-10T22:59:58.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623da0 0x5641a262f7f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2632120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 --2- 172.21.15.96:0/462357082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2623480 0x5641a2623860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- 172.21.15.96:0/462357082 >> 172.21.15.96:0/462357082 conn(0x5641a26212b0 msgr2=0x5641a2622450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- 172.21.15.96:0/462357082 shutdown_connections 2024-09-10T22:59:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- 172.21.15.96:0/462357082 wait complete. 2024-09-10T22:59:58.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.075+0000 7f98a8a71740 1 -- start start 2024-09-10T22:59:58.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.076+0000 7f98a8a71740 1 Processor -- start 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.076+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a262f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.076+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.076+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a8a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5641a23ea700 con 0x5641a262fd30 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a8a71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641a2524110 con 0x5641a2623480 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a8a71740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641a2632950 con 0x5641a2623da0 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a8a71740 10 client.0 _ll_register_callbacks cb 0x5641a253d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a4c39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a4c39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58782/0 (socket says 172.21.15.96:58782) 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.077+0000 7f98a4c39640 1 -- 172.21.15.96:0/339646177 learned_addr learned my addr 172.21.15.96:0/339646177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.079+0000 7f98a4c39640 1 -- 172.21.15.96:0/339646177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 msgr2=0x5641a262cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.079+0000 7f98a4c39640 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.079+0000 7f98a4c39640 1 -- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 msgr2=0x5641a262f660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.079+0000 7f98a4c39640 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a262f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.079+0000 7f98a4c39640 1 -- 172.21.15.96:0/339646177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a2551c40 con 0x5641a262fd30 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.080+0000 7f98a4c39640 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f989800acb0 tx=0x7f989800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9155f9df01e48b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.080+0000 7f987f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.080+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898017e50 con 0x5641a262fd30 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.081+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890003aa0 con 0x5641a262fd30 2024-09-10T22:59:58.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.081+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9898023450 con 0x5641a262fd30 2024-09-10T22:59:58.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.083+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9898023690 con 0x5641a262fd30 2024-09-10T22:59:58.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.085+0000 7f98a8a71740 10 client.18204 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:58.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.085+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5641a263e370 con 0x5641a262fd30 2024-09-10T22:59:58.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.087+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890003aa0 con 0x5641a262fd30 2024-09-10T22:59:58.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.087+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f989802c9b0 con 0x5641a262fd30 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 10 client.18204 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 20 client.18204 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a263fc70 0x5641a2642130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 4 client.18204 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:59:58.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.088+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a259bda0 con 0x5641a263fc70 2024-09-10T22:59:58.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.089+0000 7f989ffff640 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a263fc70 0x5641a2642130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.089+0000 7f989ffff640 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a263fc70 0x5641a2642130 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb8d9f964fd2184 server_cookie=136975e093ce49f5 in_seq=0 out_seq=0 2024-09-10T22:59:58.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.089+0000 7f987f7fe640 10 client.18204 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:58.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.090+0000 7f987f7fe640 1 -- 172.21.15.96:0/339646177 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641a259bda0 con 0x5641a263fc70 2024-09-10T22:59:58.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.090+0000 7f987f7fe640 10 client.18204 handle_command_reply: tid=0 2024-09-10T22:59:58.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.091+0000 7f98a8a71740 1 client.18204 shutdown 2024-09-10T22:59:58.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.091+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 msgr2=0x5641a262d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.091+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f989800acb0 tx=0x7f989800d040 comp rx=0 tx=0).stop 2024-09-10T22:59:58.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 shutdown_connections 2024-09-10T22:59:58.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a263fc70 0x5641a2642130 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262d130 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 --2- 172.21.15.96:0/339646177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a262f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 >> 172.21.15.96:0/339646177 conn(0x5641a26212b0 msgr2=0x5641a2626e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.097+0000 7f987f7fe640 0 client.18204 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:58.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.101+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 shutdown_connections 2024-09-10T22:59:58.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.101+0000 7f98a8a71740 1 -- 172.21.15.96:0/339646177 wait complete. 2024-09-10T22:59:58.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.101+0000 7f98a8a71740 20 client.18204 trim_cache size 0 max 16384 2024-09-10T22:59:58.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 Processor -- start 2024-09-10T22:59:58.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 -- start start 2024-09-10T22:59:58.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a263bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0074050 con 0x5641a262fd30 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01c7600 con 0x5641a2623480 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a8a71740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0074720 con 0x5641a2623da0 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a4c39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a4c39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58792/0 (socket says 172.21.15.96:58792) 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f989f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f98a4c39640 1 -- 172.21.15.96:0/2681758183 learned_addr learned my addr 172.21.15.96:0/2681758183 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f989ffff640 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a263bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f989f7fe640 1 -- 172.21.15.96:0/2681758183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 msgr2=0x5641a263bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.130+0000 7f989f7fe640 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a263bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f989f7fe640 1 -- 172.21.15.96:0/2681758183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 msgr2=0x5641a262b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f989f7fe640 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f989f7fe640 1 -- 172.21.15.96:0/2681758183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a2623bd0 con 0x5641a2623da0 2024-09-10T22:59:58.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f989f7fe640 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262b330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5641a2620c00 tx=0x7f98940103e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992c3d86ea4846af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f9885ffb640 1 -- 172.21.15.96:0/2681758183 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940b3e60 con 0x5641a2623da0 2024-09-10T22:59:58.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f9885ffb640 1 -- 172.21.15.96:0/2681758183 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894010eb0 con 0x5641a2623da0 2024-09-10T22:59:58.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.131+0000 7f9885ffb640 1 -- 172.21.15.96:0/2681758183 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940ac700 con 0x5641a2623da0 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 -- 172.21.15.96:0/2681758183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 msgr2=0x5641a262b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262b330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5641a2620c00 tx=0x7f98940103e0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 -- 172.21.15.96:0/2681758183 shutdown_connections 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a262fd30 0x5641a262b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a2623da0 0x5641a262b330 secure :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5641a2620c00 tx=0x7f98940103e0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 --2- 172.21.15.96:0/2681758183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a263bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 -- 172.21.15.96:0/2681758183 >> 172.21.15.96:0/2681758183 conn(0x5641a26212b0 msgr2=0x5641a2627d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 -- 172.21.15.96:0/2681758183 shutdown_connections 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.132+0000 7f98a8a71740 1 -- 172.21.15.96:0/2681758183 wait complete. 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 -- start start 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 Processor -- start 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2647750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 0x5641a2648160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01c6f30 con 0x5641a2647ce0 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.133+0000 7f98a8a71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0074df0 con 0x5641a2623480 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f98a8a71740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0072050 con 0x5641a262fd30 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f98a8a71740 10 client.0 _ll_register_callbacks cb 0x7f9898009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:59:58.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:35630/0 (socket says 172.21.15.96:35630) 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 -- 172.21.15.96:0/722429327 learned_addr learned my addr 172.21.15.96:0/722429327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989f7fe640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2647750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f98a4c39640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 0x5641a2648160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 -- 172.21.15.96:0/722429327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 msgr2=0x5641a2647750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2647750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 -- 172.21.15.96:0/722429327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 msgr2=0x5641a2648160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 0x5641a2648160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 -- 172.21.15.96:0/722429327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a26340e0 con 0x5641a2623480 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f98a4c39640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 0x5641a2648160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f989ffff640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9890009900 tx=0x7f9890005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e9d1fd3c0d2cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f987f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890035040 con 0x5641a2623480 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641a2597850 con 0x5641a2623480 2024-09-10T22:59:58.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.134+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98900314a0 con 0x5641a2623480 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.135+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f989003f040 con 0x5641a2623480 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.136+0000 7f98a8a71740 10 client.27815 fetch_fsmap learned FSMap version 32 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.136+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5641a26ad140 con 0x5641a2623480 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.136+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641a2597850 con 0x5641a2623480 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f989005d050 con 0x5641a2623480 2024-09-10T22:59:58.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 10 client.27815 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 10 client.27815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 20 client.27815 populate_metadata read hostname 'smithi096' 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a26ad420 0x5641a26af810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 4 client.27815 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641a2594c80 con 0x5641a26ad420 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f989f7fe640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a26ad420 0x5641a26af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f989f7fe640 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a26ad420 0x5641a26af810 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0e68b43bc58a3 server_cookie=21a0b1eef38161f in_seq=0 out_seq=0 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.137+0000 7f987f7fe640 10 client.27815 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f987f7fe640 1 -- 172.21.15.96:0/722429327 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x5641a2594c80 con 0x5641a26ad420 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f987f7fe640 10 client.27815 handle_command_reply: tid=0 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 client.27815 shutdown 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 msgr2=0x5641a2638450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9890009900 tx=0x7f9890005fd0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 shutdown_connections 2024-09-10T22:59:58.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5641a26ad420 0x5641a26af810 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5641a2647ce0 0x5641a2648160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641a262fd30 0x5641a2647750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f987f7fe640 0 client.27815 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 --2- 172.21.15.96:0/722429327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641a2623480 0x5641a2638450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 >> 172.21.15.96:0/722429327 conn(0x5641a26212b0 msgr2=0x5641a262b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 shutdown_connections 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 1 -- 172.21.15.96:0/722429327 wait complete. 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.138+0000 7f98a8a71740 20 client.27815 trim_cache size 0 max 16384 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T22:59:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "9e936084-729f-4a85-93c6-78633e5cb370", 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T22:59:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T22:59:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T22:59:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T22:59:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T22:59:58.152 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T22:59:58.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 msgr2=0x7f98880b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 0x7f98880b78a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f989800b6d0 tx=0x7f9898004af0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 msgr2=0x7f98a01a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f989400b450 tx=0x7f989400b910 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 shutdown_connections 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98a0108040 0x7f98a010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98880b53e0 0x7f98880b78a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a01076c0 0x7f98a01a1c20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 --2- 172.21.15.96:0/3127978318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0101cb0 0x7f98a01a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 >> 172.21.15.96:0/3127978318 conn(0x7f98a00fd700 msgr2=0x7f98a0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 shutdown_connections 2024-09-10T22:59:58.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T22:59:58.139+0000 7f98a669c640 1 -- 172.21.15.96:0/3127978318 wait complete. 2024-09-10T22:59:59.610 INFO:tasks.cephfs.filesystem:scrub status for tag:9e936084-729f-4a85-93c6-78633e5cb370 - {'path': '/', 'tag': '9e936084-729f-4a85-93c6-78633e5cb370', 'options': 'recursive,force'} 2024-09-10T22:59:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 22:59:59 smithi096 ceph-mon[30869]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2024-09-10T22:59:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 22:59:59 smithi121 ceph-mon[35119]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2024-09-10T22:59:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 22:59:59 smithi117 ceph-mon[35081]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2024-09-10T23:00:00.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:00 smithi096 ceph-mon[30869]: overall HEALTH_OK 2024-09-10T23:00:00.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:00 smithi121 ceph-mon[35119]: overall HEALTH_OK 2024-09-10T23:00:00.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:00 smithi117 ceph-mon[35081]: overall HEALTH_OK 2024-09-10T23:00:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 745.0 s, 2218.8 tps, lat 45.187 ms stddev 48.961 2024-09-10T23:00:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:01 smithi096 ceph-mon[30869]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.1 MiB/s wr, 787 op/s 2024-09-10T23:00:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:01 smithi121 ceph-mon[35119]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.1 MiB/s wr, 787 op/s 2024-09-10T23:00:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:01 smithi117 ceph-mon[35081]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.1 MiB/s wr, 787 op/s 2024-09-10T23:00:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:03 smithi096 ceph-mon[30869]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.9 MiB/s wr, 699 op/s 2024-09-10T23:00:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:03 smithi121 ceph-mon[35119]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.9 MiB/s wr, 699 op/s 2024-09-10T23:00:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:03 smithi117 ceph-mon[35081]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.9 MiB/s wr, 699 op/s 2024-09-10T23:00:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:04 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:00:04.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:04 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:00:04.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:04 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:00:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:05 smithi096 ceph-mon[30869]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 738 op/s 2024-09-10T23:00:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:05 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:00:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:05 smithi121 ceph-mon[35119]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 738 op/s 2024-09-10T23:00:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:05 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:00:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:05 smithi117 ceph-mon[35081]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 738 op/s 2024-09-10T23:00:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:05 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:00:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 750.0 s, 1757.6 tps, lat 57.043 ms stddev 66.937 2024-09-10T23:00:07.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:07 smithi096 ceph-mon[30869]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.7 MiB/s wr, 820 op/s 2024-09-10T23:00:07.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:07 smithi121 ceph-mon[35119]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.7 MiB/s wr, 820 op/s 2024-09-10T23:00:07.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:07 smithi117 ceph-mon[35081]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.7 MiB/s wr, 820 op/s 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf367fc640 1 -- 172.21.15.96:0/4004652886 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20031810 con 0x7faf38100c80 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4004652886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 msgr2=0x7faf38101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4004652886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf38101080 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faf20009900 tx=0x7faf2002a410 comp rx=0 tx=0).stop 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4004652886 shutdown_connections 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4004652886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf38101650 0x7faf38101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4004652886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf38106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4004652886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf38101080 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4004652886 >> 172.21.15.96:0/4004652886 conn(0x7faf380fc880 msgr2=0x7faf380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.803+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4004652886 shutdown_connections 2024-09-10T23:00:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.804+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4004652886 wait complete. 2024-09-10T23:00:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.804+0000 7faf3d9a3640 1 Processor -- start 2024-09-10T23:00:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 -- start start 2024-09-10T23:00:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf38101650 0x7faf3819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf381a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf38114c00 con 0x7faf38100c80 2024-09-10T23:00:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf38114a80 con 0x7faf38101650 2024-09-10T23:00:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf3d9a3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf38114d80 con 0x7faf38102210 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36234/0 (socket says 172.21.15.96:36234) 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf36ffd640 1 -- 172.21.15.96:0/4193972027 learned_addr learned my addr 172.21.15.96:0/4193972027 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.805+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf381a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 -- 172.21.15.96:0/4193972027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf38101650 msgr2=0x7faf3819fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:00:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf38101650 0x7faf3819fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 -- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 msgr2=0x7faf3819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 -- 172.21.15.96:0/4193972027 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf381a0ef0 con 0x7faf38102210 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf36ffd640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf381a0630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faf3000ec50 tx=0x7faf3000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf3c9a1640 1 -- 172.21.15.96:0/4193972027 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30003b70 con 0x7faf38102210 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf3c9a1640 1 -- 172.21.15.96:0/4193972027 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf30003d10 con 0x7faf38102210 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf381a11e0 con 0x7faf38102210 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf3c9a1640 1 -- 172.21.15.96:0/4193972027 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30010830 con 0x7faf38102210 2024-09-10T23:00:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.806+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf381a78b0 con 0x7faf38102210 2024-09-10T23:00:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.809+0000 7faf3c9a1640 1 -- 172.21.15.96:0/4193972027 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7faf30003ec0 con 0x7faf38102210 2024-09-10T23:00:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 Processor -- start 2024-09-10T23:00:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 -- start start 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc4be0 0x5577f2fc70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5577f2f206f0 con 0x5577f2fc4be0 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5577f2f0fb20 con 0x5577f2fbf870 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf4084c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5577f2f13930 con 0x5577f2fc21a0 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.810+0000 7faf3c9a1640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 0x7faf240b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf3c9a1640 1 -- 172.21.15.96:0/4193972027 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf300e6140 con 0x7faf38102210 2024-09-10T23:00:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 0x7faf240b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48012/0 (socket says 172.21.15.96:48012) 2024-09-10T23:00:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 -- 172.21.15.96:0/2434852979 learned_addr learned my addr 172.21.15.96:0/2434852979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 -- 172.21.15.96:0/2434852979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 msgr2=0x5577f2fc1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 -- 172.21.15.96:0/2434852979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc4be0 msgr2=0x5577f2fc70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc4be0 0x5577f2fc70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.811+0000 7faf36ffd640 1 -- 172.21.15.96:0/2434852979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f2ed5530 con 0x5577f2fc21a0 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.812+0000 7faf36ffd640 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf20005ec0 tx=0x7faf20005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ff3e425aa7d21d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.813+0000 7faf2dffb640 1 -- 172.21.15.96:0/2434852979 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20035030 con 0x5577f2fc21a0 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.813+0000 7faf2dffb640 1 -- 172.21.15.96:0/2434852979 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf20040070 con 0x5577f2fc21a0 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.813+0000 7faf2dffb640 1 -- 172.21.15.96:0/2434852979 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20003d30 con 0x5577f2fc21a0 2024-09-10T23:00:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.814+0000 7faf4084c740 1 -- 172.21.15.96:0/2434852979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 msgr2=0x5577f2fc4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.814+0000 7faf4084c740 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf20005ec0 tx=0x7faf20005bd0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.814+0000 7faf377fe640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 0x7faf240b7820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf300a8ed0 tx=0x7faf30004990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 -- 172.21.15.96:0/2434852979 shutdown_connections 2024-09-10T23:00:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc4be0 0x5577f2fc70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc4590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 --2- 172.21.15.96:0/2434852979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 -- 172.21.15.96:0/2434852979 >> 172.21.15.96:0/2434852979 conn(0x5577f2fa99b0 msgr2=0x5577f2faa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 -- 172.21.15.96:0/2434852979 shutdown_connections 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 -- 172.21.15.96:0/2434852979 wait complete. 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.815+0000 7faf4084c740 1 -- start start 2024-09-10T23:00:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.816+0000 7faf4084c740 1 Processor -- start 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.816+0000 7faf4084c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fbf870 0x5577f2fc8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.816+0000 7faf4084c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.816+0000 7faf4084c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf4084c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5577f2e47ab0 con 0x5577f2fbf870 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf4084c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5577f2eb5730 con 0x5577f2fc4be0 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf4084c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5577f2fc7b60 con 0x5577f2fc21a0 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf2e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fbf870 0x5577f2fc8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:48016/0 (socket says 172.21.15.96:48016) 2024-09-10T23:00:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 -- 172.21.15.96:0/871006950 learned_addr learned my addr 172.21.15.96:0/871006950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf4084c740 10 client.0 _ll_register_callbacks cb 0x5577f2f096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 -- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 msgr2=0x5577f2fca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 -- 172.21.15.96:0/871006950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fbf870 msgr2=0x5577f2fc8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.817+0000 7faf36ffd640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fbf870 0x5577f2fc8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf36ffd640 1 -- 172.21.15.96:0/871006950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f2c71790 con 0x5577f2fc21a0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf36ffd640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf20031df0 tx=0x7faf20031e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd56d125a822bcfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20035050 con 0x5577f2fc21a0 2024-09-10T23:00:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf20045880 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf20040070 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf20045a80 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf4084c740 10 client.27889 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.818+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577f2cea620 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf20040070 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faf2003cc50 con 0x5577f2fc21a0 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 10 client.27889 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 10 client.27889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 20 client.27889 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f2fd6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 4 client.27889 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f2f22640 con 0x5577f2fd4690 2024-09-10T23:00:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.819+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f2fd6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.820+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f2fd6ab0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24ac0c34a438dae server_cookie=9c4bcdabf82ae395 in_seq=0 out_seq=0 2024-09-10T23:00:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.820+0000 7faf16ffd640 10 client.27889 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.820+0000 7faf16ffd640 1 -- 172.21.15.96:0/871006950 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577f2f22640 con 0x5577f2fd4690 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.820+0000 7faf16ffd640 10 client.27889 handle_command_reply: tid=0 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.820+0000 7faf4084c740 1 client.27889 shutdown 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 msgr2=0x5577f2fc95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf20031df0 tx=0x7faf20031e20 comp rx=0 tx=0).stop 2024-09-10T23:00:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 shutdown_connections 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f2fd6ab0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc21a0 0x5577f2fc95a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf16ffd640 0 client.27889 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 --2- 172.21.15.96:0/871006950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fbf870 0x5577f2fc8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 >> 172.21.15.96:0/871006950 conn(0x5577f2fa99b0 msgr2=0x5577f2fa8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 shutdown_connections 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 1 -- 172.21.15.96:0/871006950 wait complete. 2024-09-10T23:00:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.821+0000 7faf4084c740 20 client.27889 trim_cache size 0 max 16384 2024-09-10T23:00:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.841+0000 7faf4084c740 1 Processor -- start 2024-09-10T23:00:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.841+0000 7faf4084c740 1 -- start start 2024-09-10T23:00:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc4be0 0x5577f2faa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf381a7bc0 con 0x5577f2fc21a0 2024-09-10T23:00:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf381a96b0 con 0x5577f2fbf870 2024-09-10T23:00:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf4084c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf381a8240 con 0x5577f2fc4be0 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36272/0 (socket says 172.21.15.96:36272) 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf36ffd640 1 -- 172.21.15.96:0/3387983072 learned_addr learned my addr 172.21.15.96:0/3387983072 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc4be0 0x5577f2faa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3387983072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc4be0 msgr2=0x5577f2faa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc4be0 0x5577f2faa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3387983072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 msgr2=0x5577f2fc5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.842+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3387983072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f2fd6e20 con 0x5577f2fbf870 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.843+0000 7faf36ffd640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.843+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc4fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf20031a00 tx=0x7faf280094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28551ad63555f087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.843+0000 7faf2d7fa640 1 -- 172.21.15.96:0/3387983072 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28020040 con 0x5577f2fbf870 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.843+0000 7faf2d7fa640 1 -- 172.21.15.96:0/3387983072 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf28011c50 con 0x5577f2fbf870 2024-09-10T23:00:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.843+0000 7faf2d7fa640 1 -- 172.21.15.96:0/3387983072 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28010630 con 0x5577f2fbf870 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- 172.21.15.96:0/3387983072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 msgr2=0x5577f2fc4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc4fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf20031a00 tx=0x7faf280094c0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- 172.21.15.96:0/3387983072 shutdown_connections 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fc4be0 0x5577f2faa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 --2- 172.21.15.96:0/3387983072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fbf870 0x5577f2fc4fc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- 172.21.15.96:0/3387983072 >> 172.21.15.96:0/3387983072 conn(0x5577f2fa99b0 msgr2=0x5577f2fa4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- 172.21.15.96:0/3387983072 shutdown_connections 2024-09-10T23:00:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- 172.21.15.96:0/3387983072 wait complete. 2024-09-10T23:00:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.844+0000 7faf4084c740 1 -- start start 2024-09-10T23:00:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.845+0000 7faf4084c740 1 Processor -- start 2024-09-10T23:00:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fbf870 0x5577f2fcdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf381a8fe0 con 0x5577f2fc21a0 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf381a8910 con 0x5577f2fc4be0 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf3810ea60 con 0x5577f2fbf870 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.846+0000 7faf4084c740 10 client.0 _ll_register_callbacks cb 0x7faf20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fbf870 0x5577f2fcdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:43106/0 (socket says 172.21.15.96:43106) 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3479338635 learned_addr learned my addr 172.21.15.96:0/3479338635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf36ffd640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3479338635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fbf870 msgr2=0x5577f2fcdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fbf870 0x5577f2fcdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3479338635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 msgr2=0x5577f2fce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.847+0000 7faf2e7fc640 1 -- 172.21.15.96:0/3479338635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f2fc4900 con 0x5577f2fc4be0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf2e7fc640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf28009e70 tx=0x7faf28009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdae4dbd862d4108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2802f030 con 0x5577f2fc4be0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf280265e0 con 0x5577f2fc4be0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf28011c50 con 0x5577f2fc4be0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf2803e050 con 0x5577f2fc4be0 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf4084c740 10 client.27827 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577f2fd07a0 con 0x5577f2fc4be0 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.848+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf28011c50 con 0x5577f2fc4be0 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faf28027920 con 0x5577f2fc4be0 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 10 client.27827 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 10 client.27827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 20 client.27827 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f30545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 4 client.27827 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577f2f1e0f0 con 0x5577f2fd4690 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf36ffd640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f30545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf16ffd640 10 client.27827 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.849+0000 7faf36ffd640 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f30545c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5381dfc6190e5d7d server_cookie=7900e5736fae3240 in_seq=0 out_seq=0 2024-09-10T23:00:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.867+0000 7faf16ffd640 1 -- 172.21.15.96:0/3479338635 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577f2f1e0f0 con 0x5577f2fd4690 2024-09-10T23:00:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.867+0000 7faf16ffd640 10 client.27827 handle_command_reply: tid=0 2024-09-10T23:00:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 client.27827 shutdown 2024-09-10T23:00:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 msgr2=0x5577f2fced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf28009e70 tx=0x7faf28009e00 comp rx=0 tx=0).stop 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 shutdown_connections 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5577f2fd4690 0x5577f30545c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577f2fc4be0 0x5577f2fced80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577f2fc21a0 0x5577f2fce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 --2- 172.21.15.96:0/3479338635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577f2fbf870 0x5577f2fcdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 >> 172.21.15.96:0/3479338635 conn(0x5577f2fa99b0 msgr2=0x5577f2fc4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf16ffd640 0 client.27827 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 shutdown_connections 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 1 -- 172.21.15.96:0/3479338635 wait complete. 2024-09-10T23:00:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.868+0000 7faf4084c740 20 client.27827 trim_cache size 0 max 16384 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 msgr2=0x7faf240b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 0x7faf240b7820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf300a8ed0 tx=0x7faf30004990 comp rx=0 tx=0).stop 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 msgr2=0x7faf381a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf381a0630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faf3000ec50 tx=0x7faf3000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 shutdown_connections 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf38101650 0x7faf3819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf240b5360 0x7faf240b7820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf38102210 0x7faf381a0630 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 --2- 172.21.15.96:0/4193972027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf38100c80 0x7faf3819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 >> 172.21.15.96:0/4193972027 conn(0x7faf380fc880 msgr2=0x7faf38102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 shutdown_connections 2024-09-10T23:00:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:08.869+0000 7faf3d9a3640 1 -- 172.21.15.96:0/4193972027 wait complete. 2024-09-10T23:00:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:09 smithi096 ceph-mon[30869]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-09-10T23:00:09.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:09 smithi121 ceph-mon[35119]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-09-10T23:00:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:09 smithi117 ceph-mon[35081]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 688 op/s 2024-09-10T23:00:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 755.0 s, 1563.8 tps, lat 64.140 ms stddev 74.003 2024-09-10T23:00:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:11 smithi096 ceph-mon[30869]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 861 op/s 2024-09-10T23:00:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:11 smithi121 ceph-mon[35119]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 861 op/s 2024-09-10T23:00:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:11 smithi117 ceph-mon[35081]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.9 MiB/s wr, 861 op/s 2024-09-10T23:00:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:13 smithi096 ceph-mon[30869]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 727 op/s 2024-09-10T23:00:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:13 smithi121 ceph-mon[35119]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 727 op/s 2024-09-10T23:00:13.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:13 smithi117 ceph-mon[35081]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 727 op/s 2024-09-10T23:00:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:15 smithi096 ceph-mon[30869]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 743 op/s 2024-09-10T23:00:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:15 smithi121 ceph-mon[35119]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 743 op/s 2024-09-10T23:00:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:15 smithi117 ceph-mon[35081]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 743 op/s 2024-09-10T23:00:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 760.0 s, 1776.8 tps, lat 55.284 ms stddev 60.277 2024-09-10T23:00:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:17 smithi096 ceph-mon[30869]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2024-09-10T23:00:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:17 smithi121 ceph-mon[35119]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2024-09-10T23:00:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:17 smithi117 ceph-mon[35081]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 796 op/s 2024-09-10T23:00:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:19 smithi096 ceph-mon[30869]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 644 op/s 2024-09-10T23:00:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:19 smithi121 ceph-mon[35119]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 644 op/s 2024-09-10T23:00:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:19 smithi117 ceph-mon[35081]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 644 op/s 2024-09-10T23:00:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 765.0 s, 1978.2 tps, lat 51.446 ms stddev 60.790 2024-09-10T23:00:21.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:21 smithi096 ceph-mon[30869]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-09-10T23:00:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:21 smithi121 ceph-mon[35119]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-09-10T23:00:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:21 smithi117 ceph-mon[35081]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 805 op/s 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2703244132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81016a0 msgr2=0x7fb5e8101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2703244132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81016a0 0x7fb5e8101aa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009930 tx=0x7fb5e002a420 comp rx=0 tx=0).stop 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2703244132 shutdown_connections 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2703244132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81027f0 0x7fb5e8102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2703244132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81031b0 0x7fb5e8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2703244132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81016a0 0x7fb5e8101aa0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2703244132 >> 172.21.15.96:0/2703244132 conn(0x7fb5e80fcf60 msgr2=0x7fb5e80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2703244132 shutdown_connections 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.980+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2703244132 wait complete. 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 Processor -- start 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 -- start start 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81016a0 0x7fb5e8105460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 0x7fb5e8103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810d360 con 0x7fb5e81031b0 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810d1e0 con 0x7fb5e81016a0 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed9a1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810d4e0 con 0x7fb5e81027f0 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54898/0 (socket says 172.21.15.96:54898) 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 0x7fb5e8103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ed1a0640 1 -- 172.21.15.96:0/2142357828 learned_addr learned my addr 172.21.15.96:0/2142357828 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.981+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81016a0 0x7fb5e8105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 -- 172.21.15.96:0/2142357828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81016a0 msgr2=0x7fb5e8105460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81016a0 0x7fb5e8105460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 -- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 msgr2=0x7fb5e8104200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 -- 172.21.15.96:0/2142357828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e8104a70 con 0x7fb5e81027f0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 0x7fb5e8103cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000d6d0 tx=0x7fb5d000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e5ffb640 1 -- 172.21.15.96:0/2142357828 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0017070 con 0x7fb5e81027f0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81a2cc0 con 0x7fb5e81027f0 2024-09-10T23:00:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e5ffb640 1 -- 172.21.15.96:0/2142357828 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d0009820 con 0x7fb5e81027f0 2024-09-10T23:00:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5e5ffb640 1 -- 172.21.15.96:0/2142357828 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0004c40 con 0x7fb5e81027f0 2024-09-10T23:00:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.982+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a31d0 con 0x7fb5e81027f0 2024-09-10T23:00:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 Processor -- start 2024-09-10T23:00:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 -- start start 2024-09-10T23:00:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e626270 0x55789e628660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e62daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55789e58f6f0 con 0x55789e628ba0 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55789e57eb20 con 0x55789e62b5e0 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.986+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55789e582930 con 0x55789e626270 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e626270 0x55789e628660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54914/0 (socket says 172.21.15.96:54914) 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 -- 172.21.15.96:0/2049123841 learned_addr learned my addr 172.21.15.96:0/2049123841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5e5ffb640 1 -- 172.21.15.96:0/2142357828 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb5d0005220 con 0x7fb5e81027f0 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 -- 172.21.15.96:0/2049123841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e626270 msgr2=0x55789e628660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e626270 0x55789e628660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 -- 172.21.15.96:0/2049123841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 msgr2=0x55789e62daa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:00:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e62daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 -- 172.21.15.96:0/2049123841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789e544530 con 0x55789e628ba0 2024-09-10T23:00:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be030f22fda41936 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5deffd640 1 -- 172.21.15.96:0/2049123841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0035030 con 0x55789e628ba0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5deffd640 1 -- 172.21.15.96:0/2049123841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0040070 con 0x55789e628ba0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.987+0000 7fb5deffd640 1 -- 172.21.15.96:0/2049123841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00318c0 con 0x55789e628ba0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5e5ffb640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 0x7fb5d40b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 0x7fb5d40b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5e5ffb640 1 -- 172.21.15.96:0/2142357828 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb5d00049e0 con 0x7fb5e81027f0 2024-09-10T23:00:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 -- 172.21.15.96:0/2049123841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 msgr2=0x55789e62af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e0004170 comp rx=0 tx=0).stop 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 -- 172.21.15.96:0/2049123841 shutdown_connections 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e62daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e628ba0 0x55789e62af90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/2049123841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e626270 0x55789e628660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 -- 172.21.15.96:0/2049123841 >> 172.21.15.96:0/2049123841 conn(0x55789e6213c0 msgr2=0x55789e622150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 0x7fb5d40b7a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80049d0 tx=0x7fb5d8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 -- 172.21.15.96:0/2049123841 shutdown_connections 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.988+0000 7fb5f19d5740 1 -- 172.21.15.96:0/2049123841 wait complete. 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 -- start start 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 Processor -- start 2024-09-10T23:00:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e62f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 0x55789e62ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 0x55789e630b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55789e4b6ab0 con 0x55789e626270 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55789e524730 con 0x55789e628ba0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55789e62e560 con 0x55789e62b5e0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5f19d5740 10 client.0 _ll_register_callbacks cb 0x55789e5786b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e62f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 0x55789e630b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e62f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54922/0 (socket says 172.21.15.96:54922) 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 0x55789e630b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:42516/0 (socket says 172.21.15.96:42516) 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5ec99f640 1 -- 172.21.15.96:0/1685067055 learned_addr learned my addr 172.21.15.96:0/1685067055 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.989+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 0x55789e62ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.990+0000 7fb5e7fff640 1 -- 172.21.15.96:0/1685067055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 msgr2=0x55789e630b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.990+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 0x55789e630b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.990+0000 7fb5e7fff640 1 -- 172.21.15.96:0/1685067055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 msgr2=0x55789e62f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.990+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e62f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.990+0000 7fb5e7fff640 1 -- 172.21.15.96:0/1685067055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789e2e0790 con 0x55789e628ba0 2024-09-10T23:00:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.996+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 0x55789e62ff80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00a8f70 tx=0x7fb5d0009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5454547787cf802 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:21.997+0000 7fb5c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:00:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.001+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0020070 con 0x55789e628ba0 2024-09-10T23:00:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.001+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb5d00099d0 con 0x55789e628ba0 2024-09-10T23:00:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.001+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0040070 con 0x55789e628ba0 2024-09-10T23:00:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.008+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5d00abd00 con 0x55789e628ba0 2024-09-10T23:00:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.009+0000 7fb5f19d5740 10 client.27836 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.009+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55789e359620 con 0x55789e628ba0 2024-09-10T23:00:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.015+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0040070 con 0x55789e628ba0 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.015+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb5d0091b20 con 0x55789e628ba0 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 10 client.27836 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 10 client.27836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 20 client.27836 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e63b1a0 0x55789e63d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 4 client.27836 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.016+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55789e591640 con 0x55789e63b1a0 2024-09-10T23:00:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.021+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e63b1a0 0x55789e63d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.030+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e63b1a0 0x55789e63d5b0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55cfe4b0b0200cc9 server_cookie=8eb617c8eed11908 in_seq=0 out_seq=0 2024-09-10T23:00:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.031+0000 7fb5c7fff640 10 client.27836 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.034+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1685067055 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55789e591640 con 0x55789e63b1a0 2024-09-10T23:00:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.034+0000 7fb5c7fff640 10 client.27836 handle_command_reply: tid=0 2024-09-10T23:00:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.034+0000 7fb5f19d5740 1 client.27836 shutdown 2024-09-10T23:00:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.038+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 msgr2=0x55789e62ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.038+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 0x55789e62ff80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00a8f70 tx=0x7fb5d0009f00 comp rx=0 tx=0).stop 2024-09-10T23:00:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 shutdown_connections 2024-09-10T23:00:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e63b1a0 0x55789e63d5b0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e628ba0 0x55789e62ff80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e62b5e0 0x55789e630b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1685067055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e62f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 >> 172.21.15.96:0/1685067055 conn(0x55789e6213c0 msgr2=0x55789e617c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.041+0000 7fb5c7fff640 0 client.27836 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.047+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 shutdown_connections 2024-09-10T23:00:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.052+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1685067055 wait complete. 2024-09-10T23:00:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.055+0000 7fb5f19d5740 20 client.27836 trim_cache size 0 max 16384 2024-09-10T23:00:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.085+0000 7fb5f19d5740 1 Processor -- start 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.085+0000 7fb5f19d5740 1 -- start start 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e626270 0x55789e62ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e637b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a7ae0 con 0x55789e62b5e0 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a9620 con 0x55789e626270 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5f19d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a81b0 con 0x55789e628ba0 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54930/0 (socket says 172.21.15.96:54930) 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 -- 172.21.15.96:0/3226164980 learned_addr learned my addr 172.21.15.96:0/3226164980 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e637b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e626270 0x55789e62ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 -- 172.21.15.96:0/3226164980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 msgr2=0x55789e637b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e637b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 -- 172.21.15.96:0/3226164980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e626270 msgr2=0x55789e62ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e626270 0x55789e62ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.086+0000 7fb5ed1a0640 1 -- 172.21.15.96:0/3226164980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789e63dc30 con 0x55789e62b5e0 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.087+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8012540 tx=0x7fb5d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b12a43a1d76e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.087+0000 7fb5de7fc640 1 -- 172.21.15.96:0/3226164980 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8004b20 con 0x55789e62b5e0 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.087+0000 7fb5de7fc640 1 -- 172.21.15.96:0/3226164980 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d8004cc0 con 0x55789e62b5e0 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.087+0000 7fb5de7fc640 1 -- 172.21.15.96:0/3226164980 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8005820 con 0x55789e62b5e0 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- 172.21.15.96:0/3226164980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 msgr2=0x55789e62bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8012540 tx=0x7fb5d801a000 comp rx=0 tx=0).stop 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- 172.21.15.96:0/3226164980 shutdown_connections 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e62b5e0 0x55789e62bf90 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e637b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/3226164980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e626270 0x55789e62ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- 172.21.15.96:0/3226164980 >> 172.21.15.96:0/3226164980 conn(0x55789e6213c0 msgr2=0x55789e621e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- 172.21.15.96:0/3226164980 shutdown_connections 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- 172.21.15.96:0/3226164980 wait complete. 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 -- start start 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.088+0000 7fb5f19d5740 1 Processor -- start 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:42532/0 (socket says 172.21.15.96:42532) 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5ec99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54942/0 (socket says 172.21.15.96:54942) 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5ed1a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:51756/0 (socket says 172.21.15.96:51756) 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5e7fff640 1 -- 172.21.15.96:0/1789182657 learned_addr learned my addr 172.21.15.96:0/1789182657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a8f50 con 0x55789e626270 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a8880 con 0x55789e62b5e0 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.089+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81a5b00 con 0x55789e628ba0 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5f19d5740 10 client.0 _ll_register_callbacks cb 0x7fb5d00abad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 -- 172.21.15.96:0/1789182657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 msgr2=0x55789e636560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 -- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 msgr2=0x55789e636aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789e62e930 con 0x55789e626270 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ed1a0640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d8004cc0 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5ec99f640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x55789e63de30 tx=0x7fb5e0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6272c4372fcc22b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5c7fff640 10 client.18243 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1789182657 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0035040 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1789182657 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb5e00037f0 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1789182657 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5e004a040 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5f19d5740 10 client.18243 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.090+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55789e635e00 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1789182657 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb5e0055030 con 0x55789e626270 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 10 client.18243 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 10 client.18243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 20 client.18243 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e6a82d0 0x55789e6aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 4 client.18243 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55789e58d0f0 con 0x55789e6a82d0 2024-09-10T23:00:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.091+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e6a82d0 0x55789e6aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.092+0000 7fb5c7fff640 10 client.18243 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.092+0000 7fb5e7fff640 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e6a82d0 0x55789e6aa6c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60847a48b0e01db4 server_cookie=af5c6d39fc9f01d5 in_seq=0 out_seq=0 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5c7fff640 1 -- 172.21.15.96:0/1789182657 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55789e58d0f0 con 0x55789e6a82d0 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5c7fff640 10 client.18243 handle_command_reply: tid=0 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 client.18243 shutdown 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 msgr2=0x55789e637860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x55789e63de30 tx=0x7fb5e0005e70 comp rx=0 tx=0).stop 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 shutdown_connections 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55789e6a82d0 0x55789e6aa6c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55789e62b5e0 0x55789e636aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789e628ba0 0x55789e636560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5c7fff640 0 client.18243 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 --2- 172.21.15.96:0/1789182657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55789e626270 0x55789e637860 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 >> 172.21.15.96:0/1789182657 conn(0x55789e6213c0 msgr2=0x55789e625360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 shutdown_connections 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 1 -- 172.21.15.96:0/1789182657 wait complete. 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.106+0000 7fb5f19d5740 20 client.18243 trim_cache size 0 max 16384 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:00:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 msgr2=0x7fb5d40b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 0x7fb5d40b7a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80049d0 tx=0x7fb5d8009500 comp rx=0 tx=0).stop 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 msgr2=0x7fb5e8103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 0x7fb5e8103cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000d6d0 tx=0x7fb5d000db90 comp rx=0 tx=0).stop 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 shutdown_connections 2024-09-10T23:00:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb5d40b5560 0x7fb5d40b7a20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e81031b0 0x7fb5e8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5e81027f0 0x7fb5e8103cc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 --2- 172.21.15.96:0/2142357828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5e81016a0 0x7fb5e8105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 >> 172.21.15.96:0/2142357828 conn(0x7fb5e80fcf60 msgr2=0x7fb5e80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 shutdown_connections 2024-09-10T23:00:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:22.107+0000 7fb5ed9a1640 1 -- 172.21.15.96:0/2142357828 wait complete. 2024-09-10T23:00:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:23 smithi096 ceph-mon[30869]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 674 op/s 2024-09-10T23:00:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:23 smithi121 ceph-mon[35119]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 674 op/s 2024-09-10T23:00:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:23 smithi117 ceph-mon[35081]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 674 op/s 2024-09-10T23:00:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:25 smithi121 ceph-mon[35119]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-10T23:00:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:25 smithi117 ceph-mon[35081]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-10T23:00:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 770.0 s, 1706.0 tps, lat 58.628 ms stddev 62.799 2024-09-10T23:00:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:25 smithi096 ceph-mon[30869]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 675 op/s 2024-09-10T23:00:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:27 smithi121 ceph-mon[35119]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 722 op/s 2024-09-10T23:00:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:27 smithi117 ceph-mon[35081]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 722 op/s 2024-09-10T23:00:28.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:27 smithi096 ceph-mon[30869]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 722 op/s 2024-09-10T23:00:29.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:00:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:29 smithi121 ceph-mon[35119]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-10T23:00:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:29 smithi117 ceph-mon[35081]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-10T23:00:30.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:29 smithi096 ceph-mon[30869]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 582 op/s 2024-09-10T23:00:30.012 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:30.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 -- 172.21.15.96:0/3040611172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0102dc0 msgr2=0x7fb9e01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 --2- 172.21.15.96:0/3040611172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0102dc0 0x7fb9e01031a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009900 tx=0x7fb9d802a410 comp rx=0 tx=0).stop 2024-09-10T23:00:30.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 -- 172.21.15.96:0/3040611172 shutdown_connections 2024-09-10T23:00:30.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 --2- 172.21.15.96:0/3040611172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e0109150 0x7fb9e010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 --2- 172.21.15.96:0/3040611172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e01087d0 0x7fb9e0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 --2- 172.21.15.96:0/3040611172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0102dc0 0x7fb9e01031a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 -- 172.21.15.96:0/3040611172 >> 172.21.15.96:0/3040611172 conn(0x7fb9e00fe830 msgr2=0x7fb9e0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 -- 172.21.15.96:0/3040611172 shutdown_connections 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.586+0000 7fb9e738b640 1 -- 172.21.15.96:0/3040611172 wait complete. 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 Processor -- start 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 -- start start 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e01087d0 0x7fb9e0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0109150 0x7fb9e019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112e60 con 0x7fb9e0102dc0 2024-09-10T23:00:30.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112ce0 con 0x7fb9e01087d0 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.587+0000 7fb9e738b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112fe0 con 0x7fb9e0109150 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34938/0 (socket says 172.21.15.96:34938) 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 -- 172.21.15.96:0/4246045587 learned_addr learned my addr 172.21.15.96:0/4246045587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e4926640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e01087d0 0x7fb9e0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5928640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0109150 0x7fb9e019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 -- 172.21.15.96:0/4246045587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0109150 msgr2=0x7fb9e019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0109150 0x7fb9e019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 -- 172.21.15.96:0/4246045587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e01087d0 msgr2=0x7fb9e0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e01087d0 0x7fb9e0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 -- 172.21.15.96:0/4246045587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e019f300 con 0x7fb9e0102dc0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.588+0000 7fb9e5127640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8005ab0 tx=0x7fb9d8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.589+0000 7fb9de7fc640 1 -- 172.21.15.96:0/4246045587 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d803f070 con 0x7fb9e0102dc0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.589+0000 7fb9de7fc640 1 -- 172.21.15.96:0/4246045587 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d8031d60 con 0x7fb9e0102dc0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.589+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e019f590 con 0x7fb9e0102dc0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.589+0000 7fb9de7fc640 1 -- 172.21.15.96:0/4246045587 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8035030 con 0x7fb9e0102dc0 2024-09-10T23:00:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.589+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e0075380 con 0x7fb9e0102dc0 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9de7fc640 1 -- 172.21.15.96:0/4246045587 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb9d803b460 con 0x7fb9e0102dc0 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 Processor -- start 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 -- start start 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7526480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75269c0 0x5616f7532410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616f749ce50 con 0x5616f75269c0 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5616f748c280 con 0x5616f7532950 2024-09-10T23:00:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e9760740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5616f7490090 con 0x5616f75260a0 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e4926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e4926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:59984/0 (socket says 172.21.15.96:59984) 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.592+0000 7fb9e4926640 1 -- 172.21.15.96:0/1551206597 learned_addr learned my addr 172.21.15.96:0/1551206597 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e5928640 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7526480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e4926640 1 -- 172.21.15.96:0/1551206597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 msgr2=0x5616f7526480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e4926640 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7526480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e4926640 1 -- 172.21.15.96:0/1551206597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75269c0 msgr2=0x5616f7532410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e4926640 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75269c0 0x5616f7532410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.593+0000 7fb9e4926640 1 -- 172.21.15.96:0/1551206597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f71ea650 con 0x5616f7532950 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.594+0000 7fb9de7fc640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 0x7fb9c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.594+0000 7fb9e5928640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 0x7fb9c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.594+0000 7fb9de7fc640 1 -- 172.21.15.96:0/4246045587 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb9d81067f0 con 0x7fb9e0102dc0 2024-09-10T23:00:30.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.594+0000 7fb9e4926640 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400aca0 tx=0x7fb9d400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67bd7ef574c0061f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9c77fe640 1 -- 172.21.15.96:0/1551206597 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d401d070 con 0x5616f7532950 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9c77fe640 1 -- 172.21.15.96:0/1551206597 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d4018af0 con 0x5616f7532950 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9c77fe640 1 -- 172.21.15.96:0/1551206597 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4010bf0 con 0x5616f7532950 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 -- 172.21.15.96:0/1551206597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 msgr2=0x5616f7534d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e5928640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 0x7fb9c80b77a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400aca0 tx=0x7fb9d400f790 comp rx=0 tx=0).stop 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 -- 172.21.15.96:0/1551206597 shutdown_connections 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f7532950 0x5616f7534d40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75269c0 0x5616f7532410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1551206597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7526480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.595+0000 7fb9e9760740 1 -- 172.21.15.96:0/1551206597 >> 172.21.15.96:0/1551206597 conn(0x5616f7523ed0 msgr2=0x5616f7525070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.596+0000 7fb9e9760740 1 -- 172.21.15.96:0/1551206597 shutdown_connections 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.596+0000 7fb9e9760740 1 -- 172.21.15.96:0/1551206597 wait complete. 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.596+0000 7fb9e9760740 1 -- start start 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.596+0000 7fb9e9760740 1 Processor -- start 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f752f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f752fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616f72ed700 con 0x5616f75260a0 2024-09-10T23:00:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5616f7427110 con 0x5616f75269c0 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5616f75355d0 con 0x5616f7532950 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e9760740 10 client.0 _ll_register_callbacks cb 0x5616f7440990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34952/0 (socket says 172.21.15.96:34952) 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 -- 172.21.15.96:0/2685278194 learned_addr learned my addr 172.21.15.96:0/2685278194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5928640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f752fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f752f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 -- 172.21.15.96:0/2685278194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 msgr2=0x5616f752fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f752fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 -- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 msgr2=0x5616f752f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f752f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.597+0000 7fb9e5127640 1 -- 172.21.15.96:0/2685278194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f7454c40 con 0x5616f75260a0 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9e5127640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ca220 tx=0x7fb9d80c8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e851d001051a63ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8049070 con 0x5616f75260a0 2024-09-10T23:00:30.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d4018af0 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb9d8106de0 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9d80ccac0 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9e9760740 10 client.18249 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5616f7540fe0 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.598+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d4018af0 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb9d80cd210 con 0x5616f75260a0 2024-09-10T23:00:30.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 10 client.18249 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 20 client.18249 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75425d0 0x5616f75449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 4 client.18249 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f749eda0 con 0x5616f75425d0 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.599+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75425d0 0x5616f75449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.600+0000 7fb9c4ff9640 10 client.18249 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:30.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.600+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75425d0 0x5616f75449c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab163b87d4e99dd server_cookie=5a91366c7cd4a45c in_seq=0 out_seq=0 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.600+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2685278194 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616f749eda0 con 0x5616f75425d0 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.600+0000 7fb9c4ff9640 10 client.18249 handle_command_reply: tid=0 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.600+0000 7fb9e9760740 1 client.18249 shutdown 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 msgr2=0x5616f7532310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ca220 tx=0x7fb9d80c8f10 comp rx=0 tx=0).stop 2024-09-10T23:00:30.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 shutdown_connections 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75425d0 0x5616f75449c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f752f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f752fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2685278194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f7532310 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 >> 172.21.15.96:0/2685278194 conn(0x5616f7523ed0 msgr2=0x5616f7529a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9c4ff9640 0 client.18249 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:30.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.601+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 shutdown_connections 2024-09-10T23:00:30.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.602+0000 7fb9e9760740 1 -- 172.21.15.96:0/2685278194 wait complete. 2024-09-10T23:00:30.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.602+0000 7fb9e9760740 20 client.18249 trim_cache size 0 max 16384 2024-09-10T23:00:30.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.622+0000 7fb9e9760740 1 Processor -- start 2024-09-10T23:00:30.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 -- start start 2024-09-10T23:00:30.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7523710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f75314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b03a0 con 0x5616f7532950 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b1ee0 con 0x5616f75269c0 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e9760740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b0a70 con 0x5616f75260a0 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34964/0 (socket says 172.21.15.96:34964) 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 -- 172.21.15.96:0/1393734923 learned_addr learned my addr 172.21.15.96:0/1393734923 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e4926640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f75314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5127640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7523710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 -- 172.21.15.96:0/1393734923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 msgr2=0x5616f7523710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7523710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 -- 172.21.15.96:0/1393734923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 msgr2=0x5616f75314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f75314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 -- 172.21.15.96:0/1393734923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f75267f0 con 0x5616f7532950 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e4926640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f75314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.623+0000 7fb9e5928640 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0012940 tx=0x7fb9d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcd13f9ae833216 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.624+0000 7fb9c6ffd640 1 -- 172.21.15.96:0/1393734923 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0020bb0 con 0x5616f7532950 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.624+0000 7fb9c6ffd640 1 -- 172.21.15.96:0/1393734923 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0020d50 con 0x5616f7532950 2024-09-10T23:00:30.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.624+0000 7fb9c6ffd640 1 -- 172.21.15.96:0/1393734923 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d001b640 con 0x5616f7532950 2024-09-10T23:00:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- 172.21.15.96:0/1393734923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 msgr2=0x5616f7533d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0012940 tx=0x7fb9d0014000 comp rx=0 tx=0).stop 2024-09-10T23:00:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- 172.21.15.96:0/1393734923 shutdown_connections 2024-09-10T23:00:30.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f75314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f7532950 0x5616f7533d40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 --2- 172.21.15.96:0/1393734923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f75260a0 0x5616f7523710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- 172.21.15.96:0/1393734923 >> 172.21.15.96:0/1393734923 conn(0x5616f7523ed0 msgr2=0x5616f7530d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- 172.21.15.96:0/1393734923 shutdown_connections 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- 172.21.15.96:0/1393734923 wait complete. 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 -- start start 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.625+0000 7fb9e9760740 1 Processor -- start 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f753a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f753adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f7535820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b1810 con 0x5616f75260a0 2024-09-10T23:00:30.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b1140 con 0x5616f75269c0 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01ae380 con 0x5616f7532950 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f753a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f753adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e9760740 10 client.0 _ll_register_callbacks cb 0x7fb9d803b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e5127640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f753a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34970/0 (socket says 172.21.15.96:34970) 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e5127640 1 -- 172.21.15.96:0/2178091511 learned_addr learned my addr 172.21.15.96:0/2178091511 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e5928640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f7535820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 -- 172.21.15.96:0/2178091511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 msgr2=0x5616f7535820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f7535820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 -- 172.21.15.96:0/2178091511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 msgr2=0x5616f753a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:30.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f753a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.626+0000 7fb9e4926640 1 -- 172.21.15.96:0/2178091511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f753b5e0 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e4926640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f753adb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400a580 tx=0x7fb9d400a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae78f60454de2af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4030060 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d0020d50 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb9d4010cd0 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9d401d040 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 10 client.27860 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5616f75afa50 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d0020d50 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb9d4035070 con 0x5616f75269c0 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 10 client.27860 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:30.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 10 client.27860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 20 client.27860 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75afec0 0x5616f75b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 4 client.27860 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.627+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616f749a850 con 0x5616f75afec0 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e5127640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75afec0 0x5616f75b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9c4ff9640 10 client.27860 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e5127640 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75afec0 0x5616f75b22e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3eb482807fb651 server_cookie=b315c060e5be03ed in_seq=0 out_seq=0 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9c4ff9640 1 -- 172.21.15.96:0/2178091511 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x5616f749a850 con 0x5616f75afec0 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9c4ff9640 10 client.27860 handle_command_reply: tid=0 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 client.27860 shutdown 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 msgr2=0x5616f753adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f753adb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400a580 tx=0x7fb9d400a510 comp rx=0 tx=0).stop 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 shutdown_connections 2024-09-10T23:00:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5616f75afec0 0x5616f75b22e0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616f75269c0 0x5616f753adb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.628+0000 7fb9c4ff9640 0 client.27860 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7532950 0x5616f7535820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 1 --2- 172.21.15.96:0/2178091511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f75260a0 0x5616f753a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 >> 172.21.15.96:0/2178091511 conn(0x5616f7523ed0 msgr2=0x5616f75315b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 shutdown_connections 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 1 -- 172.21.15.96:0/2178091511 wait complete. 2024-09-10T23:00:30.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.629+0000 7fb9e9760740 20 client.27860 trim_cache size 0 max 16384 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.642 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.643 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.644 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.645 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.646 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.647 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.648 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:00:30.649 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:00:30.650 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 msgr2=0x7fb9c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 0x7fb9c80b77a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009580 comp rx=0 tx=0).stop 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 msgr2=0x7fb9e0199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8005ab0 tx=0x7fb9d8003cd0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 shutdown_connections 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9e01087d0 0x7fb9e0199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb9c80b52e0 0x7fb9c80b77a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9e0109150 0x7fb9e019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 --2- 172.21.15.96:0/4246045587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9e0102dc0 0x7fb9e0199250 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:30.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 >> 172.21.15.96:0/4246045587 conn(0x7fb9e00fe830 msgr2=0x7fb9e01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:30.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.630+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 shutdown_connections 2024-09-10T23:00:30.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:30.631+0000 7fb9e738b640 1 -- 172.21.15.96:0/4246045587 wait complete. 2024-09-10T23:00:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 775.0 s, 2133.4 tps, lat 46.656 ms stddev 57.614 2024-09-10T23:00:31.436 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:00:31.436 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:00:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:31 smithi096 ceph-mon[30869]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 762 op/s 2024-09-10T23:00:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:31 smithi121 ceph-mon[35119]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 762 op/s 2024-09-10T23:00:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:31 smithi117 ceph-mon[35081]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 762 op/s 2024-09-10T23:00:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:32.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.436+0000 7f0d9ddea640 1 -- 172.21.15.96:0/680532873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9810a530 msgr2=0x7f0d9810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.436+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/680532873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9810a530 0x7f0d9810a930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009900 tx=0x7f0d8802a4d0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.442+0000 7f0d9ddea640 1 -- 172.21.15.96:0/680532873 shutdown_connections 2024-09-10T23:00:32.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.442+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/680532873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9810a530 0x7f0d9810a930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.442+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/680532873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9811c0a0 0x7f0d9811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.442+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/680532873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d981111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.442+0000 7f0d9ddea640 1 -- 172.21.15.96:0/680532873 >> 172.21.15.96:0/680532873 conn(0x7f0d9806dfc0 msgr2=0x7f0d9806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.446+0000 7f0d9ddea640 1 -- 172.21.15.96:0/680532873 shutdown_connections 2024-09-10T23:00:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.446+0000 7f0d9ddea640 1 -- 172.21.15.96:0/680532873 wait complete. 2024-09-10T23:00:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.446+0000 7f0d9ddea640 1 Processor -- start 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.447+0000 7f0d9ddea640 1 -- start start 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9810a530 0x7f0d9811bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9811c0a0 0x7f0d98115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98112180 con 0x7f0d9810a530 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98112000 con 0x7f0d9811c0a0 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d9ddea640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98112300 con 0x7f0d9810a150 2024-09-10T23:00:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35238/0 (socket says 172.21.15.96:35238) 2024-09-10T23:00:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.448+0000 7f0d977fe640 1 -- 172.21.15.96:0/2697810663 learned_addr learned my addr 172.21.15.96:0/2697810663 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.449+0000 7f0d96ffd640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9810a530 0x7f0d9811bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.449+0000 7f0d97fff640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9811c0a0 0x7f0d98115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 -- 172.21.15.96:0/2697810663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9811c0a0 msgr2=0x7f0d98115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9811c0a0 0x7f0d98115a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 -- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9810a530 msgr2=0x7f0d9811bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9810a530 0x7f0d9811bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 -- 172.21.15.96:0/2697810663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98116300 con 0x7f0d9810a150 2024-09-10T23:00:32.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.450+0000 7f0d977fe640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000ebc0 tx=0x7f0d9000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.451+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90010630 con 0x7f0d9810a150 2024-09-10T23:00:32.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.451+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d981165f0 con 0x7f0d9810a150 2024-09-10T23:00:32.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.451+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d981bd3f0 con 0x7f0d9810a150 2024-09-10T23:00:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.451+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d90010c80 con 0x7f0d9810a150 2024-09-10T23:00:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.452+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90018b60 con 0x7f0d9810a150 2024-09-10T23:00:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.453+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84005290 con 0x7f0d9810a150 2024-09-10T23:00:32.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.459+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0d900107d0 con 0x7f0d9810a150 2024-09-10T23:00:32.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.460+0000 7f0d94ff9640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 0x7f0d800b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:32.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.460+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0d900e65f0 con 0x7f0d9810a150 2024-09-10T23:00:32.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.462+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0d981165f0 con 0x7f0d9810a150 2024-09-10T23:00:32.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.463+0000 7f0d96ffd640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 0x7f0d800b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.466+0000 7f0d96ffd640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 0x7f0d800b77c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005ab0 tx=0x7f0d88000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:32.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.817+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d84005580 con 0x7f0d9810a150 2024-09-10T23:00:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.818+0000 7f0d94ff9640 1 -- 172.21.15.96:0/2697810663 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f0d900ad4b0 con 0x7f0d9810a150 2024-09-10T23:00:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:00:32.819 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:00:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:00:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.829+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 msgr2=0x7f0d800b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.829+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 0x7f0d800b77c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005ab0 tx=0x7f0d88000bc0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.829+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 msgr2=0x7f0d9811b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.829+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000ebc0 tx=0x7f0d9000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 shutdown_connections 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d9811c0a0 0x7f0d98115a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0d800b5300 0x7f0d800b77c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d9810a530 0x7f0d9811bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 --2- 172.21.15.96:0/2697810663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d9810a150 0x7f0d9811b5a0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000ebc0 tx=0x7f0d9000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:00:32.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.830+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 >> 172.21.15.96:0/2697810663 conn(0x7f0d9806dfc0 msgr2=0x7f0d98071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:32.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.834+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 shutdown_connections 2024-09-10T23:00:32.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:32.836+0000 7f0d9ddea640 1 -- 172.21.15.96:0/2697810663 wait complete. 2024-09-10T23:00:33.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T23:00:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:33 smithi117 ceph-mon[35081]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-10T23:00:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:33 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2697810663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:34.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:33 smithi096 ceph-mon[30869]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-10T23:00:34.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:33 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2697810663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:33 smithi121 ceph-mon[35119]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-10T23:00:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:33 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2697810663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:00:34.049 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:34.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 -- 172.21.15.96:0/2103255756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc4104550 msgr2=0x7f0fc410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 --2- 172.21.15.96:0/2103255756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc4104550 0x7f0fc410cb30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009900 tx=0x7f0fb002a410 comp rx=0 tx=0).stop 2024-09-10T23:00:34.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 -- 172.21.15.96:0/2103255756 shutdown_connections 2024-09-10T23:00:34.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 --2- 172.21.15.96:0/2103255756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc410d100 0x7f0fc410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 --2- 172.21.15.96:0/2103255756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc4104550 0x7f0fc410cb30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 --2- 172.21.15.96:0/2103255756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4103b50 0x7f0fc4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 -- 172.21.15.96:0/2103255756 >> 172.21.15.96:0/2103255756 conn(0x7f0fc40fd700 msgr2=0x7f0fc40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 -- 172.21.15.96:0/2103255756 shutdown_connections 2024-09-10T23:00:34.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.620+0000 7f0fcb193640 1 -- 172.21.15.96:0/2103255756 wait complete. 2024-09-10T23:00:34.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 Processor -- start 2024-09-10T23:00:34.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 -- start start 2024-09-10T23:00:34.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 0x7f0fc41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 0x7f0fc4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4112d80 con 0x7f0fc410d100 2024-09-10T23:00:34.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4112c00 con 0x7f0fc4103b50 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fcb193640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4112f00 con 0x7f0fc4104550 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fc9730640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 0x7f0fc41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fc9730640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35004/0 (socket says 172.21.15.96:35004) 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fc9730640 1 -- 172.21.15.96:0/3218562665 learned_addr learned my addr 172.21.15.96:0/3218562665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.621+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 0x7f0fc4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/3218562665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 msgr2=0x7f0fc4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 0x7f0fc4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 msgr2=0x7f0fc419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/3218562665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc419f3a0 con 0x7f0fc4103b50 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc9730640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 0x7f0fc4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 0x7f0fc41992f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800e9f0 tx=0x7f0fb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc1ffb640 1 -- 172.21.15.96:0/3218562665 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb800ccc0 con 0x7f0fc4103b50 2024-09-10T23:00:34.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc419f690 con 0x7f0fc4103b50 2024-09-10T23:00:34.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc1ffb640 1 -- 172.21.15.96:0/3218562665 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb8005020 con 0x7f0fc4103b50 2024-09-10T23:00:34.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fc1ffb640 1 -- 172.21.15.96:0/3218562665 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004a00 con 0x7f0fc4103b50 2024-09-10T23:00:34.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.622+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc41a5d50 con 0x7f0fc4103b50 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.625+0000 7f0fc1ffb640 1 -- 172.21.15.96:0/3218562665 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0fb8004ba0 con 0x7f0fc4103b50 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.627+0000 7f0fc1ffb640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 0x7f0fb40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.627+0000 7f0fcd568740 1 Processor -- start 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.627+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 0x7f0fb40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.627+0000 7f0fcd568740 1 -- start start 2024-09-10T23:00:34.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc1ffb640 1 -- 172.21.15.96:0/3218562665 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0fb80e5b80 con 0x7f0fc4103b50 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5596a0545ea0 con 0x5596a05cf4d0 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596a05352d0 con 0x5596a05dbd80 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fcd568740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5596a05390e0 con 0x5596a05cfdf0 2024-09-10T23:00:34.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc9730640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc9730640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:60036/0 (socket says 172.21.15.96:60036) 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc9730640 1 -- 172.21.15.96:0/3877996592 learned_addr learned my addr 172.21.15.96:0/3877996592 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc9730640 1 -- 172.21.15.96:0/3877996592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 msgr2=0x5596a05db840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.628+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc9730640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05db840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc9730640 1 -- 172.21.15.96:0/3877996592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 msgr2=0x5596a05cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc9730640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc9730640 1 -- 172.21.15.96:0/3877996592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a0293650 con 0x5596a05dbd80 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc9730640 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc010810 tx=0x7f0fbc010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94ec8f6019f5817 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 0x7f0fb40b7820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0031440 tx=0x7f0fb00316c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0faeffd640 1 -- 172.21.15.96:0/3877996592 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00eb30 con 0x5596a05dbd80 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0faeffd640 1 -- 172.21.15.96:0/3877996592 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fbc015070 con 0x5596a05dbd80 2024-09-10T23:00:34.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.629+0000 7f0faeffd640 1 -- 172.21.15.96:0/3877996592 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc011440 con 0x5596a05dbd80 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 -- 172.21.15.96:0/3877996592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 msgr2=0x5596a05de170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc010810 tx=0x7f0fbc010c10 comp rx=0 tx=0).stop 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 -- 172.21.15.96:0/3877996592 shutdown_connections 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05de170 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05db840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 --2- 172.21.15.96:0/3877996592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 -- 172.21.15.96:0/3877996592 >> 172.21.15.96:0/3877996592 conn(0x5596a05cd300 msgr2=0x5596a05ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 -- 172.21.15.96:0/3877996592 shutdown_connections 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.630+0000 7f0fcd568740 1 -- 172.21.15.96:0/3877996592 wait complete. 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fcd568740 1 -- start start 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fcd568740 1 Processor -- start 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.631+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fcd568740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5596a0396700 con 0x5596a05cf4d0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35022/0 (socket says 172.21.15.96:35022) 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/2271484231 learned_addr learned my addr 172.21.15.96:0/2271484231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fcd568740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596a04d0160 con 0x5596a05dbd80 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5596a05dea00 con 0x5596a05cfdf0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc9730640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fcd568740 10 client.0 _ll_register_callbacks cb 0x5596a04e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 -- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 msgr2=0x5596a05d9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05d9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 -- 172.21.15.96:0/2271484231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 msgr2=0x5596a05db630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 -- 172.21.15.96:0/2271484231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a04fdc90 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05d8a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009230 tx=0x7f0fb002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9eafc78fd22b877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb004d020 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.632+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0fb0005450 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.633+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fbc015070 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.633+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0fb0035040 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.633+0000 7f0fcd568740 10 client.27946 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5596a05ea270 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fbc015070 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0fb0048070 con 0x5596a05cfdf0 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 10 client.27946 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:34.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 10 client.27946 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 20 client.27946 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a05eb680 0x5596a05edb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 4 client.27946 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596a0547df0 con 0x5596a05eb680 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.634+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a05eb680 0x5596a05edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a05eb680 0x5596a05edb00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a2be89f88bb391a server_cookie=bbe9d32478a583fe in_seq=0 out_seq=0 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fa3fff640 10 client.27946 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fa3fff640 1 -- 172.21.15.96:0/2271484231 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596a0547df0 con 0x5596a05eb680 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fa3fff640 10 client.27946 handle_command_reply: tid=0 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fcd568740 1 client.27946 shutdown 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 msgr2=0x5596a05d8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.635+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05d8a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009230 tx=0x7f0fb002a410 comp rx=0 tx=0).stop 2024-09-10T23:00:34.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 shutdown_connections 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a05eb680 0x5596a05edb00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05d8a80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fa3fff640 0 client.27946 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2271484231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05db630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 >> 172.21.15.96:0/2271484231 conn(0x5596a05cd300 msgr2=0x5596a05d2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 shutdown_connections 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 1 -- 172.21.15.96:0/2271484231 wait complete. 2024-09-10T23:00:34.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.636+0000 7f0fcd568740 20 client.27946 trim_cache size 0 max 16384 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.653+0000 7f0fcd568740 1 Processor -- start 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.653+0000 7f0fcd568740 1 -- start start 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 0x5596a05ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05dbd80 0x5596a05ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ae300 con 0x5596a05cf4d0 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41afe40 con 0x5596a05cfdf0 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fcd568740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ae9d0 con 0x5596a05dbd80 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35032/0 (socket says 172.21.15.96:35032) 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/2059171221 learned_addr learned my addr 172.21.15.96:0/2059171221 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 0x5596a05ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc9730640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05dbd80 0x5596a05ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/2059171221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05dbd80 msgr2=0x5596a05ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05dbd80 0x5596a05ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/2059171221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 msgr2=0x5596a05ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 0x5596a05ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 -- 172.21.15.96:0/2059171221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a05cdd80 con 0x5596a05cf4d0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 0x5596a05ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80a95b0 tx=0x7f0fb80096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0fead395c76f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.654+0000 7f0fae7fc640 1 -- 172.21.15.96:0/2059171221 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb801c040 con 0x5596a05cf4d0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fae7fc640 1 -- 172.21.15.96:0/2059171221 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb80ac340 con 0x5596a05cf4d0 2024-09-10T23:00:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fae7fc640 1 -- 172.21.15.96:0/2059171221 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80a85b0 con 0x5596a05cf4d0 2024-09-10T23:00:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 -- 172.21.15.96:0/2059171221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 msgr2=0x5596a05ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80a95b0 tx=0x7f0fb80096e0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 -- 172.21.15.96:0/2059171221 shutdown_connections 2024-09-10T23:00:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05cfdf0 0x5596a05ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05dbd80 0x5596a05ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 --2- 172.21.15.96:0/2059171221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05ccc10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.655+0000 7f0fcd568740 1 -- 172.21.15.96:0/2059171221 >> 172.21.15.96:0/2059171221 conn(0x5596a05cd300 msgr2=0x5596a05d3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.656+0000 7f0fcd568740 1 -- 172.21.15.96:0/2059171221 shutdown_connections 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.656+0000 7f0fcd568740 1 -- 172.21.15.96:0/2059171221 wait complete. 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.656+0000 7f0fcd568740 1 -- start start 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.656+0000 7f0fcd568740 1 Processor -- start 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05e2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41af770 con 0x5596a05cf4d0 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41af0a0 con 0x5596a05dbd80 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ac2e0 con 0x5596a05cfdf0 2024-09-10T23:00:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fcd568740 10 client.0 _ll_register_callbacks cb 0x7f0fb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.657+0000 7f0fc9730640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05e2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc8f2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:35296/0 (socket says 172.21.15.96:35296) 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 -- 172.21.15.96:0/670169635 learned_addr learned my addr 172.21.15.96:0/670169635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 -- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 msgr2=0x5596a05e2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05e2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 -- 172.21.15.96:0/670169635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 msgr2=0x5596a05e4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05e4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 -- 172.21.15.96:0/670169635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a05e2c70 con 0x5596a05cfdf0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fc3fff640 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00038f0 tx=0x7f0fb0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a81a1529f206ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb005e070 con 0x5596a05cfdf0 2024-09-10T23:00:34.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0fb0004970 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb80ac340 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0fb006a050 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fcd568740 10 client.27952 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.658+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5596a065da10 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb80ac340 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0fb0048050 con 0x5596a05cfdf0 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 10 client.27952 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 10 client.27952 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 20 client.27952 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a065e190 0x5596a06605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 4 client.27952 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596a05438a0 con 0x5596a065e190 2024-09-10T23:00:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a065e190 0x5596a06605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.659+0000 7f0fa3fff640 10 client.27952 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fc8f2f640 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a065e190 0x5596a06605b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3631c7f0a3b497e8 server_cookie=1303b7bfe61ab2a8 in_seq=0 out_seq=0 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fa3fff640 1 -- 172.21.15.96:0/670169635 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596a05438a0 con 0x5596a065e190 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fa3fff640 10 client.27952 handle_command_reply: tid=0 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 client.27952 shutdown 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 msgr2=0x5596a05e1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00038f0 tx=0x7f0fb0009260 comp rx=0 tx=0).stop 2024-09-10T23:00:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 shutdown_connections 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5596a065e190 0x5596a06605b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a05dbd80 0x5596a05e2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596a05cfdf0 0x5596a05e1f00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 --2- 172.21.15.96:0/670169635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596a05cf4d0 0x5596a05e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 >> 172.21.15.96:0/670169635 conn(0x5596a05cd300 msgr2=0x5596a05ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fa3fff640 0 client.27952 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.660+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 shutdown_connections 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcd568740 1 -- 172.21.15.96:0/670169635 wait complete. 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcd568740 20 client.27952 trim_cache size 0 max 16384 2024-09-10T23:00:34.666 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:00:34.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 msgr2=0x7f0fb40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 0x7f0fb40b7820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0031440 tx=0x7f0fb00316c0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 msgr2=0x7f0fc41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:34.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 0x7f0fc41992f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800e9f0 tx=0x7f0fb800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 shutdown_connections 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0fb40b5360 0x7f0fb40b7820 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fc410d100 0x7f0fc419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc4104550 0x7f0fc4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 --2- 172.21.15.96:0/3218562665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc4103b50 0x7f0fc41992f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.661+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 >> 172.21.15.96:0/3218562665 conn(0x7f0fc40fd700 msgr2=0x7f0fc40fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.662+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 shutdown_connections 2024-09-10T23:00:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:34.662+0000 7f0fcb193640 1 -- 172.21.15.96:0/3218562665 wait complete. 2024-09-10T23:00:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/2516589437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc101860 msgr2=0x7f0dbc101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/2516589437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc101860 0x7f0dbc101c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009930 tx=0x7f0dac02a420 comp rx=0 tx=0).stop 2024-09-10T23:00:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/2516589437 shutdown_connections 2024-09-10T23:00:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/2516589437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc103370 0x7f0dbc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/2516589437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/2516589437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc101860 0x7f0dbc101c60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/2516589437 >> 172.21.15.96:0/2516589437 conn(0x7f0dbc0fd140 msgr2=0x7f0dbc0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/2516589437 shutdown_connections 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/2516589437 wait complete. 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.238+0000 7f0dc1bbc640 1 Processor -- start 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 -- start start 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc101860 0x7f0dbc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10d4b0 con 0x7f0dbc103370 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10d330 con 0x7f0dbc1029b0 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dc1bbc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc10d630 con 0x7f0dbc101860 2024-09-10T23:00:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35056/0 (socket says 172.21.15.96:35056) 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 -- 172.21.15.96:0/140324591 learned_addr learned my addr 172.21.15.96:0/140324591 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc101860 0x7f0dbc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0db3fff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 -- 172.21.15.96:0/140324591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc101860 msgr2=0x7f0dbc19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc101860 0x7f0dbc19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 -- 172.21.15.96:0/140324591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 msgr2=0x7f0dbc19b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc19b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 -- 172.21.15.96:0/140324591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc19ca70 con 0x7f0dbc103370 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0db3fff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.239+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0db400cce0 tx=0x7f0db400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.240+0000 7f0db97fa640 1 -- 172.21.15.96:0/140324591 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400f450 con 0x7f0dbc103370 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.240+0000 7f0db97fa640 1 -- 172.21.15.96:0/140324591 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db400faa0 con 0x7f0dbc103370 2024-09-10T23:00:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.240+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc19cd60 con 0x7f0dbc103370 2024-09-10T23:00:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.240+0000 7f0db97fa640 1 -- 172.21.15.96:0/140324591 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4018be0 con 0x7f0dbc103370 2024-09-10T23:00:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.240+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a3430 con 0x7f0dbc103370 2024-09-10T23:00:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.243+0000 7f0db97fa640 1 -- 172.21.15.96:0/140324591 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0db4020030 con 0x7f0dbc103370 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.243+0000 7f0db97fa640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 0x7f0da80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.243+0000 7f0dc4a63740 1 Processor -- start 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.243+0000 7f0dc4a63740 1 -- start start 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.243+0000 7f0db97fa640 1 -- 172.21.15.96:0/140324591 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0db40e5b00 con 0x7f0dbc103370 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0db3fff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 0x7f0da80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55833aabc6f0 con 0x55833ab4a860 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55833aaabb20 con 0x55833ab4fbd0 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55833aaaf930 con 0x55833ab4d190 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab4cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35062/0 (socket says 172.21.15.96:35062) 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/3474629818 learned_addr learned my addr 172.21.15.96:0/3474629818 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0db3fff640 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 -- 172.21.15.96:0/3474629818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 msgr2=0x55833ab4f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab4f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 -- 172.21.15.96:0/3474629818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 msgr2=0x55833ab4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0db3fff640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 0x7f0da80b78f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0da40058c0 tx=0x7f0da400a6c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab4cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 -- 172.21.15.96:0/3474629818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833aa71530 con 0x55833ab4fbd0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab52090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0db40e36f0 tx=0x7f0db40abcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2e745b80f8a13e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.244+0000 7f0db1ffb640 1 -- 172.21.15.96:0/3474629818 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40abe70 con 0x55833ab4fbd0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0db1ffb640 1 -- 172.21.15.96:0/3474629818 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db40aca60 con 0x55833ab4fbd0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0db1ffb640 1 -- 172.21.15.96:0/3474629818 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40b5460 con 0x55833ab4fbd0 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 -- 172.21.15.96:0/3474629818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 msgr2=0x55833ab52090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab52090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0db40e36f0 tx=0x7f0db40abcb0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 -- 172.21.15.96:0/3474629818 shutdown_connections 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab52090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/3474629818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 -- 172.21.15.96:0/3474629818 >> 172.21.15.96:0/3474629818 conn(0x55833ab459b0 msgr2=0x55833ab46740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 -- 172.21.15.96:0/3474629818 shutdown_connections 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.245+0000 7f0dc4a63740 1 -- 172.21.15.96:0/3474629818 wait complete. 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 -- start start 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 Processor -- start 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4d190 0x55833ab545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55833a9e3ab0 con 0x55833ab4a860 2024-09-10T23:00:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55833aa51730 con 0x55833ab4d190 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55833ab52bb0 con 0x55833ab4fbd0 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dc4a63740 10 client.0 _ll_register_callbacks cb 0x55833aaa56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35070/0 (socket says 172.21.15.96:35070) 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/248975767 learned_addr learned my addr 172.21.15.96:0/248975767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.246+0000 7f0db3fff640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4d190 0x55833ab545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/248975767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 msgr2=0x55833ab55190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab55190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4d190 msgr2=0x55833ab545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4d190 0x55833ab545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/248975767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55833a80d790 con 0x55833ab4a860 2024-09-10T23:00:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006010 tx=0x7f0dac0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe30c700709dec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0d9affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac035040 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0dac004450 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db40aca60 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0dac0035f0 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.247+0000 7f0dc4a63740 10 client.18288 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55833a886620 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db40aca60 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0dac044020 con 0x55833ab4a860 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 10 client.18288 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 10 client.18288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 20 client.18288 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833ab5f790 0x55833ab61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 4 client.18288 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55833aabe640 con 0x55833ab5f790 2024-09-10T23:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.248+0000 7f0db3fff640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833ab5f790 0x55833ab61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0d9affd640 10 client.18288 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0db3fff640 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833ab5f790 0x55833ab61b80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f78271634c80e3 server_cookie=1a383cd6db63f7b2 in_seq=0 out_seq=0 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0d9affd640 1 -- 172.21.15.96:0/248975767 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55833aabe640 con 0x55833ab5f790 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0d9affd640 10 client.18288 handle_command_reply: tid=0 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 client.18288 shutdown 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 msgr2=0x55833ab53f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006010 tx=0x7f0dac0317d0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 shutdown_connections 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833ab5f790 0x55833ab61b80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4d190 0x55833ab545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0d9affd640 0 client.18288 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.249+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/248975767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab53f20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.250+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 >> 172.21.15.96:0/248975767 conn(0x55833ab459b0 msgr2=0x55833ab44bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.250+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 shutdown_connections 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.250+0000 7f0dc4a63740 1 -- 172.21.15.96:0/248975767 wait complete. 2024-09-10T23:00:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.250+0000 7f0dc4a63740 20 client.18288 trim_cache size 0 max 16384 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 Processor -- start 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 -- start start 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 0x55833ab50040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1abf30 con 0x55833ab4d190 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1ada70 con 0x55833ab4a860 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.268+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1ac600 con 0x55833ab4fbd0 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35074/0 (socket says 172.21.15.96:35074) 2024-09-10T23:00:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 -- 172.21.15.96:0/4004550006 learned_addr learned my addr 172.21.15.96:0/4004550006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 0x55833ab50040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 -- 172.21.15.96:0/4004550006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 msgr2=0x55833ab50bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab50bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 -- 172.21.15.96:0/4004550006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 msgr2=0x55833ab50040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 0x55833ab50040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 -- 172.21.15.96:0/4004550006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833ab4cfc0 con 0x55833ab4d190 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 0x55833ab50040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db3fff640 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0da4019000 tx=0x7f0da4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764b49b0b87aa1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db17fa640 1 -- 172.21.15.96:0/4004550006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40212d0 con 0x55833ab4d190 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db17fa640 1 -- 172.21.15.96:0/4004550006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da4021470 con 0x55833ab4d190 2024-09-10T23:00:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.269+0000 7f0db17fa640 1 -- 172.21.15.96:0/4004550006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da401fd70 con 0x55833ab4d190 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- 172.21.15.96:0/4004550006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 msgr2=0x55833ab50580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0da4019000 tx=0x7f0da4014000 comp rx=0 tx=0).stop 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- 172.21.15.96:0/4004550006 shutdown_connections 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4fbd0 0x55833ab50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4d190 0x55833ab50580 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/4004550006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4a860 0x55833ab50040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- 172.21.15.96:0/4004550006 >> 172.21.15.96:0/4004550006 conn(0x55833ab459b0 msgr2=0x55833ab49d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- 172.21.15.96:0/4004550006 shutdown_connections 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- 172.21.15.96:0/4004550006 wait complete. 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.270+0000 7f0dc4a63740 1 -- start start 2024-09-10T23:00:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 Processor -- start 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab59ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1ad3a0 con 0x55833ab4a860 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1accd0 con 0x55833ab4fbd0 2024-09-10T23:00:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1a9f10 con 0x55833ab4d190 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dc4a63740 10 client.0 _ll_register_callbacks cb 0x7f0dac031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35078/0 (socket says 172.21.15.96:35078) 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/2712135008 learned_addr learned my addr 172.21.15.96:0/2712135008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0db3fff640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbbfff640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab59ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/2712135008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 msgr2=0x55833ab59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 msgr2=0x55833ab59ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab59ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0dbb7fe640 1 -- 172.21.15.96:0/2712135008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55833ab5a510 con 0x55833ab4a860 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.271+0000 7f0db3fff640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0dbb7fe640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0469a0 tx=0x7f0dac0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687f38e74ee9b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0d9affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac035040 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0dac004450 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da4021470 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0dac031d40 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0dc4a63740 10 client.18294 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55833ab5b8f0 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.272+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da4021470 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.273+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0dac044040 con 0x55833ab4a860 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 10 client.18294 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 10 client.18294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 20 client.18294 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833abcc810 0x55833abcec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 4 client.18294 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55833aaba0f0 con 0x55833abcc810 2024-09-10T23:00:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.274+0000 7f0db3fff640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833abcc810 0x55833abcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.275+0000 7f0db3fff640 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833abcc810 0x55833abcec00 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7408c9cc89cfb9e1 server_cookie=2468aa1b154da573 in_seq=0 out_seq=0 2024-09-10T23:00:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.275+0000 7f0d9affd640 10 client.18294 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0d9affd640 1 -- 172.21.15.96:0/2712135008 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55833aaba0f0 con 0x55833abcc810 2024-09-10T23:00:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0d9affd640 10 client.18294 handle_command_reply: tid=0 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 client.18294 shutdown 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 msgr2=0x55833ab58a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0469a0 tx=0x7f0dac0058f0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 shutdown_connections 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55833abcc810 0x55833abcec00 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55833ab4fbd0 0x55833ab59ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.298+0000 7f0d9affd640 0 client.18294 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833ab4d190 0x55833ab59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 1 --2- 172.21.15.96:0/2712135008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55833ab4a860 0x55833ab58a70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 >> 172.21.15.96:0/2712135008 conn(0x55833ab459b0 msgr2=0x55833ab49950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 shutdown_connections 2024-09-10T23:00:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 1 -- 172.21.15.96:0/2712135008 wait complete. 2024-09-10T23:00:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.299+0000 7f0dc4a63740 20 client.18294 trim_cache size 0 max 16384 2024-09-10T23:00:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:00:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:00:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:00:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 msgr2=0x7f0da80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 0x7f0da80b78f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0da40058c0 tx=0x7f0da400a6c0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 msgr2=0x7f0dbc19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0db400cce0 tx=0x7f0db400a620 comp rx=0 tx=0).stop 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 shutdown_connections 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dbc1029b0 0x7f0dbc19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0da80b5430 0x7f0da80b78f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dbc103370 0x7f0dbc19c1b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 --2- 172.21.15.96:0/140324591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dbc101860 0x7f0dbc19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 >> 172.21.15.96:0/140324591 conn(0x7f0dbc0fd140 msgr2=0x7f0dbc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 shutdown_connections 2024-09-10T23:00:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:35.300+0000 7f0dc1bbc640 1 -- 172.21.15.96:0/140324591 wait complete. 2024-09-10T23:00:35.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T23:00:35.633 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:35 smithi096 ceph-mon[30869]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 713 op/s 2024-09-10T23:00:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:35 smithi117 ceph-mon[35081]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 713 op/s 2024-09-10T23:00:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 780.0 s, 1690.6 tps, lat 59.321 ms stddev 67.325 2024-09-10T23:00:35.988 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:36.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:35 smithi121 ceph-mon[35119]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 713 op/s 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 -- 172.21.15.96:0/530544685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94102de0 msgr2=0x7f2a941031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 --2- 172.21.15.96:0/530544685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94102de0 0x7f2a941031c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009860 tx=0x7f2a8402a340 comp rx=0 tx=0).stop 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 -- 172.21.15.96:0/530544685 shutdown_connections 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 --2- 172.21.15.96:0/530544685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94109170 0x7f2a9410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 --2- 172.21.15.96:0/530544685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 0x7f2a94108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 --2- 172.21.15.96:0/530544685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94102de0 0x7f2a941031c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 -- 172.21.15.96:0/530544685 >> 172.21.15.96:0/530544685 conn(0x7f2a940fe870 msgr2=0x7f2a94100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 -- 172.21.15.96:0/530544685 shutdown_connections 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.570+0000 7f2a9a679640 1 -- 172.21.15.96:0/530544685 wait complete. 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 Processor -- start 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 -- start start 2024-09-10T23:00:36.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94102de0 0x7f2a94199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 0x7f2a941998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94112e30 con 0x7f2a94109170 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94112cb0 con 0x7f2a94102de0 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a9a679640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94112fb0 con 0x7f2a941087f0 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a98c16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a98c16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50330/0 (socket says 172.21.15.96:50330) 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a98c16640 1 -- 172.21.15.96:0/198071424 learned_addr learned my addr 172.21.15.96:0/198071424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94102de0 0x7f2a94199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.571+0000 7f2a937fe640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 0x7f2a941998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 -- 172.21.15.96:0/198071424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 msgr2=0x7f2a941998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 0x7f2a941998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 -- 172.21.15.96:0/198071424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94102de0 msgr2=0x7f2a94199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94102de0 0x7f2a94199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 -- 172.21.15.96:0/198071424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9419f410 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a98c16640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00eb10 tx=0x7f2a8c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a917fa640 1 -- 172.21.15.96:0/198071424 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00ce20 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a917fa640 1 -- 172.21.15.96:0/198071424 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8c005020 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9419f700 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.572+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941a5dc0 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.573+0000 7f2a917fa640 1 -- 172.21.15.96:0/198071424 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c010690 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.575+0000 7f2a9ca4e740 1 Processor -- start 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a917fa640 1 -- 172.21.15.96:0/198071424 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f2a8c005510 con 0x7f2a94109170 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 -- start start 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 0x55ff355f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 0x55ff355fb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff3555eea0 con 0x55ff355f9190 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ff3554e2d0 con 0x55ff355e8df0 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ff355520e0 con 0x55ff355e84d0 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 0x55ff355fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 0x55ff355fb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50340/0 (socket says 172.21.15.96:50340) 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a937fe640 1 -- 172.21.15.96:0/4186535718 learned_addr learned my addr 172.21.15.96:0/4186535718 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a93fff640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 0x55ff355f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a937fe640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.576+0000 7f2a917fa640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 0x7f2a780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 -- 172.21.15.96:0/4186535718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 msgr2=0x55ff355e88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355e88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 -- 172.21.15.96:0/4186535718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 msgr2=0x55ff355fb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 0x55ff355fb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 -- 172.21.15.96:0/4186535718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff352ac650 con 0x55ff355e8df0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a937fe640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a917fa640 1 -- 172.21.15.96:0/198071424 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2a8c0e6060 con 0x7f2a94109170 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a98c16640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 0x7f2a780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a93fff640 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 0x55ff355f8c50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035040 tx=0x7f2a84005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c77d0956a8c7df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a8a7fc640 1 -- 172.21.15.96:0/4186535718 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84041020 con 0x55ff355e8df0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a8a7fc640 1 -- 172.21.15.96:0/4186535718 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a84030450 con 0x55ff355e8df0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a8a7fc640 1 -- 172.21.15.96:0/4186535718 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037040 con 0x55ff355e8df0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.577+0000 7f2a98c16640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 0x7f2a780b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0ac5c0 tx=0x7f2a8c0058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.578+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/4186535718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 msgr2=0x55ff355f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.578+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 0x55ff355f8c50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035040 tx=0x7f2a84005d90 comp rx=0 tx=0).stop 2024-09-10T23:00:36.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.579+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/4186535718 shutdown_connections 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.579+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355e8df0 0x55ff355f8c50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.579+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355f9190 0x55ff355fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.579+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/4186535718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.579+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/4186535718 >> 172.21.15.96:0/4186535718 conn(0x55ff355e6300 msgr2=0x55ff355e74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.580+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/4186535718 shutdown_connections 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.580+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/4186535718 wait complete. 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.581+0000 7f2a9ca4e740 1 -- start start 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.581+0000 7f2a9ca4e740 1 Processor -- start 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff355f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355f6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff353af700 con 0x55ff355e84d0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ff354e9160 con 0x55ff355f9190 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ff355fbde0 con 0x55ff355e8df0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45330/0 (socket says 172.21.15.96:45330) 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 -- 172.21.15.96:0/2175703529 learned_addr learned my addr 172.21.15.96:0/2175703529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff355f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a9ca4e740 10 client.0 _ll_register_callbacks cb 0x55ff355029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 -- 172.21.15.96:0/2175703529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 msgr2=0x55ff355f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 -- 172.21.15.96:0/2175703529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 msgr2=0x55ff355f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff355f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.582+0000 7f2a93fff640 1 -- 172.21.15.96:0/2175703529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff35516c90 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a84030fc0 tx=0x7f2a84005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3d833c43f15a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84041030 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2a8403d680 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a84030450 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a84037040 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a9ca4e740 10 client.27976 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff356075c0 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.583+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a84030450 con 0x55ff355e8df0 2024-09-10T23:00:36.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2a84046940 con 0x55ff355e8df0 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 10 client.27976 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 10 client.27976 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 20 client.27976 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35608910 0x55ff3560ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 4 client.27976 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.584+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff35560df0 con 0x55ff35608910 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.585+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35608910 0x55ff3560ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.585+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35608910 0x55ff3560ad00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68c7ab96768750ba server_cookie=af3f80a6a7332494 in_seq=0 out_seq=0 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.585+0000 7f2a777fe640 10 client.27976 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a777fe640 1 -- 172.21.15.96:0/2175703529 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff35560df0 con 0x55ff35608910 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a777fe640 10 client.27976 handle_command_reply: tid=0 2024-09-10T23:00:36.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 client.27976 shutdown 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 msgr2=0x55ff355f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a84030fc0 tx=0x7f2a84005850 comp rx=0 tx=0).stop 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 shutdown_connections 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35608910 0x55ff3560ad00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a777fe640 0 client.27976 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff355f5e30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2175703529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff355f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 >> 172.21.15.96:0/2175703529 conn(0x55ff355e6300 msgr2=0x55ff355e5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.586+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 shutdown_connections 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.590+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2175703529 wait complete. 2024-09-10T23:00:36.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.595+0000 7f2a9ca4e740 20 client.27976 trim_cache size 0 max 16384 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.627+0000 7f2a9ca4e740 1 Processor -- start 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.628+0000 7f2a9ca4e740 1 -- start start 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.628+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.628+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 0x55ff355f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355faee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ac030 con 0x55ff355e8df0 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941adb70 con 0x55ff355f9190 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ac700 con 0x55ff355e84d0 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a93fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 0x55ff355f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a93fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 0x55ff355f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50358/0 (socket says 172.21.15.96:50358) 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a93fff640 1 -- 172.21.15.96:0/3692076897 learned_addr learned my addr 172.21.15.96:0/3692076897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.629+0000 7f2a937fe640 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.637+0000 7f2a937fe640 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.641+0000 7f2a937fe640 1 -- 172.21.15.96:0/3692076897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 msgr2=0x55ff355faee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.641+0000 7f2a937fe640 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355faee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.641+0000 7f2a937fe640 1 -- 172.21.15.96:0/3692076897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 msgr2=0x55ff355f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.641+0000 7f2a937fe640 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 0x55ff355f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.641+0000 7f2a937fe640 1 -- 172.21.15.96:0/3692076897 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff355f6700 con 0x55ff355e84d0 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.644+0000 7f2a937fe640 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355f46d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55ff3554e390 tx=0x7f2a80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42b390f12b4e784 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.649+0000 7f2a89ffb640 1 -- 172.21.15.96:0/3692076897 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80010920 con 0x55ff355e84d0 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.649+0000 7f2a89ffb640 1 -- 172.21.15.96:0/3692076897 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a80005020 con 0x55ff355e84d0 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.650+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/3692076897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 msgr2=0x55ff355f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.650+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355f46d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55ff3554e390 tx=0x7f2a80004730 comp rx=0 tx=0).stop 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.651+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/3692076897 shutdown_connections 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.651+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff355faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.651+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e8df0 0x55ff355f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.651+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/3692076897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e84d0 0x55ff355f46d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.651+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/3692076897 >> 172.21.15.96:0/3692076897 conn(0x55ff355e6300 msgr2=0x55ff355f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.658+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/3692076897 shutdown_connections 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.660+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/3692076897 wait complete. 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.660+0000 7f2a9ca4e740 1 -- start start 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.660+0000 7f2a9ca4e740 1 Processor -- start 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff3560c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff3560cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff3566e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941ad4a0 con 0x55ff355e84d0 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941acdd0 con 0x55ff355f9190 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941aa010 con 0x55ff355e8df0 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.661+0000 7f2a9ca4e740 10 client.0 _ll_register_callbacks cb 0x7f2a80005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:36.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.663+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff3566e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.663+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff3560c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.663+0000 7f2a937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff3560c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50366/0 (socket says 172.21.15.96:50366) 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.663+0000 7f2a937fe640 1 -- 172.21.15.96:0/2131256570 learned_addr learned my addr 172.21.15.96:0/2131256570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.663+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff3560cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 -- 172.21.15.96:0/2131256570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 msgr2=0x55ff3566e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff3566e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 -- 172.21.15.96:0/2131256570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 msgr2=0x55ff3560c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff3560c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 -- 172.21.15.96:0/2131256570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3566ebd0 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a93fff640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff3560cbb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ff355f87d0 tx=0x7f2a84004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d4b0f4be093c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.664+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84041040 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80005020 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2a84005230 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a84005450 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a9ca4e740 10 client.27982 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.665+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff35686f80 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a80005020 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2a84046980 con 0x55ff355e8df0 2024-09-10T23:00:36.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 10 client.27982 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:36.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 10 client.27982 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:00:36.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 20 client.27982 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:36.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35687710 0x55ff35689bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:36.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 4 client.27982 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:36.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.666+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff3555c8a0 con 0x55ff35687710 2024-09-10T23:00:36.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.672+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35687710 0x55ff35689bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:36.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a937fe640 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35687710 0x55ff35689bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c3cb9e13cae8b7d server_cookie=e5933870b2739fa4 in_seq=0 out_seq=0 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a777fe640 10 client.27982 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a777fe640 1 -- 172.21.15.96:0/2131256570 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff3555c8a0 con 0x55ff35687710 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a777fe640 10 client.27982 handle_command_reply: tid=0 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 client.27982 shutdown 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 msgr2=0x55ff3560cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff3560cbb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ff355f87d0 tx=0x7f2a84004270 comp rx=0 tx=0).stop 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 shutdown_connections 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ff35687710 0x55ff35689bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.675+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff355f9190 0x55ff3566e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff355e8df0 0x55ff3560cbb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 1 --2- 172.21.15.96:0/2131256570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff355e84d0 0x55ff3560c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 >> 172.21.15.96:0/2131256570 conn(0x55ff355e6300 msgr2=0x55ff355faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a777fe640 0 client.27982 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:00:36.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 shutdown_connections 2024-09-10T23:00:36.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 1 -- 172.21.15.96:0/2131256570 wait complete. 2024-09-10T23:00:36.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.676+0000 7f2a9ca4e740 20 client.27982 trim_cache size 0 max 16384 2024-09-10T23:00:36.679 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.677+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 msgr2=0x7f2a780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.677+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 0x7f2a780b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0ac5c0 tx=0x7f2a8c0058b0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.677+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 msgr2=0x7f2a9419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.677+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00eb10 tx=0x7f2a8c00ef10 comp rx=0 tx=0).stop 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 shutdown_connections 2024-09-10T23:00:36.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f2a780b52e0 0x7f2a780b77a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a94109170 0x7f2a9419ec40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a941087f0 0x7f2a941998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 --2- 172.21.15.96:0/198071424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a94102de0 0x7f2a94199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.678+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 >> 172.21.15.96:0/198071424 conn(0x7f2a940fe870 msgr2=0x7f2a94100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.679+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 shutdown_connections 2024-09-10T23:00:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:36.679+0000 7f2a9a679640 1 -- 172.21.15.96:0/198071424 wait complete. 2024-09-10T23:00:37.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T23:00:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:37 smithi117 ceph-mon[35081]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-09-10T23:00:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:37 smithi096 ceph-mon[30869]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-09-10T23:00:38.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:37 smithi121 ceph-mon[35119]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 756 op/s 2024-09-10T23:00:38.055 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:38.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 -- 172.21.15.96:0/432781705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075540 msgr2=0x7f3d60075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 --2- 172.21.15.96:0/432781705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075540 0x7f3d60075920 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009ca0 tx=0x7f3d5802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 -- 172.21.15.96:0/432781705 shutdown_connections 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 --2- 172.21.15.96:0/432781705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60075e60 0x7f3d60113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 --2- 172.21.15.96:0/432781705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60113b40 0x7f3d60115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 --2- 172.21.15.96:0/432781705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075540 0x7f3d60075920 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 -- 172.21.15.96:0/432781705 >> 172.21.15.96:0/432781705 conn(0x7f3d600fe830 msgr2=0x7f3d60100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 -- 172.21.15.96:0/432781705 shutdown_connections 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.650+0000 7f3d66782640 1 -- 172.21.15.96:0/432781705 wait complete. 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.651+0000 7f3d66782640 1 Processor -- start 2024-09-10T23:00:38.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.651+0000 7f3d66782640 1 -- start start 2024-09-10T23:00:38.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.651+0000 7f3d66782640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.651+0000 7f3d66782640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075e60 0x7f3d6019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.651+0000 7f3d66782640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 0x7f3d601a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d66782640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60119420 con 0x7f3d60075e60 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d66782640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601192a0 con 0x7f3d60113b40 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d66782640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601195a0 con 0x7f3d60075540 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45376/0 (socket says 172.21.15.96:45376) 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2438317751 learned_addr learned my addr 172.21.15.96:0/2438317751 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075e60 0x7f3d6019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 0x7f3d601a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2438317751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 msgr2=0x7f3d601a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 0x7f3d601a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075e60 msgr2=0x7f3d6019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075e60 0x7f3d6019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2438317751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601a5980 con 0x7f3d60075540 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 0x7f3d601a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005e30 tx=0x7f3d58031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5d7fa640 1 -- 172.21.15.96:0/2438317751 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58003b70 con 0x7f3d60075540 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5d7fa640 1 -- 172.21.15.96:0/2438317751 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58003d10 con 0x7f3d60075540 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601a5c10 con 0x7f3d60075540 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.652+0000 7f3d5d7fa640 1 -- 172.21.15.96:0/2438317751 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5802fa30 con 0x7f3d60075540 2024-09-10T23:00:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.653+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601ac330 con 0x7f3d60075540 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 Processor -- start 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 -- start start 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d5d7fa640 1 -- 172.21.15.96:0/2438317751 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3d5802fbd0 con 0x7f3d60075540 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 0x5568d36964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36a29a0 0x5568d36a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568d360cea0 con 0x5568d3696a10 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568d35fc2d0 con 0x5568d36960f0 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d68b57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5568d36000e0 con 0x5568d36a29a0 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50388/0 (socket says 172.21.15.96:50388) 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1401167275 learned_addr learned my addr 172.21.15.96:0/1401167275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36a29a0 0x5568d36a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.656+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 0x5568d36964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1401167275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36a29a0 msgr2=0x5568d36a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36a29a0 0x5568d36a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1401167275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 msgr2=0x5568d36964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 0x5568d36964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1401167275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d335a650 con 0x5568d3696a10 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 0x5568d36964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3d5810ac30 tx=0x7f3d5810ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba8943fe2b272c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5d7fa640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 0x7f3d480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d467fc640 1 -- 172.21.15.96:0/1401167275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580c9a90 con 0x5568d3696a10 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 0x7f3d480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d467fc640 1 -- 172.21.15.96:0/1401167275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d580cdb40 con 0x5568d3696a10 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d467fc640 1 -- 172.21.15.96:0/1401167275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580ccd90 con 0x5568d3696a10 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.657+0000 7f3d5d7fa640 1 -- 172.21.15.96:0/2438317751 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d58106be0 con 0x7f3d60075540 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 0x7f3d480b77a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3d6006c430 tx=0x7f3d50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- 172.21.15.96:0/1401167275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 msgr2=0x5568d36a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3d5810ac30 tx=0x7f3d5810ad20 comp rx=0 tx=0).stop 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- 172.21.15.96:0/1401167275 shutdown_connections 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36a29a0 0x5568d36a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d3696a10 0x5568d36a2460 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1401167275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d36960f0 0x5568d36964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- 172.21.15.96:0/1401167275 >> 172.21.15.96:0/1401167275 conn(0x5568d3693f20 msgr2=0x5568d36950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- 172.21.15.96:0/1401167275 shutdown_connections 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- 172.21.15.96:0/1401167275 wait complete. 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.658+0000 7f3d68b57740 1 -- start start 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 Processor -- start 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d369fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568d345d700 con 0x5568d36a29a0 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568d3597160 con 0x5568d3696a10 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5568d360cea0 con 0x5568d36960f0 2024-09-10T23:00:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d68b57740 10 client.0 _ll_register_callbacks cb 0x5568d35b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45404/0 (socket says 172.21.15.96:45404) 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1408288617 learned_addr learned my addr 172.21.15.96:0/1408288617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d369fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 msgr2=0x5568d369f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1408288617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 msgr2=0x5568d369fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.659+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d369fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1408288617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d35c4c90 con 0x5568d36960f0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d580cd140 tx=0x7f3d58106ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde65f9247836e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580ccf80 con 0x5568d36960f0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d580c96b0 con 0x5568d36960f0 2024-09-10T23:00:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d580cdb40 con 0x5568d36960f0 2024-09-10T23:00:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d580cd210 con 0x5568d36960f0 2024-09-10T23:00:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d68b57740 10 client.27994 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568d36b0b00 con 0x5568d36960f0 2024-09-10T23:00:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.660+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d580cdb40 con 0x5568d36960f0 2024-09-10T23:00:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3d5809fce0 con 0x5568d36960f0 2024-09-10T23:00:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 10 client.27994 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 10 client.27994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:00:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 20 client.27994 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d36b1c80 0x5568d36b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 4 client.27994 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.661+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568d360edf0 con 0x5568d36b1c80 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.662+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d36b1c80 0x5568d36b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.662+0000 7f3d3f7fe640 10 client.27994 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.662+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d36b1c80 0x5568d36b4070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3cb16cd45633a0d server_cookie=e0d66213b22c0ce6 in_seq=0 out_seq=0 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.663+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1408288617 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568d360edf0 con 0x5568d36b1c80 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.663+0000 7f3d3f7fe640 10 client.27994 handle_command_reply: tid=0 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.663+0000 7f3d68b57740 1 client.27994 shutdown 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.663+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 msgr2=0x5568d36a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.663+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d580cd140 tx=0x7f3d58106ee0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 shutdown_connections 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d36b1c80 0x5568d36b4070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d369fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d3f7fe640 0 client.27994 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:00:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1408288617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36a2280 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 >> 172.21.15.96:0/1408288617 conn(0x5568d3693f20 msgr2=0x5568d3699ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 shutdown_connections 2024-09-10T23:00:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 1 -- 172.21.15.96:0/1408288617 wait complete. 2024-09-10T23:00:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.664+0000 7f3d68b57740 20 client.27994 trim_cache size 0 max 16384 2024-09-10T23:00:38.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 Processor -- start 2024-09-10T23:00:38.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 -- start start 2024-09-10T23:00:38.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d369df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b4940 con 0x5568d36a29a0 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b6480 con 0x5568d3696a10 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.680+0000 7f3d68b57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b5010 con 0x5568d36960f0 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d64d1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d369df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d64d1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50396/0 (socket says 172.21.15.96:50396) 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d64d1f640 1 -- 172.21.15.96:0/2988083586 learned_addr learned my addr 172.21.15.96:0/2988083586 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2988083586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 msgr2=0x5568d369e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2988083586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 msgr2=0x5568d36a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 -- 172.21.15.96:0/2988083586 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d36a0070 con 0x5568d36960f0 2024-09-10T23:00:38.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:38.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d369df50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d580c9c00 tx=0x7f3d58106f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8a1c6c8079246f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d45ffb640 1 -- 172.21.15.96:0/2988083586 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580c8bb0 con 0x5568d36960f0 2024-09-10T23:00:38.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d45ffb640 1 -- 172.21.15.96:0/2988083586 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5810a020 con 0x5568d36960f0 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.681+0000 7f3d45ffb640 1 -- 172.21.15.96:0/2988083586 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580c8d90 con 0x5568d36960f0 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 -- 172.21.15.96:0/2988083586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 msgr2=0x5568d369df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d369df50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d580c9c00 tx=0x7f3d58106f50 comp rx=0 tx=0).stop 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 -- 172.21.15.96:0/2988083586 shutdown_connections 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d369e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 --2- 172.21.15.96:0/2988083586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d369df50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.682+0000 7f3d68b57740 1 -- 172.21.15.96:0/2988083586 >> 172.21.15.96:0/2988083586 conn(0x5568d3693f20 msgr2=0x5568d369a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 -- 172.21.15.96:0/2988083586 shutdown_connections 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 -- 172.21.15.96:0/2988083586 wait complete. 2024-09-10T23:00:38.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 -- start start 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 Processor -- start 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d36aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.683+0000 7f3d68b57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d68b57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b5db0 con 0x5568d36a29a0 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d68b57740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b56e0 con 0x5568d3696a10 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d68b57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b2920 con 0x5568d36960f0 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45416/0 (socket says 172.21.15.96:45416) 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1552199014 learned_addr learned my addr 172.21.15.96:0/1552199014 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d36aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d64d1f640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d68b57740 10 client.0 _ll_register_callbacks cb 0x7f3d540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 msgr2=0x5568d36aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d36aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1552199014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 msgr2=0x5568d36a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 -- 172.21.15.96:0/1552199014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d36a5bb0 con 0x5568d36960f0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d36aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5810a020 con 0x5568d36960f0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d5ffff640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d580ccd20 tx=0x7f3d580ccdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360e8db492b90e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.684+0000 7f3d3f7fe640 10 client.28000 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.685+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1552199014 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580c9110 con 0x5568d36960f0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.685+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1552199014 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d580d6740 con 0x5568d36960f0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.685+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1552199014 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d580c92e0 con 0x5568d36960f0 2024-09-10T23:00:38.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.685+0000 7f3d68b57740 10 client.28000 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:38.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.685+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5568d36a5bb0 con 0x5568d36960f0 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1552199014 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3d580cd6a0 con 0x5568d36960f0 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 10 client.28000 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 10 client.28000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 20 client.28000 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d371f560 0x5568d3721980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 4 client.28000 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.686+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568d360a8a0 con 0x5568d371f560 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d371f560 0x5568d3721980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d3f7fe640 10 client.28000 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d5f7fe640 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d371f560 0x5568d3721980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2e459926fda4e7a server_cookie=c9645edc13915ea9 in_seq=0 out_seq=0 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1552199014 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568d360a8a0 con 0x5568d371f560 2024-09-10T23:00:38.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d3f7fe640 10 client.28000 handle_command_reply: tid=0 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d68b57740 1 client.28000 shutdown 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 msgr2=0x5568d36aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.687+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d580ccd20 tx=0x7f3d580ccdb0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 shutdown_connections 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x5568d371f560 0x5568d3721980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568d3696a10 0x5568d36aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568d36a29a0 0x5568d36a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 --2- 172.21.15.96:0/1552199014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568d36960f0 0x5568d36aa6a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d3f7fe640 0 client.28000 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 >> 172.21.15.96:0/1552199014 conn(0x5568d3693f20 msgr2=0x5568d369de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 shutdown_connections 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 1 -- 172.21.15.96:0/1552199014 wait complete. 2024-09-10T23:00:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.688+0000 7f3d68b57740 20 client.28000 trim_cache size 0 max 16384 2024-09-10T23:00:38.695 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 msgr2=0x7f3d480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 0x7f3d480b77a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3d6006c430 tx=0x7f3d50005f90 comp rx=0 tx=0).stop 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 msgr2=0x7f3d6019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005e30 tx=0x7f3d58031840 comp rx=0 tx=0).stop 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 shutdown_connections 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60113b40 0x7f3d601a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d480b52e0 0x7f3d480b77a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d60075e60 0x7f3d6019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 --2- 172.21.15.96:0/2438317751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d60075540 0x7f3d6019f8d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:38.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 >> 172.21.15.96:0/2438317751 conn(0x7f3d600fe830 msgr2=0x7f3d600ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:38.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 shutdown_connections 2024-09-10T23:00:38.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:38.689+0000 7f3d66782640 1 -- 172.21.15.96:0/2438317751 wait complete. 2024-09-10T23:00:39.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T23:00:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:39 smithi096 ceph-mon[30869]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-09-10T23:00:39.900 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:40.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:39 smithi121 ceph-mon[35119]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-09-10T23:00:40.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:39 smithi117 ceph-mon[35081]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 617 op/s 2024-09-10T23:00:40.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 -- 172.21.15.96:0/1617043493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544101c90 msgr2=0x7f2544102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 --2- 172.21.15.96:0/1617043493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544101c90 0x7f2544102070 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2534009900 tx=0x7f253402a410 comp rx=0 tx=0).stop 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 -- 172.21.15.96:0/1617043493 shutdown_connections 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 --2- 172.21.15.96:0/1617043493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25441076a0 0x7f2544107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 --2- 172.21.15.96:0/1617043493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 0x7f254410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 --2- 172.21.15.96:0/1617043493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544101c90 0x7f2544102070 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 -- 172.21.15.96:0/1617043493 >> 172.21.15.96:0/1617043493 conn(0x7f25440fd700 msgr2=0x7f25440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 -- 172.21.15.96:0/1617043493 shutdown_connections 2024-09-10T23:00:40.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.582+0000 7f2549fa4640 1 -- 172.21.15.96:0/1617043493 wait complete. 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 Processor -- start 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 -- start start 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 0x7f25441993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 0x7f254419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2544111ce0 con 0x7f25441076a0 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2544111b60 con 0x7f2544101c90 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2549fa4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2544111e60 con 0x7f2544108020 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50422/0 (socket says 172.21.15.96:50422) 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2542ffd640 1 -- 172.21.15.96:0/472421533 learned_addr learned my addr 172.21.15.96:0/472421533 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:40.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f2543fff640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 0x7f254419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.583+0000 7f25437fe640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 0x7f25441993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 -- 172.21.15.96:0/472421533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 msgr2=0x7f254419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 0x7f254419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 -- 172.21.15.96:0/472421533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 msgr2=0x7f25441993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 0x7f25441993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 -- 172.21.15.96:0/472421533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254419f490 con 0x7f25441076a0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f25437fe640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 0x7f25441993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2542ffd640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f253800d950 tx=0x7f253800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2540ff9640 1 -- 172.21.15.96:0/472421533 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538017030 con 0x7f25441076a0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2540ff9640 1 -- 172.21.15.96:0/472421533 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2538004020 con 0x7f25441076a0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2540ff9640 1 -- 172.21.15.96:0/472421533 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25380047a0 con 0x7f25441076a0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2549fa4640 1 -- 172.21.15.96:0/472421533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254419f780 con 0x7f25441076a0 2024-09-10T23:00:40.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.584+0000 7f2549fa4640 1 -- 172.21.15.96:0/472421533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441a5e40 con 0x7f25441076a0 2024-09-10T23:00:40.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.588+0000 7f254c379740 1 Processor -- start 2024-09-10T23:00:40.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 -- start start 2024-09-10T23:00:40.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48ecef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ecfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a48e3fea0 con 0x559a48ecf5b0 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a48e2f2d0 con 0x559a48ec7800 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.592+0000 7f254c379740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a48e330e0 con 0x559a48ec6e00 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.593+0000 7f2543fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.593+0000 7f2543fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45442/0 (socket says 172.21.15.96:45442) 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.593+0000 7f2543fff640 1 -- 172.21.15.96:0/1259880833 learned_addr learned my addr 172.21.15.96:0/1259880833 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.594+0000 7f2543fff640 1 -- 172.21.15.96:0/1259880833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 msgr2=0x559a48ecef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.594+0000 7f2543fff640 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48ecef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.594+0000 7f2543fff640 1 -- 172.21.15.96:0/1259880833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 msgr2=0x559a48ecfa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.594+0000 7f2543fff640 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ecfa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.594+0000 7f2543fff640 1 -- 172.21.15.96:0/1259880833 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a48b8d650 con 0x559a48ec6e00 2024-09-10T23:00:40.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.595+0000 7f2543fff640 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f90 tx=0x7f253c032870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e923d90ebc7c265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.602+0000 7f2549fa4640 1 -- 172.21.15.96:0/1259880833 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c033710 con 0x559a48ec6e00 2024-09-10T23:00:40.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.602+0000 7f2549fa4640 1 -- 172.21.15.96:0/1259880833 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c033d60 con 0x559a48ec6e00 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 -- 172.21.15.96:0/1259880833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 msgr2=0x559a48ec71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f90 tx=0x7f253c032870 comp rx=0 tx=0).stop 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 -- 172.21.15.96:0/1259880833 shutdown_connections 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48ecef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ecfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.603+0000 7f254c379740 1 --2- 172.21.15.96:0/1259880833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ec71e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.604+0000 7f254c379740 1 -- 172.21.15.96:0/1259880833 >> 172.21.15.96:0/1259880833 conn(0x559a48ec9380 msgr2=0x559a48ec9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.604+0000 7f2540ff9640 1 -- 172.21.15.96:0/472421533 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f25380041d0 con 0x7f25441076a0 2024-09-10T23:00:40.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.605+0000 7f2540ff9640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 0x7f252c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.605+0000 7f2540ff9640 1 -- 172.21.15.96:0/472421533 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f25380e5410 con 0x7f25441076a0 2024-09-10T23:00:40.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.606+0000 7f25437fe640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 0x7f252c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.606+0000 7f25437fe640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 0x7f252c0b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2534031440 tx=0x7f2534005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.613+0000 7f254c379740 1 -- 172.21.15.96:0/1259880833 shutdown_connections 2024-09-10T23:00:40.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.613+0000 7f254c379740 1 -- 172.21.15.96:0/1259880833 wait complete. 2024-09-10T23:00:40.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.613+0000 7f254c379740 1 -- start start 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.613+0000 7f254c379740 1 Processor -- start 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ed5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ecf5b0 0x559a48edaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f25437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f25437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40584/0 (socket says 172.21.15.96:40584) 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f25437fe640 1 -- 172.21.15.96:0/2031227381 learned_addr learned my addr 172.21.15.96:0/2031227381 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f2543fff640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ecf5b0 0x559a48edaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a48c90700 con 0x559a48ec7800 2024-09-10T23:00:40.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a48dca160 con 0x559a48ec6e00 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.614+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a48ed2670 con 0x559a48ecf5b0 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f254c379740 10 client.0 _ll_register_callbacks cb 0x559a48de39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 -- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ecf5b0 msgr2=0x559a48edaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ecf5b0 0x559a48edaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 -- 172.21.15.96:0/2031227381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 msgr2=0x559a48ed5e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ed5e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a48df7c90 con 0x559a48ec6e00 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f25437fe640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2534009900 tx=0x7f253402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c22b215432a945b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.615+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a48e41df0 con 0x559a48ec6e00 2024-09-10T23:00:40.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.618+0000 7f2522ffd640 10 client.27944 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:00:40.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.619+0000 7f2522ffd640 1 -- 172.21.15.96:0/2031227381 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253404d020 con 0x559a48ec6e00 2024-09-10T23:00:40.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.620+0000 7f2522ffd640 1 -- 172.21.15.96:0/2031227381 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2534005040 con 0x559a48ec6e00 2024-09-10T23:00:40.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.620+0000 7f2522ffd640 1 -- 172.21.15.96:0/2031227381 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2534035040 con 0x559a48ec6e00 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.620+0000 7f254c379740 10 client.27944 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.620+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x559a48ee4020 con 0x559a48ec6e00 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.620+0000 7f2522ffd640 1 -- 172.21.15.96:0/2031227381 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f253405a020 con 0x559a48ec6e00 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 10 client.27944 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 10 client.27944 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 20 client.27944 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48ee5370 0x559a48ee7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 4 client.27944 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a48e41df0 con 0x559a48ee5370 2024-09-10T23:00:40.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f2542ffd640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48ee5370 0x559a48ee7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f2522ffd640 10 client.27944 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.621+0000 7f2542ffd640 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48ee5370 0x559a48ee7780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=229a0fd756a43ff0 server_cookie=de2cc1001fb68008 in_seq=0 out_seq=0 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.622+0000 7f2522ffd640 1 -- 172.21.15.96:0/2031227381 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a48e41df0 con 0x559a48ee5370 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.622+0000 7f2522ffd640 10 client.27944 handle_command_reply: tid=0 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 client.27944 shutdown 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 msgr2=0x559a48ed58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2534009900 tx=0x7f253402a410 comp rx=0 tx=0).stop 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 shutdown_connections 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ecf5b0 0x559a48edaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ed5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48ee5370 0x559a48ee7780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 --2- 172.21.15.96:0/2031227381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec6e00 0x559a48ed58c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 >> 172.21.15.96:0/2031227381 conn(0x559a48ec9380 msgr2=0x559a48ec8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f2522ffd640 0 client.27944 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:00:40.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 shutdown_connections 2024-09-10T23:00:40.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.623+0000 7f254c379740 1 -- 172.21.15.96:0/2031227381 wait complete. 2024-09-10T23:00:40.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.624+0000 7f254c379740 20 client.27944 trim_cache size 0 max 16384 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 Processor -- start 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 -- start start 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48eed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ec5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f253000e420 con 0x559a48ecf5b0 2024-09-10T23:00:40.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f253000ff60 con 0x559a48ec7800 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f254c379740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f253000eaf0 con 0x559a48ec6e00 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f2543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ec5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.645+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40592/0 (socket says 172.21.15.96:40592) 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2542ffd640 1 -- 172.21.15.96:0/2348067217 learned_addr learned my addr 172.21.15.96:0/2348067217 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f25437fe640 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48eed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 -- 172.21.15.96:0/2348067217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 msgr2=0x559a48eed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48eed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 -- 172.21.15.96:0/2348067217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 msgr2=0x559a48eed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 -- 172.21.15.96:0/2348067217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a48ec7be0 con 0x559a48ecf5b0 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2542ffd640 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f2543fff640 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ec5ab0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ee0 tx=0x7f253c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ab0d4ec341aa06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f25297fa640 1 -- 172.21.15.96:0/2348067217 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c04b070 con 0x559a48ecf5b0 2024-09-10T23:00:40.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f25297fa640 1 -- 172.21.15.96:0/2348067217 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c03be30 con 0x559a48ecf5b0 2024-09-10T23:00:40.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.646+0000 7f25297fa640 1 -- 172.21.15.96:0/2348067217 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c033510 con 0x559a48ecf5b0 2024-09-10T23:00:40.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 -- 172.21.15.96:0/2348067217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 msgr2=0x559a48ec5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ec5ab0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ee0 tx=0x7f253c009bb0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 -- 172.21.15.96:0/2348067217 shutdown_connections 2024-09-10T23:00:40.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ec7800 0x559a48eed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ecf5b0 0x559a48ec5ab0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 --2- 172.21.15.96:0/2348067217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48eed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.647+0000 7f254c379740 1 -- 172.21.15.96:0/2348067217 >> 172.21.15.96:0/2348067217 conn(0x559a48ec9380 msgr2=0x559a48ed2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.648+0000 7f254c379740 1 -- 172.21.15.96:0/2348067217 shutdown_connections 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.648+0000 7f254c379740 1 -- 172.21.15.96:0/2348067217 wait complete. 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.648+0000 7f254c379740 1 -- start start 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 Processor -- start 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ee00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ecf5b0 0x559a48f4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f253000f890 con 0x559a48ec7800 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f253000f1c0 con 0x559a48ecf5b0 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f253000c400 con 0x559a48ec6e00 2024-09-10T23:00:40.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f254c379740 10 client.0 _ll_register_callbacks cb 0x7f2534040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50430/0 (socket says 172.21.15.96:50430) 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f25437fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ee00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 -- 172.21.15.96:0/3276350008 learned_addr learned my addr 172.21.15.96:0/3276350008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2543fff640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ecf5b0 0x559a48f4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 -- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 msgr2=0x559a48ee00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ee00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 -- 172.21.15.96:0/3276350008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ecf5b0 msgr2=0x559a48f4b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ecf5b0 0x559a48f4b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.649+0000 7f2542ffd640 1 -- 172.21.15.96:0/3276350008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a48f4bbe0 con 0x559a48ec7800 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2542ffd640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f25380a81a0 tx=0x7f25380a7460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9698319169a473 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:40.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25380ab030 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c03be30 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f25380a7520 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f25380ac2d0 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 10 client.18342 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a48f530f0 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c03be30 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f25380a7720 con 0x559a48ec7800 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 10 client.18342 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 10 client.18342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 20 client.18342 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48f53870 0x559a48f55c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 4 client.18342 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a48e3d8a0 con 0x559a48f53870 2024-09-10T23:00:40.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.650+0000 7f25437fe640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48f53870 0x559a48f55c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f2522ffd640 10 client.18342 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f25437fe640 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48f53870 0x559a48f55c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f33a292791e2e78b server_cookie=986e306af2a327d4 in_seq=0 out_seq=0 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f2522ffd640 1 -- 172.21.15.96:0/3276350008 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a48e3d8a0 con 0x559a48f53870 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f2522ffd640 10 client.18342 handle_command_reply: tid=0 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 client.18342 shutdown 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 msgr2=0x559a48ee0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f25380a81a0 tx=0x7f25380a7460 comp rx=0 tx=0).stop 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 shutdown_connections 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x559a48f53870 0x559a48f55c90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f2522ffd640 0 client.18342 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a48ecf5b0 0x559a48f4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a48ec7800 0x559a48ee0660 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 --2- 172.21.15.96:0/3276350008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a48ec6e00 0x559a48ee00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 >> 172.21.15.96:0/3276350008 conn(0x559a48ec9380 msgr2=0x559a48ece3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 shutdown_connections 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.651+0000 7f254c379740 1 -- 172.21.15.96:0/3276350008 wait complete. 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.652+0000 7f254c379740 20 client.18342 trim_cache size 0 max 16384 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:00:40.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 msgr2=0x7f252c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 0x7f252c0b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2534031440 tx=0x7f2534005bc0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 msgr2=0x7f2544199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f253800d950 tx=0x7f253800de10 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 shutdown_connections 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f252c0b5360 0x7f252c0b7820 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2544108020 0x7f254419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25441076a0 0x7f2544199920 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 --2- 172.21.15.96:0/472421533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2544101c90 0x7f25441993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 >> 172.21.15.96:0/472421533 conn(0x7f25440fd700 msgr2=0x7f254406ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 shutdown_connections 2024-09-10T23:00:40.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:40.653+0000 7f252a7fc640 1 -- 172.21.15.96:0/472421533 wait complete. 2024-09-10T23:00:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 785.0 s, 1760.4 tps, lat 56.806 ms stddev 63.892 2024-09-10T23:00:41.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:00:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:41 smithi096 ceph-mon[30869]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T23:00:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:41 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:42.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:41 smithi121 ceph-mon[35119]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T23:00:42.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:41 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:42.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:41 smithi117 ceph-mon[35081]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 773 op/s 2024-09-10T23:00:42.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:41 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:00:42.226 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:42 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:00:43.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:43.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:43.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:42 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:00:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:00:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:00:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:42 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:00:43.617 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:43 smithi096 ceph-mon[30869]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-09-10T23:00:43.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.745+0000 7fa0751e2640 1 -- 172.21.15.96:0/1485780819 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0318a0 con 0x7fa070101830 2024-09-10T23:00:43.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 -- 172.21.15.96:0/1485780819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 msgr2=0x7fa070101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 --2- 172.21.15.96:0/1485780819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa070101c10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009900 tx=0x7fa06c02a480 comp rx=0 tx=0).stop 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 -- 172.21.15.96:0/1485780819 shutdown_connections 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 --2- 172.21.15.96:0/1485780819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa07010f3e0 0x7fa0701117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 --2- 172.21.15.96:0/1485780819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa070102150 0x7fa07010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 --2- 172.21.15.96:0/1485780819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa070101c10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.746+0000 7fa078448640 1 -- 172.21.15.96:0/1485780819 >> 172.21.15.96:0/1485780819 conn(0x7fa0700fd700 msgr2=0x7fa0700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.747+0000 7fa078448640 1 -- 172.21.15.96:0/1485780819 shutdown_connections 2024-09-10T23:00:43.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.747+0000 7fa078448640 1 -- 172.21.15.96:0/1485780819 wait complete. 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.747+0000 7fa078448640 1 Processor -- start 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.747+0000 7fa078448640 1 -- start start 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa07010df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa07010f3e0 0x7fa0701a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa070105380 con 0x7fa070101830 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa070105200 con 0x7fa070102150 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa078448640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa070105500 con 0x7fa07010f3e0 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa07010df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40624/0 (socket says 172.21.15.96:40624) 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa07010df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50440/0 (socket says 172.21.15.96:50440) 2024-09-10T23:00:43.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 -- 172.21.15.96:0/397025450 learned_addr learned my addr 172.21.15.96:0/397025450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0769e5640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa07010f3e0 0x7fa0701a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 -- 172.21.15.96:0/397025450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa07010f3e0 msgr2=0x7fa0701a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa07010f3e0 0x7fa0701a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 -- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 msgr2=0x7fa07010df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa07010df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.748+0000 7fa0759e3640 1 -- 172.21.15.96:0/397025450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0701a2550 con 0x7fa070102150 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa0759e3640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e990 tx=0x7fa06000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa0677fe640 1 -- 172.21.15.96:0/397025450 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06000cc60 con 0x7fa070102150 2024-09-10T23:00:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0701a2840 con 0x7fa070102150 2024-09-10T23:00:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701ac2f0 con 0x7fa070102150 2024-09-10T23:00:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa0677fe640 1 -- 172.21.15.96:0/397025450 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa06000ce00 con 0x7fa070102150 2024-09-10T23:00:43.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.749+0000 7fa0677fe640 1 -- 172.21.15.96:0/397025450 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060011470 con 0x7fa070102150 2024-09-10T23:00:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.752+0000 7fa0677fe640 1 -- 172.21.15.96:0/397025450 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa06000f040 con 0x7fa070102150 2024-09-10T23:00:43.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.752+0000 7fa07a81d740 1 Processor -- start 2024-09-10T23:00:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 -- start start 2024-09-10T23:00:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251b59a0 0x55ba251b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba2511fea0 con 0x55ba251b59a0 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba2510f2d0 con 0x55ba251a9a10 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa07a81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba251130e0 con 0x55ba251a90f0 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40632/0 (socket says 172.21.15.96:40632) 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0761e4640 1 -- 172.21.15.96:0/1071311031 learned_addr learned my addr 172.21.15.96:0/1071311031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0759e3640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251b59a0 0x55ba251b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0769e5640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0769e5640 1 -- 172.21.15.96:0/1071311031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 msgr2=0x55ba251b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0769e5640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.753+0000 7fa0769e5640 1 -- 172.21.15.96:0/1071311031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251b59a0 msgr2=0x55ba251b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.754+0000 7fa0769e5640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251b59a0 0x55ba251b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.754+0000 7fa0769e5640 1 -- 172.21.15.96:0/1071311031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba24e6d650 con 0x55ba251a90f0 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.754+0000 7fa0677fe640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 0x7fa05c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.755+0000 7fa0761e4640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.755+0000 7fa0761e4640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 0x7fa05c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.755+0000 7fa0677fe640 1 -- 172.21.15.96:0/397025450 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa0600e5970 con 0x7fa070102150 2024-09-10T23:00:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.755+0000 7fa0769e5640 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251a94d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa068009f10 tx=0x7fa068011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ca8dea7b2f64c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.756+0000 7fa0761e4640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 0x7fa05c0b7820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55ba251a7e00 tx=0x7fa06c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.756+0000 7fa064de6640 1 -- 172.21.15.96:0/1071311031 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068004d20 con 0x55ba251a90f0 2024-09-10T23:00:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.756+0000 7fa064de6640 1 -- 172.21.15.96:0/1071311031 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa068004050 con 0x55ba251a90f0 2024-09-10T23:00:43.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.756+0000 7fa064de6640 1 -- 172.21.15.96:0/1071311031 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068005260 con 0x55ba251a90f0 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- 172.21.15.96:0/1071311031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 msgr2=0x55ba251a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251a94d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa068009f10 tx=0x7fa068011810 comp rx=0 tx=0).stop 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- 172.21.15.96:0/1071311031 shutdown_connections 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a9a10 0x55ba251b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251b59a0 0x55ba251b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 --2- 172.21.15.96:0/1071311031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251a94d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- 172.21.15.96:0/1071311031 >> 172.21.15.96:0/1071311031 conn(0x55ba251a6f20 msgr2=0x55ba251a80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- 172.21.15.96:0/1071311031 shutdown_connections 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- 172.21.15.96:0/1071311031 wait complete. 2024-09-10T23:00:43.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.757+0000 7fa07a81d740 1 -- start start 2024-09-10T23:00:43.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 Processor -- start 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 0x55ba251b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50462/0 (socket says 172.21.15.96:50462) 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba24f70700 con 0x55ba251a90f0 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0761e4640 1 -- 172.21.15.96:0/464320974 learned_addr learned my addr 172.21.15.96:0/464320974 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba250aa160 con 0x55ba251b59a0 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba251b8620 con 0x55ba251a9a10 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0759e3640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 0x55ba251b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa07a81d740 10 client.0 _ll_register_callbacks cb 0x55ba250c39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0769e5640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0759e3640 1 -- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 msgr2=0x55ba251b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.758+0000 7fa0759e3640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa0759e3640 1 -- 172.21.15.96:0/464320974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 msgr2=0x55ba251b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa0759e3640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa0759e3640 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba250d7c90 con 0x55ba251a9a10 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa0761e4640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa0759e3640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 0x55ba251b2650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa0600e3540 tx=0x7fa0600a8110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4066e85bc6a7459b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06001f070 con 0x55ba251a9a10 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa0600ac410 con 0x55ba251a9a10 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa068004050 con 0x55ba251a9a10 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0600ab920 con 0x55ba251a9a10 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 10 client.28030 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:43.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ba251c3f40 con 0x55ba251a9a10 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa068004050 con 0x55ba251a9a10 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa0600b52b0 con 0x55ba251a9a10 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 10 client.28030 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 10 client.28030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.759+0000 7fa07a81d740 20 client.28030 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba251c5350 0x55ba251c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa07a81d740 4 client.28030 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba25121df0 con 0x55ba251c5350 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa0761e4640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba251c5350 0x55ba251c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa0761e4640 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba251c5350 0x55ba251c7740 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ae99f0b5415477 server_cookie=618224d7c87a990e in_seq=0 out_seq=0 2024-09-10T23:00:43.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.760+0000 7fa059ffb640 10 client.28030 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:43.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.761+0000 7fa059ffb640 1 -- 172.21.15.96:0/464320974 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba25121df0 con 0x55ba251c5350 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.761+0000 7fa059ffb640 10 client.28030 handle_command_reply: tid=0 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.761+0000 7fa07a81d740 1 client.28030 shutdown 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.761+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 msgr2=0x55ba251b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.761+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 0x55ba251b2650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa0600e3540 tx=0x7fa0600a8110 comp rx=0 tx=0).stop 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 shutdown_connections 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba251c5350 0x55ba251c7740 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa059ffb640 0 client.28030 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a9a10 0x55ba251b2650 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 --2- 172.21.15.96:0/464320974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a90f0 0x55ba251b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 >> 172.21.15.96:0/464320974 conn(0x55ba251a6f20 msgr2=0x55ba251aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 shutdown_connections 2024-09-10T23:00:43.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 1 -- 172.21.15.96:0/464320974 wait complete. 2024-09-10T23:00:43.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.762+0000 7fa07a81d740 20 client.28030 trim_cache size 0 max 16384 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 Processor -- start 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 -- start start 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 0x55ba251c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 0x55ba251b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400e280 con 0x55ba251a9a10 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400fdc0 con 0x55ba251a90f0 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa07a81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400e950 con 0x55ba251b59a0 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 0x55ba251c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 0x55ba251c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:40660/0 (socket says 172.21.15.96:40660) 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa0761e4640 1 -- 172.21.15.96:0/2874004127 learned_addr learned my addr 172.21.15.96:0/2874004127 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.778+0000 7fa0769e5640 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 0x55ba251b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 -- 172.21.15.96:0/2874004127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 msgr2=0x55ba251c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 0x55ba251c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 -- 172.21.15.96:0/2874004127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 msgr2=0x55ba251b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 -- 172.21.15.96:0/2874004127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba251c7d70 con 0x55ba251b59a0 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa0769e5640 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 0x55ba251b1410 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa068000f80 tx=0x7fa068003aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac02a3e592a48e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa05bfff640 1 -- 172.21.15.96:0/2874004127 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068003d90 con 0x55ba251b59a0 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa05bfff640 1 -- 172.21.15.96:0/2874004127 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0680054f0 con 0x55ba251b59a0 2024-09-10T23:00:43.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.779+0000 7fa05bfff640 1 -- 172.21.15.96:0/2874004127 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680199f0 con 0x55ba251b59a0 2024-09-10T23:00:43.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 -- 172.21.15.96:0/2874004127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 msgr2=0x55ba251b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 0x55ba251b1410 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa068000f80 tx=0x7fa068003aa0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 -- 172.21.15.96:0/2874004127 shutdown_connections 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251b59a0 0x55ba251b1410 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 --2- 172.21.15.96:0/2874004127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251a90f0 0x55ba251c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 -- 172.21.15.96:0/2874004127 >> 172.21.15.96:0/2874004127 conn(0x55ba251a6f20 msgr2=0x55ba251b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 -- 172.21.15.96:0/2874004127 shutdown_connections 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.780+0000 7fa07a81d740 1 -- 172.21.15.96:0/2874004127 wait complete. 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 -- start start 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 Processor -- start 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400f6f0 con 0x55ba251a9a10 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400f020 con 0x55ba251b59a0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa05400c260 con 0x55ba251a90f0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa0759e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.781+0000 7fa07a81d740 10 client.0 _ll_register_callbacks cb 0x7fa0600eb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0769e5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:45516/0 (socket says 172.21.15.96:45516) 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 -- 172.21.15.96:0/3705033709 learned_addr learned my addr 172.21.15.96:0/3705033709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 -- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 msgr2=0x55ba251b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 -- 172.21.15.96:0/3705033709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 msgr2=0x55ba251be070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251be070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 -- 172.21.15.96:0/3705033709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba251b9040 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa0761e4640 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa06c031a00 tx=0x7fa06c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbe91017d49feaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c05e060 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa06c005470 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.782+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0680054f0 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa06c035040 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 10 client.28036 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ba25232c70 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa06c005670 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa06c063070 con 0x55ba251a90f0 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 10 client.28036 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 10 client.28036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 20 client.28036 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:43.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba25233400 0x55ba25235820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 4 client.28036 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa0680054f0 con 0x55ba25233400 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.783+0000 7fa0759e3640 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba25233400 0x55ba25235820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa059ffb640 10 client.28036 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa0759e3640 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba25233400 0x55ba25235820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d9e19d196fee11 server_cookie=39973c2989815ede in_seq=0 out_seq=0 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa059ffb640 1 -- 172.21.15.96:0/3705033709 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa0680054f0 con 0x55ba25233400 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa059ffb640 10 client.28036 handle_command_reply: tid=0 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 client.28036 shutdown 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 msgr2=0x55ba251bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa06c031a00 tx=0x7fa06c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 shutdown_connections 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55ba25233400 0x55ba25235820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba251b59a0 0x55ba251b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba251a9a10 0x55ba251be070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 --2- 172.21.15.96:0/3705033709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba251a90f0 0x55ba251bdb30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 >> 172.21.15.96:0/3705033709 conn(0x55ba251a6f20 msgr2=0x55ba251b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa059ffb640 0 client.28036 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 shutdown_connections 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 1 -- 172.21.15.96:0/3705033709 wait complete. 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.784+0000 7fa07a81d740 20 client.28036 trim_cache size 0 max 16384 2024-09-10T23:00:43.792 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 msgr2=0x7fa05c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 0x7fa05c0b7820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55ba251a7e00 tx=0x7fa06c00b040 comp rx=0 tx=0).stop 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 msgr2=0x7fa07010e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e990 tx=0x7fa06000ed90 comp rx=0 tx=0).stop 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 shutdown_connections 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa070102150 0x7fa07010e4f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa05c0b5360 0x7fa05c0b7820 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa07010f3e0 0x7fa0701a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 --2- 172.21.15.96:0/397025450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa070101830 0x7fa07010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:43.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 >> 172.21.15.96:0/397025450 conn(0x7fa0700fd700 msgr2=0x7fa07010fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:43.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 shutdown_connections 2024-09-10T23:00:43.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:43.785+0000 7fa078448640 1 -- 172.21.15.96:0/397025450 wait complete. 2024-09-10T23:00:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:43 smithi121 ceph-mon[35119]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-09-10T23:00:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:43 smithi117 ceph-mon[35081]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-09-10T23:00:45.947 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:00:45.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:00:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 790.0 s, 1227.0 tps, lat 81.486 ms stddev 129.994 2024-09-10T23:00:45.992 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:45 smithi096 ceph-mon[30869]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-10T23:00:46.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:45 smithi121 ceph-mon[35119]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-10T23:00:46.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:45 smithi117 ceph-mon[35081]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.06k op/s 2024-09-10T23:00:46.296 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.792+0000 7f6a1327c640 1 -- 172.21.15.96:0/1030087861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a150 msgr2=0x7f6a0c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.792+0000 7f6a1327c640 1 --2- 172.21.15.96:0/1030087861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1111b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009860 tx=0x7f69f802a3d0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.793+0000 7f6a1327c640 1 -- 172.21.15.96:0/1030087861 shutdown_connections 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.793+0000 7f6a1327c640 1 --2- 172.21.15.96:0/1030087861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.793+0000 7f6a1327c640 1 --2- 172.21.15.96:0/1030087861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c10a530 0x7f6a0c10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.793+0000 7f6a1327c640 1 --2- 172.21.15.96:0/1030087861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1111b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.793+0000 7f6a1327c640 1 -- 172.21.15.96:0/1030087861 >> 172.21.15.96:0/1030087861 conn(0x7f6a0c06dfc0 msgr2=0x7f6a0c06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.795+0000 7f6a1327c640 1 -- 172.21.15.96:0/1030087861 shutdown_connections 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.798+0000 7f6a1327c640 1 -- 172.21.15.96:0/1030087861 wait complete. 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.798+0000 7f6a1327c640 1 Processor -- start 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 -- start start 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1a8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a530 0x7f6a0c1a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c1ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c121bb0 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c121a30 con 0x7f6a0c10a530 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.802+0000 7f6a1327c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c121d30 con 0x7f6a0c10a150 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.805+0000 7f6a11018640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.805+0000 7f6a11018640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43138/0 (socket says 172.21.15.96:43138) 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.805+0000 7f6a11018640 1 -- 172.21.15.96:0/3865850981 learned_addr learned my addr 172.21.15.96:0/3865850981 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.805+0000 7f6a11819640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c1ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.807+0000 7f6a10817640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a530 0x7f6a0c1a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.811+0000 7f6a11819640 1 -- 172.21.15.96:0/3865850981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 msgr2=0x7f6a0c1a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.811+0000 7f6a11819640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.811+0000 7f6a11819640 1 -- 172.21.15.96:0/3865850981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a530 msgr2=0x7f6a0c1a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.811+0000 7f6a11819640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a530 0x7f6a0c1a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.811+0000 7f6a11819640 1 -- 172.21.15.96:0/3865850981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c1ae130 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.812+0000 7f6a11819640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c1ad960 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e970 tx=0x7f6a0400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.815+0000 7f6a0a7fc640 1 -- 172.21.15.96:0/3865850981 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400ccc0 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.815+0000 7f6a0a7fc640 1 -- 172.21.15.96:0/3865850981 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04005020 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.815+0000 7f6a0a7fc640 1 -- 172.21.15.96:0/3865850981 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004a00 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.815+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c1ae420 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.815+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c111880 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.822+0000 7f6a15651740 1 Processor -- start 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.823+0000 7f6a0a7fc640 1 -- 172.21.15.96:0/3865850981 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f6a0401f030 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.830+0000 7f6a15651740 1 -- start start 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.830+0000 7f6a15651740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4b9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.830+0000 7f6a15651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4b9f800 0x55d4f4bab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a15651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a15651740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4b15e60 con 0x55d4f4b9eee0 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a15651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4b05290 con 0x55d4f4b9f800 2024-09-10T23:00:46.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a10817640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a10817640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43154/0 (socket says 172.21.15.96:43154) 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a10817640 1 -- 172.21.15.96:0/1219468054 learned_addr learned my addr 172.21.15.96:0/1219468054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4b090a0 con 0x55d4f4bab790 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.831+0000 7f6a10817640 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4b9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 -- 172.21.15.96:0/1219468054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4b9f800 msgr2=0x55d4f4bab250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4b9f800 0x55d4f4bab250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 -- 172.21.15.96:0/1219468054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 msgr2=0x55d4f4b9f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4b9f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 -- 172.21.15.96:0/1219468054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f4863930 con 0x55d4f4bab790 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.832+0000 7f6a10817640 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000fa00 tx=0x7f6a00009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af1e5e91292e49bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.833+0000 7f69f77fe640 1 -- 172.21.15.96:0/1219468054 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004b20 con 0x55d4f4bab790 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.834+0000 7f6a0a7fc640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 0x7f69fc0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.835+0000 7f6a11018640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 0x7f69fc0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.835+0000 7f69f77fe640 1 -- 172.21.15.96:0/1219468054 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a00003e20 con 0x55d4f4bab790 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.835+0000 7f69f77fe640 1 -- 172.21.15.96:0/1219468054 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004020 con 0x55d4f4bab790 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.836+0000 7f6a0a7fc640 1 -- 172.21.15.96:0/3865850981 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6a040e5640 con 0x7f6a0c11c0a0 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.837+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 msgr2=0x55d4f4badb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.837+0000 7f6a15651740 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000fa00 tx=0x7f6a00009a80 comp rx=0 tx=0).stop 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 shutdown_connections 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4b9f800 0x55d4f4bab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4badb80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 --2- 172.21.15.96:0/1219468054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4b9f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 >> 172.21.15.96:0/1219468054 conn(0x55d4f4b9cd10 msgr2=0x55d4f4b9deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a11018640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 0x7f69fc0b7970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f69f8032ff0 tx=0x7f69f8005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.838+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 shutdown_connections 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.839+0000 7f6a15651740 1 -- 172.21.15.96:0/1219468054 wait complete. 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.839+0000 7f6a15651740 1 -- start start 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.839+0000 7f6a15651740 1 Processor -- start 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4ba8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54012/0 (socket says 172.21.15.96:54012) 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a11018640 1 -- 172.21.15.96:0/629571258 learned_addr learned my addr 172.21.15.96:0/629571258 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4966890 con 0x55d4f4b9eee0 2024-09-10T23:00:46.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4a9fd40 con 0x55d4f4bab790 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4bae440 con 0x55d4f4b9f800 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a10817640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4ba8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.840+0000 7f6a15651740 10 client.0 _ll_register_callbacks cb 0x55d4f4b03c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 -- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 msgr2=0x55d4f4ba91a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba91a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 -- 172.21.15.96:0/629571258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 msgr2=0x55d4f4ba8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f4bb7880 con 0x55d4f4b9f800 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.841+0000 7f6a10817640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4ba8c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006010 tx=0x7f6a0000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d861f42dac478dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f69f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0002a060 con 0x55d4f4b9f800 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6a00003b80 con 0x55d4f4b9f800 2024-09-10T23:00:46.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a00003e20 con 0x55d4f4b9f800 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a00004a40 con 0x55d4f4b9f800 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f6a15651740 10 client.28048 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.842+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4f4bba220 con 0x55d4f4b9f800 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a00004be0 con 0x55d4f4b9f800 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a0002f070 con 0x55d4f4b9f800 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 10 client.28048 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 10 client.28048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 20 client.28048 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4bbb630 0x55d4f4bbda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 4 client.28048 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:46.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.843+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a00003e20 con 0x55d4f4bbb630 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.844+0000 7f6a11018640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4bbb630 0x55d4f4bbda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.845+0000 7f6a11018640 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4bbb630 0x55d4f4bbda20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9863764af283239 server_cookie=47a0257f277c23c7 in_seq=0 out_seq=0 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.845+0000 7f69f4ff9640 10 client.28048 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.846+0000 7f69f4ff9640 1 -- 172.21.15.96:0/629571258 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6a00003e20 con 0x55d4f4bbb630 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.846+0000 7f69f4ff9640 10 client.28048 handle_command_reply: tid=0 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.846+0000 7f6a15651740 1 client.28048 shutdown 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 msgr2=0x55d4f4ba8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4ba8c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006010 tx=0x7f6a0000cf90 comp rx=0 tx=0).stop 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 shutdown_connections 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4bbb630 0x55d4f4bbda20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f69f4ff9640 0 client.28048 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4ba8c60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 --2- 172.21.15.96:0/629571258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 >> 172.21.15.96:0/629571258 conn(0x55d4f4b9cd10 msgr2=0x55d4f4b9bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 shutdown_connections 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 1 -- 172.21.15.96:0/629571258 wait complete. 2024-09-10T23:00:46.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.847+0000 7f6a15651740 20 client.28048 trim_cache size 0 max 16384 2024-09-10T23:00:46.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.873+0000 7f6a15651740 1 Processor -- start 2024-09-10T23:00:46.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.873+0000 7f6a15651740 1 -- start start 2024-09-10T23:00:46.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000e300 con 0x55d4f4b9eee0 2024-09-10T23:00:46.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000fe40 con 0x55d4f4bab790 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a15651740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000e9d0 con 0x55d4f4b9f800 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54028/0 (socket says 172.21.15.96:54028) 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 -- 172.21.15.96:0/1433118659 learned_addr learned my addr 172.21.15.96:0/1433118659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a10817640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11819640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 -- 172.21.15.96:0/1433118659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 msgr2=0x55d4f4bb6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 -- 172.21.15.96:0/1433118659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 msgr2=0x55d4f4ba5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a11018640 1 -- 172.21.15.96:0/1433118659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f4b9d790 con 0x55d4f4b9eee0 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.874+0000 7f6a10817640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:46.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.875+0000 7f6a11018640 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a9c0 tx=0x7f69f803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a96c5ee16e4648 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.875+0000 7f69f6ffd640 1 -- 172.21.15.96:0/1433118659 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8057070 con 0x55d4f4b9eee0 2024-09-10T23:00:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.875+0000 7f69f6ffd640 1 -- 172.21.15.96:0/1433118659 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f8005040 con 0x55d4f4b9eee0 2024-09-10T23:00:46.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.875+0000 7f69f6ffd640 1 -- 172.21.15.96:0/1433118659 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8004bf0 con 0x55d4f4b9eee0 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 -- 172.21.15.96:0/1433118659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 msgr2=0x55d4f4ba2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a9c0 tx=0x7f69f803f000 comp rx=0 tx=0).stop 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 -- 172.21.15.96:0/1433118659 shutdown_connections 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bab790 0x55d4f4ba5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 --2- 172.21.15.96:0/1433118659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9eee0 0x55d4f4ba2f50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 -- 172.21.15.96:0/1433118659 >> 172.21.15.96:0/1433118659 conn(0x55d4f4b9cd10 msgr2=0x55d4f4bb66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 -- 172.21.15.96:0/1433118659 shutdown_connections 2024-09-10T23:00:46.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.876+0000 7f6a15651740 1 -- 172.21.15.96:0/1433118659 wait complete. 2024-09-10T23:00:46.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.877+0000 7f6a15651740 1 -- start start 2024-09-10T23:00:46.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.877+0000 7f6a15651740 1 Processor -- start 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.878+0000 7f6a15651740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.878+0000 7f6a15651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a15651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bbb630 0x55d4f4bb0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a11018640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54042/0 (socket says 172.21.15.96:54042) 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43176/0 (socket says 172.21.15.96:43176) 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a11018640 1 -- 172.21.15.96:0/691068041 learned_addr learned my addr 172.21.15.96:0/691068041 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000f770 con 0x55d4f4b9f800 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000f0a0 con 0x55d4f4bbb630 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000c2e0 con 0x55d4f4bab790 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a11819640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bbb630 0x55d4f4bb0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a15651740 10 client.0 _ll_register_callbacks cb 0x7f6a0000f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 -- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bbb630 msgr2=0x55d4f4bb0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bbb630 0x55d4f4bb0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 -- 172.21.15.96:0/691068041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 msgr2=0x55d4f4bb3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.879+0000 7f6a10817640 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f4bb10a0 con 0x55d4f4bab790 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f6a10817640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000f490 tx=0x7f6a0000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30d39ae8659bccb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f69f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00035070 con 0x55d4f4bab790 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f6a00004df0 con 0x55d4f4bab790 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f8005040 con 0x55d4f4bab790 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a0000cbc0 con 0x55d4f4bab790 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f6a15651740 10 client.28054 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:46.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4f4c29070 con 0x55d4f4bab790 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.880+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f8005040 con 0x55d4f4bab790 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a0002f040 con 0x55d4f4bab790 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 10 client.28054 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 10 client.28054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 20 client.28054 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4c29700 0x55d4f4c2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 4 client.28054 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d4f4b17db0 con 0x55d4f4c29700 2024-09-10T23:00:46.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.881+0000 7f6a11018640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4c29700 0x55d4f4c2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.882+0000 7f69f4ff9640 10 client.28054 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.882+0000 7f6a11018640 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4c29700 0x55d4f4c2bb20 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b5cb34f98994a9 server_cookie=f5485a20ed0cbe46 in_seq=0 out_seq=0 2024-09-10T23:00:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.883+0000 7f69f4ff9640 1 -- 172.21.15.96:0/691068041 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d4f4b17db0 con 0x55d4f4c29700 2024-09-10T23:00:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.884+0000 7f69f4ff9640 10 client.28054 handle_command_reply: tid=0 2024-09-10T23:00:46.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.884+0000 7f6a15651740 1 client.28054 shutdown 2024-09-10T23:00:46.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 msgr2=0x55d4f4bb4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000f490 tx=0x7f6a0000f690 comp rx=0 tx=0).stop 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 shutdown_connections 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55d4f4c29700 0x55d4f4c2bb20 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d4f4bbb630 0x55d4f4bb0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4bab790 0x55d4f4bb4090 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000f490 tx=0x7f6a0000f690 comp rx=0 tx=0).stop 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 --2- 172.21.15.96:0/691068041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4f4b9f800 0x55d4f4bb3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 >> 172.21.15.96:0/691068041 conn(0x55d4f4b9cd10 msgr2=0x55d4f4b9bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f69f4ff9640 0 client.28054 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:46.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.886+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 shutdown_connections 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.887+0000 7f6a15651740 1 -- 172.21.15.96:0/691068041 wait complete. 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.887+0000 7f6a15651740 20 client.28054 trim_cache size 0 max 16384 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "b3181c62-cfec-4c60-99c6-52f389cd7930", 2024-09-10T23:00:46.889 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T23:00:46.890 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:00:46.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 msgr2=0x7f69fc0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 0x7f69fc0b7970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f69f8032ff0 tx=0x7f69f8005cb0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 msgr2=0x7f6a0c1ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:46.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c1ad960 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e970 tx=0x7f6a0400ee30 comp rx=0 tx=0).stop 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 shutdown_connections 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a530 0x7f6a0c1a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f69fc0b54b0 0x7f69fc0b7970 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a0c11c0a0 0x7f6a0c1ad960 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 --2- 172.21.15.96:0/3865850981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a0c10a150 0x7f6a0c1a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 >> 172.21.15.96:0/3865850981 conn(0x7f6a0c06dfc0 msgr2=0x7f6a0c11c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 shutdown_connections 2024-09-10T23:00:46.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:46.888+0000 7f6a1327c640 1 -- 172.21.15.96:0/3865850981 wait complete. 2024-09-10T23:00:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-09-10T23:00:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T23:00:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:47.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:00:47.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:00:47.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:47 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:47.827 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-09-10T23:00:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T23:00:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:00:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:00:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:47 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:47.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:00:48.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-09-10T23:00:48.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T23:00:48.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:48.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:00:48.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:00:48.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:47 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:48.261 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:00:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 -- 172.21.15.96:0/462440793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102b20 msgr2=0x7f80981094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 --2- 172.21.15.96:0/462440793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102b20 0x7f80981094a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f808800aa40 tx=0x7f808802b750 comp rx=0 tx=0).stop 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 -- 172.21.15.96:0/462440793 shutdown_connections 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 --2- 172.21.15.96:0/462440793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102b20 0x7f80981094a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 --2- 172.21.15.96:0/462440793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098102160 0x7f80981025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 --2- 172.21.15.96:0/462440793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098101010 0x7f8098101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.415+0000 7f80a0452640 1 -- 172.21.15.96:0/462440793 >> 172.21.15.96:0/462440793 conn(0x7f80980fc8f0 msgr2=0x7f80980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- 172.21.15.96:0/462440793 shutdown_connections 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- 172.21.15.96:0/462440793 wait complete. 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 Processor -- start 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- start start 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 0x7f8098199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 0x7f8098199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f809810c410 con 0x7f8098102160 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f809810c290 con 0x7f8098102b20 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f80a0452640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f809810c590 con 0x7f8098101010 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f809d9ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 0x7f8098199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43206/0 (socket says 172.21.15.96:43206) 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.416+0000 7f809e1ee640 1 -- 172.21.15.96:0/1219818989 learned_addr learned my addr 172.21.15.96:0/1219818989 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 -- 172.21.15.96:0/1219818989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 msgr2=0x7f8098199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e9ef640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 0x7f8098199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 0x7f8098199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 -- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 msgr2=0x7f8098199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 0x7f8098199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 -- 172.21.15.96:0/1219818989 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809819a880 con 0x7f8098101010 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809d9ed640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 0x7f8098199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e9ef640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 0x7f8098199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f809e1ee640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9f0 tx=0x7f809400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f808f7fe640 1 -- 172.21.15.96:0/1219818989 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809400cd80 con 0x7f8098101010 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809819ab70 con 0x7f8098101010 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f808f7fe640 1 -- 172.21.15.96:0/1219818989 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940108e0 con 0x7f8098101010 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f808f7fe640 1 -- 172.21.15.96:0/1219818989 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094018980 con 0x7f8098101010 2024-09-10T23:00:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.417+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8098103650 con 0x7f8098101010 2024-09-10T23:00:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.421+0000 7f808f7fe640 1 -- 172.21.15.96:0/1219818989 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f8094018b20 con 0x7f8098101010 2024-09-10T23:00:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.422+0000 7f808f7fe640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 0x7f80840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 Processor -- start 2024-09-10T23:00:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809d9ed640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 0x7f80840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f808f7fe640 1 -- 172.21.15.96:0/1219818989 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f80940e5f00 con 0x7f8098101010 2024-09-10T23:00:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 -- start start 2024-09-10T23:00:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809d9ed640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 0x7f80840b77a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8090005fd0 tx=0x7f8090009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d897896f0 con 0x563d89817860 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d89778b20 con 0x563d8981cbd0 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f80a3220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563d8977c930 con 0x563d8981a190 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809e9ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809e9ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:54872/0 (socket says 172.21.15.96:54872) 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809e9ef640 1 -- 172.21.15.96:0/667375483 learned_addr learned my addr 172.21.15.96:0/667375483 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.423+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54062/0 (socket says 172.21.15.96:54062) 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809d9ed640 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 -- 172.21.15.96:0/667375483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 msgr2=0x563d8981c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 -- 172.21.15.96:0/667375483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 msgr2=0x563d89819c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 -- 172.21.15.96:0/667375483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8973e530 con 0x563d8981cbd0 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e1ee640 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f809e9ef640 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f808800a7b0 tx=0x7f8088030c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dd56ef4d7ac56e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f808cbfe640 1 -- 172.21.15.96:0/667375483 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088036040 con 0x563d8981cbd0 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.424+0000 7f808cbfe640 1 -- 172.21.15.96:0/667375483 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8088004040 con 0x563d8981cbd0 2024-09-10T23:00:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.425+0000 7f808cbfe640 1 -- 172.21.15.96:0/667375483 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880055a0 con 0x563d8981cbd0 2024-09-10T23:00:48.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- 172.21.15.96:0/667375483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 msgr2=0x563d8981f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f808800a7b0 tx=0x7f8088030c20 comp rx=0 tx=0).stop 2024-09-10T23:00:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- 172.21.15.96:0/667375483 shutdown_connections 2024-09-10T23:00:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d8981f090 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- 172.21.15.96:0/667375483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89819c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- 172.21.15.96:0/667375483 >> 172.21.15.96:0/667375483 conn(0x563d898129b0 msgr2=0x563d89813740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- 172.21.15.96:0/667375483 shutdown_connections 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- 172.21.15.96:0/667375483 wait complete. 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 -- start start 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 Processor -- start 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.426+0000 7f80a3220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 0x563d89821530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89822100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d896b0ab0 con 0x563d89817860 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d8971e730 con 0x563d8981a190 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563d8981fb50 con 0x563d8981cbd0 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 10 client.0 _ll_register_callbacks cb 0x563d897726b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54066/0 (socket says 172.21.15.96:54066) 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 -- 172.21.15.96:0/706495555 learned_addr learned my addr 172.21.15.96:0/706495555 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809d9ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 0x563d89821530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e9ef640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89822100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 -- 172.21.15.96:0/706495555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 msgr2=0x563d89822100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89822100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 -- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 msgr2=0x563d89821530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 0x563d89821530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 -- 172.21.15.96:0/706495555 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d894da790 con 0x563d89817860 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809d9ed640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 0x563d89821530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f809e1ee640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f809400c6a0 tx=0x7f80940acfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae56b097c16f524f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f8081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940b1020 con 0x563d89817860 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f80940ab9c0 con 0x563d89817860 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8088004040 con 0x563d89817860 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80940a6e60 con 0x563d89817860 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 10 client.18384 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.427+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d89553620 con 0x563d89817860 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8088004040 con 0x563d89817860 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f80940ada90 con 0x563d89817860 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 10 client.18384 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 20 client.18384 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8982c7e0 0x563d8982ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 4 client.18384 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8978b640 con 0x563d8982c7e0 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f809d9ed640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8982c7e0 0x563d8982ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f8081ffb640 10 client.18384 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.428+0000 7f809d9ed640 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8982c7e0 0x563d8982ebd0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518fb5601d5be802 server_cookie=e1c204e38a998df8 in_seq=0 out_seq=0 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f8081ffb640 1 -- 172.21.15.96:0/706495555 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d8978b640 con 0x563d8982c7e0 2024-09-10T23:00:48.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f8081ffb640 10 client.18384 handle_command_reply: tid=0 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 client.18384 shutdown 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 msgr2=0x563d89820e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f809400c6a0 tx=0x7f80940acfa0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 shutdown_connections 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8982c7e0 0x563d8982ebd0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981a190 0x563d89821530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89822100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 --2- 172.21.15.96:0/706495555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d89820e90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 >> 172.21.15.96:0/706495555 conn(0x563d898129b0 msgr2=0x563d89811bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f8081ffb640 0 client.18384 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.429+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 shutdown_connections 2024-09-10T23:00:48.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.430+0000 7f80a3220740 1 -- 172.21.15.96:0/706495555 wait complete. 2024-09-10T23:00:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.430+0000 7f80a3220740 20 client.18384 trim_cache size 0 max 16384 2024-09-10T23:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 Processor -- start 2024-09-10T23:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 -- start start 2024-09-10T23:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d898147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a7af0 con 0x563d89817860 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a9630 con 0x563d8981cbd0 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f80a3220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a81c0 con 0x563d8981a190 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.448+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54074/0 (socket says 172.21.15.96:54074) 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809d9ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 -- 172.21.15.96:0/2048534419 learned_addr learned my addr 172.21.15.96:0/2048534419 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e9ef640 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d898147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 -- 172.21.15.96:0/2048534419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 msgr2=0x563d8981d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 -- 172.21.15.96:0/2048534419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 msgr2=0x563d898147e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d898147e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 -- 172.21.15.96:0/2048534419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d89813c10 con 0x563d89817860 2024-09-10T23:00:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809d9ed640 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f809e1ee640 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f80940e6220 tx=0x7f8094000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ac6522f197e50a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f8083fff640 1 -- 172.21.15.96:0/2048534419 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940b1040 con 0x563d89817860 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f8083fff640 1 -- 172.21.15.96:0/2048534419 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940b24c0 con 0x563d89817860 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.449+0000 7f8083fff640 1 -- 172.21.15.96:0/2048534419 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940a8420 con 0x563d89817860 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- 172.21.15.96:0/2048534419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 msgr2=0x563d8981cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f80940e6220 tx=0x7f8094000bc0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- 172.21.15.96:0/2048534419 shutdown_connections 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d8981cbd0 0x563d898147e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981a190 0x563d8981d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 --2- 172.21.15.96:0/2048534419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d89817860 0x563d8981cfb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- 172.21.15.96:0/2048534419 >> 172.21.15.96:0/2048534419 conn(0x563d898129b0 msgr2=0x563d89816d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- 172.21.15.96:0/2048534419 shutdown_connections 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- 172.21.15.96:0/2048534419 wait complete. 2024-09-10T23:00:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 -- start start 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.450+0000 7f80a3220740 1 Processor -- start 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d89817860 0x563d89828c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89828160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a8f60 con 0x563d8981a190 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a8890 con 0x563d89817860 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a5ad0 con 0x563d8981cbd0 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f80a3220740 10 client.0 _ll_register_callbacks cb 0x7f80940ac5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809e1ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d89817860 0x563d89828c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54080/0 (socket says 172.21.15.96:54080) 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 -- 172.21.15.96:0/921902345 learned_addr learned my addr 172.21.15.96:0/921902345 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809e9ef640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89828160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 -- 172.21.15.96:0/921902345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 msgr2=0x563d89828160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89828160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 -- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d89817860 msgr2=0x563d89828c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d89817860 0x563d89828c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 -- 172.21.15.96:0/921902345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d89819fc0 con 0x563d8981a190 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809e9ef640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89828160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f809d9ed640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x563d8982f410 tx=0x7f8090004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a56f96f7eca822d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.451+0000 7f8081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090035060 con 0x563d8981a190 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f809002a790 con 0x563d8981a190 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80940b24c0 con 0x563d8981a190 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f809002a990 con 0x563d8981a190 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 10 client.18390 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d898998f0 con 0x563d8981a190 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809002ab30 con 0x563d8981a190 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f809003a070 con 0x563d8981a190 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 10 client.18390 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 10 client.18390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 20 client.18390 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8989a300 0x563d8989c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 4 client.18390 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f80940b24c0 con 0x563d8989a300 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.452+0000 7f809e1ee640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8989a300 0x563d8989c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.454+0000 7f809e1ee640 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8989a300 0x563d8989c720 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73de9ecc699ecf1 server_cookie=5e72d6f011336cb in_seq=0 out_seq=0 2024-09-10T23:00:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.454+0000 7f8081ffb640 10 client.18390 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.508+0000 7f8081ffb640 1 -- 172.21.15.96:0/921902345 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f80940b24c0 con 0x563d8989a300 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.508+0000 7f8081ffb640 10 client.18390 handle_command_reply: tid=0 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.508+0000 7f80a3220740 1 client.18390 shutdown 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.508+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 msgr2=0x563d89827a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.508+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x563d8982f410 tx=0x7f8090004ab0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 shutdown_connections 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563d8989a300 0x563d8989c720 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563d8981cbd0 0x563d89828160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f8081ffb640 0 client.18390 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d8981a190 0x563d89827a90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 --2- 172.21.15.96:0/921902345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d89817860 0x563d89828c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 >> 172.21.15.96:0/921902345 conn(0x563d898129b0 msgr2=0x563d89816920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 shutdown_connections 2024-09-10T23:00:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 1 -- 172.21.15.96:0/921902345 wait complete. 2024-09-10T23:00:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a3220740 20 client.18390 trim_cache size 0 max 16384 2024-09-10T23:00:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:00:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:00:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:00:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 msgr2=0x7f80840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 0x7f80840b77a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8090005fd0 tx=0x7f8090009500 comp rx=0 tx=0).stop 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 msgr2=0x7f8098198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f809400e9f0 tx=0x7f809400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 shutdown_connections 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102b20 0x7f8098199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f80840b52e0 0x7f80840b77a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8098102160 0x7f8098199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 --2- 172.21.15.96:0/1219818989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8098101010 0x7f8098198d40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 >> 172.21.15.96:0/1219818989 conn(0x7f80980fc8f0 msgr2=0x7f8098105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.509+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 shutdown_connections 2024-09-10T23:00:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:00:48.510+0000 7f80a0452640 1 -- 172.21.15.96:0/1219818989 wait complete. 2024-09-10T23:00:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:48 smithi096 ceph-mon[30869]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.58k op/s 2024-09-10T23:00:48.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:48 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:48.844 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:48 smithi117 ceph-mon[35081]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.58k op/s 2024-09-10T23:00:48.844 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:48 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:49.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:48 smithi121 ceph-mon[35119]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.58k op/s 2024-09-10T23:00:49.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:48 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.200+0000 7fe5fc899640 1 -- 172.21.15.96:0/3840796704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f810a3b0 msgr2=0x7fe5f810a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.200+0000 7fe5fc899640 1 --2- 172.21.15.96:0/3840796704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f810a3b0 0x7fe5f810a790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000bc00 tx=0x7fe5f002e690 comp rx=0 tx=0).stop 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 -- 172.21.15.96:0/3840796704 shutdown_connections 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 --2- 172.21.15.96:0/3840796704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f8113ae0 0x7fe5f8115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 --2- 172.21.15.96:0/3840796704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5f81111b0 0x7fe5f81135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 --2- 172.21.15.96:0/3840796704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f810a3b0 0x7fe5f810a790 secure :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000bc00 tx=0x7fe5f002e690 comp rx=0 tx=0).stop 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 -- 172.21.15.96:0/3840796704 >> 172.21.15.96:0/3840796704 conn(0x7fe5f806cf20 msgr2=0x7fe5f806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.201+0000 7fe5fc899640 1 -- 172.21.15.96:0/3840796704 shutdown_connections 2024-09-10T23:00:49.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.205+0000 7fe5fc899640 1 -- 172.21.15.96:0/3840796704 wait complete. 2024-09-10T23:00:49.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 Processor -- start 2024-09-10T23:00:49.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 -- start start 2024-09-10T23:00:49.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 0x7fe5f81a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 0x7fe5f81a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5f81ad9a0 0x7fe5f81ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f8119260 con 0x7fe5f81ad9a0 2024-09-10T23:00:49.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.206+0000 7fe5fc899640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81190e0 con 0x7fe5f8113ae0 2024-09-10T23:00:49.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5fc899640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81193e0 con 0x7fe5f81111b0 2024-09-10T23:00:49.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 0x7fe5f81a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 0x7fe5f81a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:54920/0 (socket says 172.21.15.96:54920) 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f659c640 1 -- 172.21.15.96:0/978153987 learned_addr learned my addr 172.21.15.96:0/978153987 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 0x7fe5f81a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/978153987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 msgr2=0x7fe5f81a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 0x7fe5f81a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5f81ad9a0 msgr2=0x7fe5f81ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5f81ad9a0 0x7fe5f81ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.207+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/978153987 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f81ae530 con 0x7fe5f81111b0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 0x7fe5f81a80c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a000 tx=0x7fe5f000ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5dffff640 1 -- 172.21.15.96:0/978153987 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f003a050 con 0x7fe5f81111b0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5dffff640 1 -- 172.21.15.96:0/978153987 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f0003970 con 0x7fe5f81111b0 2024-09-10T23:00:49.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5dffff640 1 -- 172.21.15.96:0/978153987 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00043a0 con 0x7fe5f81111b0 2024-09-10T23:00:49.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f81ae7c0 con 0x7fe5f81111b0 2024-09-10T23:00:49.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.208+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f81b4dd0 con 0x7fe5f81111b0 2024-09-10T23:00:49.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 Processor -- start 2024-09-10T23:00:49.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 -- start start 2024-09-10T23:00:49.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 0x562ea77e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ea774ce50 con 0x562ea77d6650 2024-09-10T23:00:49.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ea773c280 con 0x562ea77e2f00 2024-09-10T23:00:49.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ea7740090 con 0x562ea77d6f70 2024-09-10T23:00:49.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.212+0000 7fe5dffff640 1 -- 172.21.15.96:0/978153987 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fe5f0004540 con 0x7fe5f81111b0 2024-09-10T23:00:49.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.213+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.213+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:54934/0 (socket says 172.21.15.96:54934) 2024-09-10T23:00:49.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.213+0000 7fe5f659c640 1 -- 172.21.15.96:0/4004329074 learned_addr learned my addr 172.21.15.96:0/4004329074 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:49.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.213+0000 7fe5f759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.214+0000 7fe5f759e640 1 -- 172.21.15.96:0/4004329074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 msgr2=0x562ea77e29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.215+0000 7fe5dffff640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 0x7fe5e00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.215+0000 7fe5dffff640 1 -- 172.21.15.96:0/978153987 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe5f01056a0 con 0x7fe5f81111b0 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.215+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 0x562ea77e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.215+0000 7fe5f759e640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 0x562ea77e29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f759e640 1 -- 172.21.15.96:0/4004329074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 msgr2=0x562ea77e52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f759e640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f759e640 1 -- 172.21.15.96:0/4004329074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea749a650 con 0x562ea77d6650 2024-09-10T23:00:49.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f659c640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:00:49.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 0x562ea77e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:00:49.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f759e640 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77d6a30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0096a0 tx=0x7fe5ec009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c61b54b5404bb26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.216+0000 7fe5f759e640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 0x7fe5e00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.217+0000 7fe5dd7fa640 1 -- 172.21.15.96:0/4004329074 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec015020 con 0x562ea77d6650 2024-09-10T23:00:49.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.217+0000 7fe5dd7fa640 1 -- 172.21.15.96:0/4004329074 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec00bd30 con 0x562ea77d6650 2024-09-10T23:00:49.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.217+0000 7fe5dd7fa640 1 -- 172.21.15.96:0/4004329074 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0118c0 con 0x562ea77d6650 2024-09-10T23:00:49.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.218+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4004329074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 msgr2=0x562ea77d6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.218+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77d6a30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0096a0 tx=0x7fe5ec009aa0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.218+0000 7fe5f759e640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 0x7fe5e00b7920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009fd0 tx=0x7fe5ec011f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4004329074 shutdown_connections 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6f70 0x562ea77e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4004329074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77d6a30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4004329074 >> 172.21.15.96:0/4004329074 conn(0x562ea77d4480 msgr2=0x562ea77d5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4004329074 shutdown_connections 2024-09-10T23:00:49.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.219+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4004329074 wait complete. 2024-09-10T23:00:49.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 -- start start 2024-09-10T23:00:49.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 Processor -- start 2024-09-10T23:00:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77d6f70 0x562ea77dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 0x562ea77e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ea759d700 con 0x562ea77d6650 2024-09-10T23:00:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ea76d7110 con 0x562ea77d6f70 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ea77e5b80 con 0x562ea77e2f00 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.220+0000 7fe5ff46f740 10 client.0 _ll_register_callbacks cb 0x562ea76f0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.221+0000 7fe5f759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 0x562ea77e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.221+0000 7fe5f759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 0x562ea77e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43260/0 (socket says 172.21.15.96:43260) 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.221+0000 7fe5f759e640 1 -- 172.21.15.96:0/96688092 learned_addr learned my addr 172.21.15.96:0/96688092 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.221+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.221+0000 7fe5f659c640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77d6f70 0x562ea77dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/96688092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 msgr2=0x562ea77e03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 0x562ea77e03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77d6f70 msgr2=0x562ea77dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77d6f70 0x562ea77dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 -- 172.21.15.96:0/96688092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea7704c40 con 0x562ea77d6650 2024-09-10T23:00:49.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77e28b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0103290 tx=0x7fe5f00cbd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd3460e0fef843e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.222+0000 7fe5daff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00cc600 con 0x562ea77d6650 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe5f00b30e0 con 0x562ea77d6650 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec00bd30 con 0x562ea77d6650 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5f00cbf00 con 0x562ea77d6650 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5ff46f740 10 client.18399 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.223+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ea77f1500 con 0x562ea77d6650 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f0077e50 con 0x562ea77d6650 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f0004b40 con 0x562ea77d6650 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 10 client.18399 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 20 client.18399 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea77f28f0 0x562ea77f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 4 client.18399 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:00:49.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea774eda0 con 0x562ea77f28f0 2024-09-10T23:00:49.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.224+0000 7fe5f659c640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea77f28f0 0x562ea77f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.225+0000 7fe5daff5640 10 client.18399 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:49.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.225+0000 7fe5f659c640 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea77f28f0 0x562ea77f4ce0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36972bbf23161d81 server_cookie=a2a6c80ddc97706e in_seq=0 out_seq=0 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5daff5640 1 -- 172.21.15.96:0/96688092 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ea774eda0 con 0x562ea77f28f0 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5daff5640 10 client.18399 handle_command_reply: tid=0 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 client.18399 shutdown 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 msgr2=0x562ea77e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77e28b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0103290 tx=0x7fe5f00cbd90 comp rx=0 tx=0).stop 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 shutdown_connections 2024-09-10T23:00:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea77f28f0 0x562ea77f4ce0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77d6f70 0x562ea77dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77e2f00 0x562ea77e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/96688092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6650 0x562ea77e28b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 >> 172.21.15.96:0/96688092 conn(0x562ea77d4480 msgr2=0x562ea77da020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.227+0000 7fe5daff5640 0 client.18399 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.228+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 shutdown_connections 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.228+0000 7fe5ff46f740 1 -- 172.21.15.96:0/96688092 wait complete. 2024-09-10T23:00:49.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.228+0000 7fe5ff46f740 20 client.18399 trim_cache size 0 max 16384 2024-09-10T23:00:49.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.255+0000 7fe5ff46f740 1 Processor -- start 2024-09-10T23:00:49.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.255+0000 7fe5ff46f740 1 -- start start 2024-09-10T23:00:49.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81bd4a0 con 0x562ea77d6f70 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81befe0 con 0x562ea77e2f00 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81bdb70 con 0x562ea77d6650 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54142/0 (socket says 172.21.15.96:54142) 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 -- 172.21.15.96:0/707828878 learned_addr learned my addr 172.21.15.96:0/707828878 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f759e640 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 -- 172.21.15.96:0/707828878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 msgr2=0x562ea77d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 -- 172.21.15.96:0/707828878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 msgr2=0x562ea77e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.256+0000 7fe5f659c640 1 -- 172.21.15.96:0/707828878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea77e0740 con 0x562ea77d6f70 2024-09-10T23:00:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.257+0000 7fe5f659c640 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005dc0 tx=0x7fe5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256689c64825524 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.258+0000 7fe5dcff9640 1 -- 172.21.15.96:0/707828878 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004230 con 0x562ea77d6f70 2024-09-10T23:00:49.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.258+0000 7fe5dcff9640 1 -- 172.21.15.96:0/707828878 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e80043d0 con 0x562ea77d6f70 2024-09-10T23:00:49.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.258+0000 7fe5dcff9640 1 -- 172.21.15.96:0/707828878 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80035f0 con 0x562ea77d6f70 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 -- 172.21.15.96:0/707828878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 msgr2=0x562ea77e1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005dc0 tx=0x7fe5e80046c0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 -- 172.21.15.96:0/707828878 shutdown_connections 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77e1a40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/707828878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.259+0000 7fe5ff46f740 1 -- 172.21.15.96:0/707828878 >> 172.21.15.96:0/707828878 conn(0x562ea77d4480 msgr2=0x562ea77e1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- 172.21.15.96:0/707828878 shutdown_connections 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- 172.21.15.96:0/707828878 wait complete. 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- start start 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 Processor -- start 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81be910 con 0x562ea77d6f70 2024-09-10T23:00:49.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81be240 con 0x562ea77e2f00 2024-09-10T23:00:49.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f81bb480 con 0x562ea77d6650 2024-09-10T23:00:49.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.260+0000 7fe5ff46f740 10 client.0 _ll_register_callbacks cb 0x7fe5f0004860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54158/0 (socket says 172.21.15.96:54158) 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 -- 172.21.15.96:0/4051242800 learned_addr learned my addr 172.21.15.96:0/4051242800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f759e640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 -- 172.21.15.96:0/4051242800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 msgr2=0x562ea77eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 -- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 msgr2=0x562ea77e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.261+0000 7fe5f659c640 1 -- 172.21.15.96:0/4051242800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea77e63c0 con 0x562ea77d6f70 2024-09-10T23:00:49.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5f659c640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ac40 tx=0x7fe5e8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac5062e4f1f5b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5daff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003a00 con 0x562ea77d6f70 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fe5e8003ba0 con 0x562ea77d6f70 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e80043d0 con 0x562ea77d6f70 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.262+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5e8003da0 con 0x562ea77d6f70 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 10 client.18405 fetch_fsmap learned FSMap version 32 2024-09-10T23:00:49.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ea7860030 con 0x562ea77d6f70 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e8045070 con 0x562ea77d6f70 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe5e804b660 con 0x562ea77d6f70 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 10 client.18405 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 20 client.18405 populate_metadata read hostname 'smithi096' 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea7860630 0x562ea7862a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:00:49.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 4 client.18405 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:00:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.263+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe5e80043d0 con 0x562ea7860630 2024-09-10T23:00:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.264+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea7860630 0x562ea7862a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:00:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.264+0000 7fe5f6d9d640 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea7860630 0x562ea7862a50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ecffdc1b713d77 server_cookie=7d3454f5f89c39c8 in_seq=0 out_seq=0 2024-09-10T23:00:49.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.265+0000 7fe5daff5640 10 client.18405 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5daff5640 1 -- 172.21.15.96:0/4051242800 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x7fe5e80043d0 con 0x562ea7860630 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5daff5640 10 client.18405 handle_command_reply: tid=0 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 client.18405 shutdown 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 msgr2=0x562ea77eb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ac40 tx=0x7fe5e8004260 comp rx=0 tx=0).stop 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 shutdown_connections 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562ea7860630 0x562ea7862a50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ea77e2f00 0x562ea77e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5daff5640 0 client.18405 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ea77d6f70 0x562ea77eb3a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 --2- 172.21.15.96:0/4051242800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ea77d6650 0x562ea77eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.266+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 >> 172.21.15.96:0/4051242800 conn(0x562ea77d4480 msgr2=0x562ea77e1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.267+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 shutdown_connections 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.267+0000 7fe5ff46f740 1 -- 172.21.15.96:0/4051242800 wait complete. 2024-09-10T23:00:49.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.267+0000 7fe5ff46f740 20 client.18405 trim_cache size 0 max 16384 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "b3181c62-cfec-4c60-99c6-52f389cd7930", 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:00:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:00:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:00:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:00:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:00:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:00:49.281 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 msgr2=0x7fe5e00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 0x7fe5e00b7920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009fd0 tx=0x7fe5ec011f70 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 msgr2=0x7fe5f81a80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 0x7fe5f81a80c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a000 tx=0x7fe5f000ac80 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 shutdown_connections 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5f8113ae0 0x7fe5f81a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fe5e00b5460 0x7fe5e00b7920 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5f81ad9a0 0x7fe5f81ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 --2- 172.21.15.96:0/978153987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f81111b0 0x7fe5f81a80c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:00:49.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.269+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 >> 172.21.15.96:0/978153987 conn(0x7fe5f806cf20 msgr2=0x7fe5f81151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:00:49.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.270+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 shutdown_connections 2024-09-10T23:00:49.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:00:49.270+0000 7fe5fc899640 1 -- 172.21.15.96:0/978153987 wait complete. 2024-09-10T23:00:50.805 INFO:tasks.cephfs.filesystem:scrub status for tag:b3181c62-cfec-4c60-99c6-52f389cd7930 - {'path': '/', 'tag': 'b3181c62-cfec-4c60-99c6-52f389cd7930', 'options': 'recursive,force'} 2024-09-10T23:00:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 795.0 s, 1986.2 tps, lat 50.427 ms stddev 57.243 2024-09-10T23:00:51.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:50 smithi096 ceph-mon[30869]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 36 MiB/s wr, 1.78k op/s 2024-09-10T23:00:51.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:50 smithi121 ceph-mon[35119]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 36 MiB/s wr, 1.78k op/s 2024-09-10T23:00:51.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:50 smithi117 ceph-mon[35081]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 36 MiB/s wr, 1.78k op/s 2024-09-10T23:00:53.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:52 smithi096 ceph-mon[30869]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 35 MiB/s wr, 1.73k op/s 2024-09-10T23:00:53.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:52 smithi121 ceph-mon[35119]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 35 MiB/s wr, 1.73k op/s 2024-09-10T23:00:53.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:52 smithi117 ceph-mon[35081]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 35 MiB/s wr, 1.73k op/s 2024-09-10T23:00:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:54 smithi096 ceph-mon[30869]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-10T23:00:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:54 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:00:55.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:54 smithi121 ceph-mon[35119]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-10T23:00:55.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:54 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:00:55.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:54 smithi117 ceph-mon[35081]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-10T23:00:55.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:54 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:00:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 800.0 s, 1853.0 tps, lat 54.288 ms stddev 62.278 2024-09-10T23:00:56.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:55 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:00:56.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:55 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:00:56.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:55 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:00:57.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:56 smithi096 ceph-mon[30869]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 27 MiB/s wr, 1.50k op/s 2024-09-10T23:00:57.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:56 smithi121 ceph-mon[35119]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 27 MiB/s wr, 1.50k op/s 2024-09-10T23:00:57.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:56 smithi117 ceph-mon[35081]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 27 MiB/s wr, 1.50k op/s 2024-09-10T23:00:59.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:00:58 smithi096 ceph-mon[30869]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:00:59.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:00:58 smithi121 ceph-mon[35119]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:00:59.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:00:58 smithi117 ceph-mon[35081]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:01:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 805.0 s, 2160.0 tps, lat 45.682 ms stddev 54.928 2024-09-10T23:01:01.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:00 smithi096 ceph-mon[30869]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-09-10T23:01:01.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:00 smithi121 ceph-mon[35119]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-09-10T23:01:01.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:00 smithi117 ceph-mon[35081]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-09-10T23:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 -- 172.21.15.96:0/194509973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd400105ad0 msgr2=0x7fd400107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 --2- 172.21.15.96:0/194509973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd400105ad0 0x7fd400107f40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009860 tx=0x7fd3f402a340 comp rx=0 tx=0).stop 2024-09-10T23:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 -- 172.21.15.96:0/194509973 shutdown_connections 2024-09-10T23:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 --2- 172.21.15.96:0/194509973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd400105ad0 0x7fd400107f40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 --2- 172.21.15.96:0/194509973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400108680 0x7fd40010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 --2- 172.21.15.96:0/194509973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd40006bc90 0x7fd400105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 -- 172.21.15.96:0/194509973 >> 172.21.15.96:0/194509973 conn(0x7fd4000fd130 msgr2=0x7fd4000ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 -- 172.21.15.96:0/194509973 shutdown_connections 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.628+0000 7fd4086a2640 1 -- 172.21.15.96:0/194509973 wait complete. 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 Processor -- start 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 -- start start 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 0x7fd400196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd400108680 0x7fd400197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010c420 con 0x7fd400108680 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010c2a0 con 0x7fd40006bc90 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd4086a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010c5a0 con 0x7fd400105ad0 2024-09-10T23:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd40643e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 0x7fd400196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd40643e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 0x7fd400196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:54636/0 (socket says 172.21.15.96:54636) 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd40643e640 1 -- 172.21.15.96:0/612419992 learned_addr learned my addr 172.21.15.96:0/612419992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.629+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:55162/0 (socket says 172.21.15.96:55162) 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd406c3f640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd400108680 0x7fd400197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 -- 172.21.15.96:0/612419992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 msgr2=0x7fd400196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 0x7fd400196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 -- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd400108680 msgr2=0x7fd400197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd400108680 0x7fd400197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 -- 172.21.15.96:0/612419992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4001986c0 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd405c3d640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd3f402a860 tx=0x7fd3f4032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd3f37fe640 1 -- 172.21.15.96:0/612419992 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4037040 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd3f37fe640 1 -- 172.21.15.96:0/612419992 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f4030c50 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd3f37fe640 1 -- 172.21.15.96:0/612419992 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4003850 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.630+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd400198950 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.631+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40019eea0 con 0x7fd400105ad0 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 Processor -- start 2024-09-10T23:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 -- start start 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 0x55774facc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774facf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55774fa396f0 con 0x55774faccbd0 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55774fa28b20 con 0x55774faca190 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd40b470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55774fa2c930 con 0x55774fac7860 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd3f37fe640 1 -- 172.21.15.96:0/612419992 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd3f4004050 con 0x7fd400105ad0 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd406c3f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.633+0000 7fd406c3f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:55170/0 (socket says 172.21.15.96:55170) 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 -- 172.21.15.96:0/4249013812 learned_addr learned my addr 172.21.15.96:0/4249013812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd405c3d640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774facf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd40643e640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 0x55774facc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 -- 172.21.15.96:0/4249013812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 msgr2=0x55774facc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 0x55774facc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 -- 172.21.15.96:0/4249013812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 msgr2=0x55774facf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774facf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd406c3f640 1 -- 172.21.15.96:0/4249013812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774f9ee530 con 0x55774fac7860 2024-09-10T23:01:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd405c3d640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774facf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.634+0000 7fd40643e640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 0x55774facc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.635+0000 7fd406c3f640 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800ffa0 tx=0x7fd3f8011540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48244d43ee097241 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd3f0ff9640 1 -- 172.21.15.96:0/4249013812 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8004b20 con 0x55774fac7860 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd3f0ff9640 1 -- 172.21.15.96:0/4249013812 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f8004cc0 con 0x55774fac7860 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd3f0ff9640 1 -- 172.21.15.96:0/4249013812 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8004b20 con 0x55774fac7860 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd3f37fe640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 0x7fd3ec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd40643e640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 0x7fd3ec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.636+0000 7fd3f37fe640 1 -- 172.21.15.96:0/612419992 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd3f41052a0 con 0x7fd400105ad0 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- 172.21.15.96:0/4249013812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 msgr2=0x55774fac9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40643e640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 0x7fd3ec0b7820 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0058c0 tx=0x7fd3fc011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800ffa0 tx=0x7fd3f8011540 comp rx=0 tx=0).stop 2024-09-10T23:01:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- 172.21.15.96:0/4249013812 shutdown_connections 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faca190 0x55774facc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774facf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 --2- 172.21.15.96:0/4249013812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fac9c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- 172.21.15.96:0/4249013812 >> 172.21.15.96:0/4249013812 conn(0x55774fac29b0 msgr2=0x55774fac3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- 172.21.15.96:0/4249013812 shutdown_connections 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- 172.21.15.96:0/4249013812 wait complete. 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 -- start start 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.637+0000 7fd40b470740 1 Processor -- start 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fad0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40643e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fad0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd406c3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40643e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fad0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:54654/0 (socket says 172.21.15.96:54654) 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd406c3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49580/0 (socket says 172.21.15.96:49580) 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:55182/0 (socket says 172.21.15.96:55182) 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40643e640 1 -- 172.21.15.96:0/2444199508 learned_addr learned my addr 172.21.15.96:0/2444199508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55774f960ab0 con 0x55774faccbd0 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55774f9ce730 con 0x55774fac7860 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.638+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55774facfbb0 con 0x55774faca190 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 -- 172.21.15.96:0/2444199508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 msgr2=0x55774fad1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 -- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 msgr2=0x55774fad0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fad0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774f78a790 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd405c3d640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd40b470740 10 client.0 _ll_register_callbacks cb 0x55774fa226b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f8004cc0 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd406c3f640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004af0 tx=0x7fd3f8017d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d99c895201a96dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd3e9ffb640 10 client.18414 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2444199508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8036050 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.639+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2444199508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd3f8005280 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2444199508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3f802c070 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 10 client.18414 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55774f803620 con 0x55774faccbd0 2024-09-10T23:01:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2444199508 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3f80054c0 con 0x55774faccbd0 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 10 client.18414 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 10 client.18414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 20 client.18414 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fade460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 4 client.18414 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55774fa3b640 con 0x55774fadc070 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.640+0000 7fd40643e640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fade460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd3e9ffb640 10 client.18414 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40643e640 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fade460 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd1dc9f3891d102 server_cookie=96b1663ed69c3154 in_seq=0 out_seq=0 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2444199508 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55774fa3b640 con 0x55774fadc070 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd3e9ffb640 10 client.18414 handle_command_reply: tid=0 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 client.18414 shutdown 2024-09-10T23:01:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 msgr2=0x55774fad20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004af0 tx=0x7fd3f8017d20 comp rx=0 tx=0).stop 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 shutdown_connections 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fade460 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faccbd0 0x55774fad20d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd3e9ffb640 0 client.18414 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faca190 0x55774fad1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 --2- 172.21.15.96:0/2444199508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fad0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 >> 172.21.15.96:0/2444199508 conn(0x55774fac29b0 msgr2=0x55774fac1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.641+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 shutdown_connections 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.642+0000 7fd40b470740 1 -- 172.21.15.96:0/2444199508 wait complete. 2024-09-10T23:01:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.642+0000 7fd40b470740 20 client.18414 trim_cache size 0 max 16384 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.659+0000 7fd40b470740 1 Processor -- start 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 -- start start 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fac13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faccbd0 0x55774fac47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a7be0 con 0x55774faca190 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a9720 con 0x55774fac7860 2024-09-10T23:01:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40b470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a82b0 con 0x55774faccbd0 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49582/0 (socket says 172.21.15.96:49582) 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd405c3d640 1 -- 172.21.15.96:0/3091567179 learned_addr learned my addr 172.21.15.96:0/3091567179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd406c3f640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faccbd0 0x55774fac47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd40643e640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fac13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd405c3d640 1 -- 172.21.15.96:0/3091567179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faccbd0 msgr2=0x55774fac47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.660+0000 7fd405c3d640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faccbd0 0x55774fac47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.661+0000 7fd405c3d640 1 -- 172.21.15.96:0/3091567179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 msgr2=0x55774fac13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.661+0000 7fd405c3d640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fac13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.661+0000 7fd405c3d640 1 -- 172.21.15.96:0/3091567179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774fac9fc0 con 0x55774faca190 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.661+0000 7fd40643e640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fac13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.661+0000 7fd405c3d640 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40c80d0 tx=0x7fd3f4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ce13289f61ed23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.662+0000 7fd3ebfff640 1 -- 172.21.15.96:0/3091567179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40cb270 con 0x55774faca190 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.662+0000 7fd3ebfff640 1 -- 172.21.15.96:0/3091567179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f40cb610 con 0x55774faca190 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.662+0000 7fd3ebfff640 1 -- 172.21.15.96:0/3091567179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40c7270 con 0x55774faca190 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- 172.21.15.96:0/3091567179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 msgr2=0x55774fac1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40c80d0 tx=0x7fd3f4005bd0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- 172.21.15.96:0/3091567179 shutdown_connections 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774faccbd0 0x55774fac47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fac1910 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- 172.21.15.96:0/3091567179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774fac7860 0x55774fac13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- 172.21.15.96:0/3091567179 >> 172.21.15.96:0/3091567179 conn(0x55774fac29b0 msgr2=0x55774fac6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- 172.21.15.96:0/3091567179 shutdown_connections 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- 172.21.15.96:0/3091567179 wait complete. 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- start start 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 Processor -- start 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fad8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 0x55774fad7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a9050 con 0x55774faca190 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a8980 con 0x55774faccbd0 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001a5bc0 con 0x55774fac7860 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.663+0000 7fd40b470740 10 client.0 _ll_register_callbacks cb 0x7fd3f800fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49594/0 (socket says 172.21.15.96:49594) 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 -- 172.21.15.96:0/2333558186 learned_addr learned my addr 172.21.15.96:0/2333558186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd406c3f640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 0x55774fad7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd40643e640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fad8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 -- 172.21.15.96:0/2333558186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 msgr2=0x55774fad8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fad8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 -- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 msgr2=0x55774fad7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 0x55774fad7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 -- 172.21.15.96:0/2333558186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774fad8680 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd406c3f640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 0x55774fad7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd405c3d640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009830 tx=0x7fd3f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcadf81ed04508d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd3e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40cb270 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.664+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd3f40cad60 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.665+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f40cb610 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.665+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3f4030e00 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.665+0000 7fd40b470740 10 client.18420 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.665+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55774fb48fe0 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.665+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f40cb610 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd3f4030550 con 0x55774faca190 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 10 client.18420 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 10 client.18420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 20 client.18420 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fb4b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 4 client.18420 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55774fa370f0 con 0x55774fadc070 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40643e640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fb4b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd3e9ffb640 10 client.18420 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.666+0000 7fd40643e640 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fb4b710 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dae9c360e697bc4 server_cookie=287b262e3823401e in_seq=0 out_seq=0 2024-09-10T23:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd3e9ffb640 1 -- 172.21.15.96:0/2333558186 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55774fa370f0 con 0x55774fadc070 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd3e9ffb640 10 client.18420 handle_command_reply: tid=0 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 client.18420 shutdown 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 msgr2=0x55774fad7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009830 tx=0x7fd3f4005f90 comp rx=0 tx=0).stop 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 shutdown_connections 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55774fadc070 0x55774fb4b710 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55774faccbd0 0x55774fad7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55774faca190 0x55774fad7910 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 --2- 172.21.15.96:0/2333558186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55774fac7860 0x55774fad8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd3e9ffb640 0 client.18420 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 >> 172.21.15.96:0/2333558186 conn(0x55774fac29b0 msgr2=0x55774fac1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 shutdown_connections 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 1 -- 172.21.15.96:0/2333558186 wait complete. 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.683+0000 7fd40b470740 20 client.18420 trim_cache size 0 max 16384 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:01:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 msgr2=0x7fd3ec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 0x7fd3ec0b7820 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0058c0 tx=0x7fd3fc011040 comp rx=0 tx=0).stop 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 msgr2=0x7fd4001971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd3f402a860 tx=0x7fd3f4032e90 comp rx=0 tx=0).stop 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 shutdown_connections 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd3ec0b5360 0x7fd3ec0b7820 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd400108680 0x7fd400197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd400105ad0 0x7fd4001971d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 --2- 172.21.15.96:0/612419992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40006bc90 0x7fd400196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 >> 172.21.15.96:0/612419992 conn(0x7fd4000fd130 msgr2=0x7fd400103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 shutdown_connections 2024-09-10T23:01:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:01.684+0000 7fd4086a2640 1 -- 172.21.15.96:0/612419992 wait complete. 2024-09-10T23:01:03.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:02 smithi096 ceph-mon[30869]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-09-10T23:01:03.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:02 smithi121 ceph-mon[35119]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-09-10T23:01:03.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:02 smithi117 ceph-mon[35081]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-09-10T23:01:05.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:04 smithi096 ceph-mon[30869]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 800 op/s 2024-09-10T23:01:05.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:04 smithi121 ceph-mon[35119]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 800 op/s 2024-09-10T23:01:05.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:04 smithi117 ceph-mon[35081]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 800 op/s 2024-09-10T23:01:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 810.0 s, 2003.2 tps, lat 50.099 ms stddev 58.819 2024-09-10T23:01:07.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:06 smithi096 ceph-mon[30869]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:01:07.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:06 smithi121 ceph-mon[35119]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:01:07.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:06 smithi117 ceph-mon[35081]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-09-10T23:01:09.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:08 smithi096 ceph-mon[30869]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 618 op/s 2024-09-10T23:01:09.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:08 smithi121 ceph-mon[35119]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 618 op/s 2024-09-10T23:01:09.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:08 smithi117 ceph-mon[35081]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 618 op/s 2024-09-10T23:01:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 815.0 s, 2161.6 tps, lat 46.557 ms stddev 55.596 2024-09-10T23:01:11.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:10 smithi096 ceph-mon[30869]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 759 op/s 2024-09-10T23:01:11.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:10 smithi121 ceph-mon[35119]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 759 op/s 2024-09-10T23:01:11.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:10 smithi117 ceph-mon[35081]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 759 op/s 2024-09-10T23:01:13.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:12 smithi096 ceph-mon[30869]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 644 op/s 2024-09-10T23:01:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:12 smithi121 ceph-mon[35119]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 644 op/s 2024-09-10T23:01:13.025 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:12 smithi117 ceph-mon[35081]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 644 op/s 2024-09-10T23:01:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/844259598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd41029e0 msgr2=0x7f4bd4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/844259598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd41029e0 0x7f4bd4102e60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009930 tx=0x7f4bc402a420 comp rx=0 tx=0).stop 2024-09-10T23:01:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/844259598 shutdown_connections 2024-09-10T23:01:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/844259598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/844259598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd41029e0 0x7f4bd4102e60 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/844259598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd4101890 0x7f4bd4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/844259598 >> 172.21.15.96:0/844259598 conn(0x7f4bd40fd130 msgr2=0x7f4bd40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.791+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/844259598 shutdown_connections 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/844259598 wait complete. 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 Processor -- start 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 -- start start 2024-09-10T23:01:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 0x7f4bd41055c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd41029e0 0x7f4bd4103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd41043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410d500 con 0x7f4bd4101890 2024-09-10T23:01:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410d380 con 0x7f4bd41033a0 2024-09-10T23:01:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bdbcd4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410d680 con 0x7f4bd41029e0 2024-09-10T23:01:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bd9a70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 0x7f4bd41055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bd9a70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 0x7f4bd41055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38398/0 (socket says 172.21.15.96:38398) 2024-09-10T23:01:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bd9a70640 1 -- 172.21.15.96:0/539835915 learned_addr learned my addr 172.21.15.96:0/539835915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bda271640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd41043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.792+0000 7f4bd926f640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd41029e0 0x7f4bd4103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 -- 172.21.15.96:0/539835915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd41029e0 msgr2=0x7f4bd4103e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd41029e0 0x7f4bd4103e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 -- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 msgr2=0x7f4bd41055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 0x7f4bd41055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 -- 172.21.15.96:0/539835915 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd4104bd0 con 0x7f4bd41033a0 2024-09-10T23:01:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bda271640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd41043d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009fd0 tx=0x7f4bcc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.793+0000 7f4bcaffd640 1 -- 172.21.15.96:0/539835915 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00ce20 con 0x7f4bd41033a0 2024-09-10T23:01:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.794+0000 7f4bcaffd640 1 -- 172.21.15.96:0/539835915 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bcc005020 con 0x7f4bd41033a0 2024-09-10T23:01:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.794+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd41a2ea0 con 0x7f4bd41033a0 2024-09-10T23:01:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.794+0000 7f4bcaffd640 1 -- 172.21.15.96:0/539835915 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc010670 con 0x7f4bd41033a0 2024-09-10T23:01:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.794+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd41a3360 con 0x7f4bd41033a0 2024-09-10T23:01:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.796+0000 7f4bdeaa8740 1 Processor -- start 2024-09-10T23:01:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bcaffd640 1 -- 172.21.15.96:0/539835915 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f4bcc0054b0 con 0x7f4bd41033a0 2024-09-10T23:01:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 -- start start 2024-09-10T23:01:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b8b7e6f0 con 0x5589b8c0c860 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b8b6db20 con 0x5589b8c11bd0 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b8b71930 con 0x5589b8c0f190 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bd926f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bd926f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38408/0 (socket says 172.21.15.96:38408) 2024-09-10T23:01:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bd926f640 1 -- 172.21.15.96:0/866581176 learned_addr learned my addr 172.21.15.96:0/866581176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bd926f640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.797+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 -- 172.21.15.96:0/866581176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 msgr2=0x5589b8c14090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c14090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 -- 172.21.15.96:0/866581176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 msgr2=0x5589b8c0ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 -- 172.21.15.96:0/866581176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b8b33530 con 0x5589b8c0f190 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bcaffd640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 0x7f4bc00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd926f640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd926f640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 0x7f4bc00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c11580 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009cf0 tx=0x7f4bd0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9529013019784c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bcaffd640 1 -- 172.21.15.96:0/539835915 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4bcc0e6170 con 0x7f4bd41033a0 2024-09-10T23:01:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bbffff640 1 -- 172.21.15.96:0/866581176 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0005220 con 0x5589b8c0f190 2024-09-10T23:01:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bbffff640 1 -- 172.21.15.96:0/866581176 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd0004c50 con 0x5589b8c0f190 2024-09-10T23:01:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.798+0000 7f4bbffff640 1 -- 172.21.15.96:0/866581176 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00197a0 con 0x5589b8c0f190 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.799+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/866581176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 msgr2=0x5589b8c11580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.799+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c11580 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009cf0 tx=0x7f4bd0011810 comp rx=0 tx=0).stop 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.799+0000 7f4bd926f640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 0x7f4bc00b78f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4000c00 tx=0x7f4bc402a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/866581176 shutdown_connections 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c11580 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/866581176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c0ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/866581176 >> 172.21.15.96:0/866581176 conn(0x5589b8c079b0 msgr2=0x5589b8c08740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/866581176 shutdown_connections 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.800+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/866581176 wait complete. 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 -- start start 2024-09-10T23:01:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 Processor -- start 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 0x5589b8c15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c164d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589b8aa5ab0 con 0x5589b8c11bd0 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589b8b13730 con 0x5589b8c0c860 2024-09-10T23:01:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589b8c14bb0 con 0x5589b8c0f190 2024-09-10T23:01:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bdeaa8740 10 client.0 _ll_register_callbacks cb 0x5589b8b676b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bda271640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bda271640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38412/0 (socket says 172.21.15.96:38412) 2024-09-10T23:01:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bda271640 1 -- 172.21.15.96:0/1284499233 learned_addr learned my addr 172.21.15.96:0/1284499233 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bd9a70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 0x5589b8c15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.801+0000 7f4bd926f640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 -- 172.21.15.96:0/1284499233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 msgr2=0x5589b8c164d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c164d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 -- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 msgr2=0x5589b8c15e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 0x5589b8c15e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 -- 172.21.15.96:0/1284499233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b88cf790 con 0x5589b8c11bd0 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 0x5589b8c15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bda271640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0a8f00 tx=0x7f4bcc0a8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102e2466c9f78d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0d3020 con 0x5589b8c11bd0 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4bcc0b5c30 con 0x5589b8c11bd0 2024-09-10T23:01:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd0004c50 con 0x5589b8c11bd0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4bcc0b11e0 con 0x5589b8c11bd0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bdeaa8740 10 client.18432 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b8948620 con 0x5589b8c11bd0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd0004c50 con 0x5589b8c11bd0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.802+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4bcc0ab600 con 0x5589b8c11bd0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 10 client.18432 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 10 client.18432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 20 client.18432 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c21770 0x5589b8c23c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 4 client.18432 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b8b80640 con 0x5589b8c21770 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c21770 0x5589b8c23c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c21770 0x5589b8c23c10 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cecc05cc2dff3ad server_cookie=983a653062ab7936 in_seq=0 out_seq=0 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.803+0000 7f4bbd7fa640 10 client.18432 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1284499233 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589b8b80640 con 0x5589b8c21770 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bbd7fa640 10 client.18432 handle_command_reply: tid=0 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 client.18432 shutdown 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 msgr2=0x5589b8c170a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0a8f00 tx=0x7f4bcc0a8160 comp rx=0 tx=0).stop 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 shutdown_connections 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c21770 0x5589b8c23c10 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c11bd0 0x5589b8c170a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bbd7fa640 0 client.18432 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1284499233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c0c860 0x5589b8c15e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 >> 172.21.15.96:0/1284499233 conn(0x5589b8c079b0 msgr2=0x5589b8c06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 shutdown_connections 2024-09-10T23:01:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1284499233 wait complete. 2024-09-10T23:01:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.804+0000 7f4bdeaa8740 20 client.18432 trim_cache size 0 max 16384 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.822+0000 7f4bdeaa8740 1 Processor -- start 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.822+0000 7f4bdeaa8740 1 -- start start 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.822+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41abf40 con 0x5589b8c0c860 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41ada80 con 0x5589b8c11bd0 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41ac610 con 0x5589b8c0f190 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bda271640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bda271640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:56448/0 (socket says 172.21.15.96:56448) 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bda271640 1 -- 172.21.15.96:0/2192334382 learned_addr learned my addr 172.21.15.96:0/2192334382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bd926f640 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.823+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 -- 172.21.15.96:0/2192334382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 msgr2=0x5589b8c0bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c0bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 -- 172.21.15.96:0/2192334382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 msgr2=0x5589b8c17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 -- 172.21.15.96:0/2192334382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b8c118f0 con 0x5589b8c11bd0 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bda271640 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc01a040 tx=0x7f4bcc005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df04899d49bed547 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bbf7fe640 1 -- 172.21.15.96:0/2192334382 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0d3040 con 0x5589b8c11bd0 2024-09-10T23:01:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.824+0000 7f4bbf7fe640 1 -- 172.21.15.96:0/2192334382 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bcc0ac930 con 0x5589b8c11bd0 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bbf7fe640 1 -- 172.21.15.96:0/2192334382 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0a84e0 con 0x5589b8c11bd0 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/2192334382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 msgr2=0x5589b8c0c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc01a040 tx=0x7f4bcc005400 comp rx=0 tx=0).stop 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/2192334382 shutdown_connections 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c0c170 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c0bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/2192334382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/2192334382 >> 172.21.15.96:0/2192334382 conn(0x5589b8c079b0 msgr2=0x5589b8c0aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/2192334382 shutdown_connections 2024-09-10T23:01:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.825+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/2192334382 wait complete. 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 -- start start 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 Processor -- start 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c1c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c1c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41ad3b0 con 0x5589b8c0c860 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41acce0 con 0x5589b8c11bd0 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd41a9f20 con 0x5589b8c0f190 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bdeaa8740 10 client.0 _ll_register_callbacks cb 0x7f4bcc0ac2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bd9a70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bd9a70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38430/0 (socket says 172.21.15.96:38430) 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bd9a70640 1 -- 172.21.15.96:0/1104463369 learned_addr learned my addr 172.21.15.96:0/1104463369 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bda271640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c1c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.826+0000 7f4bd926f640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c1c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 -- 172.21.15.96:0/1104463369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 msgr2=0x5589b8c1c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c1c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 -- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 msgr2=0x5589b8c1c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c1c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 -- 172.21.15.96:0/1104463369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b8c1d000 con 0x5589b8c0c860 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bd9a70640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4bd001b040 tx=0x7f4bd0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c40cea357d454 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0005330 con 0x5589b8c0c860 2024-09-10T23:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f4bd00054d0 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bcc0ac930 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4bd00098f0 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bdeaa8740 10 client.18438 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589b8c8e6e0 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.827+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bcc0ac930 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4bd0019430 con 0x5589b8c0c860 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 10 client.18438 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 10 client.18438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 20 client.18438 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c8eec0 0x5589b8c912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 4 client.18438 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b8b7c0f0 con 0x5589b8c8eec0 2024-09-10T23:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.828+0000 7f4bd926f640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c8eec0 0x5589b8c912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.829+0000 7f4bd926f640 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c8eec0 0x5589b8c912e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a5ef213757b797 server_cookie=4c0db6c2807db055 in_seq=0 out_seq=0 2024-09-10T23:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.829+0000 7f4bbd7fa640 10 client.18438 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bbd7fa640 1 -- 172.21.15.96:0/1104463369 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b8b7c0f0 con 0x5589b8c8eec0 2024-09-10T23:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bbd7fa640 10 client.18438 handle_command_reply: tid=0 2024-09-10T23:01:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 client.18438 shutdown 2024-09-10T23:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 msgr2=0x5589b8c1d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4bd001b040 tx=0x7f4bd0009310 comp rx=0 tx=0).stop 2024-09-10T23:01:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 shutdown_connections 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5589b8c8eec0 0x5589b8c912e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589b8c11bd0 0x5589b8c1c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589b8c0f190 0x5589b8c1c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bbd7fa640 0 client.18438 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.843+0000 7f4bdeaa8740 1 --2- 172.21.15.96:0/1104463369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589b8c0c860 0x5589b8c1d750 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.844+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 >> 172.21.15.96:0/1104463369 conn(0x5589b8c079b0 msgr2=0x5589b8c0bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.844+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 shutdown_connections 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.844+0000 7f4bdeaa8740 1 -- 172.21.15.96:0/1104463369 wait complete. 2024-09-10T23:01:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.844+0000 7f4bdeaa8740 20 client.18438 trim_cache size 0 max 16384 2024-09-10T23:01:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:01:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:01:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:01:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 msgr2=0x7f4bc00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 0x7f4bc00b78f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4000c00 tx=0x7f4bc402a420 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 msgr2=0x7f4bd41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd41043d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009fd0 tx=0x7f4bcc00ef10 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 shutdown_connections 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bd41033a0 0x7f4bd41043d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f4bc00b5430 0x7f4bc00b78f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bd41029e0 0x7f4bd4103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 --2- 172.21.15.96:0/539835915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bd4101890 0x7f4bd41055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 >> 172.21.15.96:0/539835915 conn(0x7f4bd40fd130 msgr2=0x7f4bd40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 shutdown_connections 2024-09-10T23:01:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:14.845+0000 7f4bdbcd4640 1 -- 172.21.15.96:0/539835915 wait complete. 2024-09-10T23:01:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:15 smithi096 ceph-mon[30869]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T23:01:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:15 smithi121 ceph-mon[35119]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T23:01:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:15 smithi117 ceph-mon[35081]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T23:01:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 820.0 s, 2005.8 tps, lat 49.301 ms stddev 60.696 2024-09-10T23:01:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:17 smithi096 ceph-mon[30869]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-09-10T23:01:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:17 smithi121 ceph-mon[35119]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-09-10T23:01:17.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:17 smithi117 ceph-mon[35081]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-09-10T23:01:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:19 smithi096 ceph-mon[30869]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 580 op/s 2024-09-10T23:01:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:19 smithi121 ceph-mon[35119]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 580 op/s 2024-09-10T23:01:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:19 smithi117 ceph-mon[35081]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 MiB/s wr, 580 op/s 2024-09-10T23:01:20.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 825.0 s, 1827.2 tps, lat 55.350 ms stddev 62.520 2024-09-10T23:01:21.234 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:21.566 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:21 smithi096 ceph-mon[30869]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-09-10T23:01:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:21 smithi121 ceph-mon[35119]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-09-10T23:01:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:21 smithi117 ceph-mon[35081]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 712 op/s 2024-09-10T23:01:21.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.840+0000 7efd00c61640 1 -- 172.21.15.96:0/3740573858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc108040 msgr2=0x7efcfc10e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.840+0000 7efd00c61640 1 --2- 172.21.15.96:0/3740573858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc108040 0x7efcfc10e840 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009ca0 tx=0x7efcf402c7a0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.846+0000 7efd00c61640 1 -- 172.21.15.96:0/3740573858 shutdown_connections 2024-09-10T23:01:21.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.846+0000 7efd00c61640 1 --2- 172.21.15.96:0/3740573858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc108040 0x7efcfc10e840 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.846+0000 7efd00c61640 1 --2- 172.21.15.96:0/3740573858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcfc1076c0 0x7efcfc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.846+0000 7efd00c61640 1 --2- 172.21.15.96:0/3740573858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc101cb0 0x7efcfc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.846+0000 7efd00c61640 1 -- 172.21.15.96:0/3740573858 >> 172.21.15.96:0/3740573858 conn(0x7efcfc0fd700 msgr2=0x7efcfc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.854+0000 7efd00c61640 1 -- 172.21.15.96:0/3740573858 shutdown_connections 2024-09-10T23:01:21.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.854+0000 7efd00c61640 1 -- 172.21.15.96:0/3740573858 wait complete. 2024-09-10T23:01:21.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.855+0000 7efd00c61640 1 Processor -- start 2024-09-10T23:01:21.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 -- start start 2024-09-10T23:01:21.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc101cb0 0x7efcfc196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcfc108040 0x7efcfc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc110c00 con 0x7efcfc101cb0 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc110a80 con 0x7efcfc1076c0 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.859+0000 7efd00c61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc110d80 con 0x7efcfc108040 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39592/0 (socket says 172.21.15.96:39592) 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 -- 172.21.15.96:0/1936155166 learned_addr learned my addr 172.21.15.96:0/1936155166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 -- 172.21.15.96:0/1936155166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcfc108040 msgr2=0x7efcfc197940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcfc108040 0x7efcfc197940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 -- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc101cb0 msgr2=0x7efcfc196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc101cb0 0x7efcfc196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.860+0000 7efcf9d9b640 1 -- 172.21.15.96:0/1936155166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc19b230 con 0x7efcfc1076c0 2024-09-10T23:01:21.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.861+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efcf000e9f0 tx=0x7efcf000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.861+0000 7efceb7fe640 1 -- 172.21.15.96:0/1936155166 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf000cd80 con 0x7efcfc1076c0 2024-09-10T23:01:21.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.861+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc19b520 con 0x7efcfc1076c0 2024-09-10T23:01:21.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.861+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc19bab0 con 0x7efcfc1076c0 2024-09-10T23:01:21.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.861+0000 7efceb7fe640 1 -- 172.21.15.96:0/1936155166 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf00108e0 con 0x7efcfc1076c0 2024-09-10T23:01:21.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.862+0000 7efceb7fe640 1 -- 172.21.15.96:0/1936155166 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00188e0 con 0x7efcfc1076c0 2024-09-10T23:01:21.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.867+0000 7efd03036740 1 Processor -- start 2024-09-10T23:01:21.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 -- start start 2024-09-10T23:01:21.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efceb7fe640 1 -- 172.21.15.96:0/1936155166 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7efcf0010430 con 0x7efcfc1076c0 2024-09-10T23:01:21.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562bd0327e50 con 0x562bd03b1da0 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562bd0317280 con 0x562bd03b1480 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efd03036740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562bd031b090 con 0x562bd03bdd30 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39608/0 (socket says 172.21.15.96:39608) 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.869+0000 7efcfad9d640 1 -- 172.21.15.96:0/1575572418 learned_addr learned my addr 172.21.15.96:0/1575572418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.870+0000 7efcfa59c640 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.870+0000 7efceb7fe640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 0x7efce40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.871+0000 7efceb7fe640 1 -- 172.21.15.96:0/1936155166 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efcf00e5df0 con 0x7efcfc1076c0 2024-09-10T23:01:21.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.872+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.874+0000 7efcfa59c640 1 -- 172.21.15.96:0/1575572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 msgr2=0x562bd03c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.874+0000 7efcfa59c640 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.874+0000 7efcfa59c640 1 -- 172.21.15.96:0/1575572418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 msgr2=0x562bd03b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.874+0000 7efcfa59c640 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.874+0000 7efcfa59c640 1 -- 172.21.15.96:0/1575572418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd0075650 con 0x562bd03b1da0 2024-09-10T23:01:21.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.876+0000 7efcfa59c640 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03bd7f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efcec009a90 tx=0x7efcec009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb943ae71301519d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.877+0000 7efce8ff9640 1 -- 172.21.15.96:0/1575572418 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec00ec50 con 0x562bd03b1da0 2024-09-10T23:01:21.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.877+0000 7efce8ff9640 1 -- 172.21.15.96:0/1575572418 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcec00edf0 con 0x562bd03b1da0 2024-09-10T23:01:21.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.878+0000 7efce8ff9640 1 -- 172.21.15.96:0/1575572418 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec018780 con 0x562bd03b1da0 2024-09-10T23:01:21.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.878+0000 7efcfad9d640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 0x7efce40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.879+0000 7efd03036740 1 -- 172.21.15.96:0/1575572418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 msgr2=0x562bd03bd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.879+0000 7efd03036740 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03bd7f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efcec009a90 tx=0x7efcec009e90 comp rx=0 tx=0).stop 2024-09-10T23:01:21.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efcfad9d640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 0x7efce40b7820 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009430 tx=0x7efcf40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efd03036740 1 -- 172.21.15.96:0/1575572418 shutdown_connections 2024-09-10T23:01:21.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efd03036740 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efd03036740 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03bd7f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efd03036740 1 --2- 172.21.15.96:0/1575572418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.880+0000 7efd03036740 1 -- 172.21.15.96:0/1575572418 >> 172.21.15.96:0/1575572418 conn(0x562bd03af2b0 msgr2=0x562bd03b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.881+0000 7efd03036740 1 -- 172.21.15.96:0/1575572418 shutdown_connections 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.881+0000 7efd03036740 1 -- 172.21.15.96:0/1575572418 wait complete. 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.881+0000 7efd03036740 1 -- start start 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 Processor -- start 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03bd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562bd0178700 con 0x562bd03b1da0 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562bd02b2110 con 0x562bd03b1480 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562bd03c09b0 con 0x562bd03bdd30 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.882+0000 7efd03036740 10 client.0 _ll_register_callbacks cb 0x562bd02cb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.883+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.883+0000 7efcfa59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03bd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.883+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03bafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43428/0 (socket says 172.21.15.96:43428) 2024-09-10T23:01:21.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.883+0000 7efcfad9d640 1 -- 172.21.15.96:0/3479421546 learned_addr learned my addr 172.21.15.96:0/3479421546 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:21.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 -- 172.21.15.96:0/3479421546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 msgr2=0x562bd03bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 -- 172.21.15.96:0/3479421546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 msgr2=0x562bd03baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 -- 172.21.15.96:0/3479421546 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd02dfc40 con 0x562bd03b1480 2024-09-10T23:01:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.884+0000 7efcfa59c640 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03bd5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efcec004700 tx=0x7efcec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127b9a7217c34f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.885+0000 7efce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:21.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.885+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec028020 con 0x562bd03b1480 2024-09-10T23:01:21.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.885+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcec00edf0 con 0x562bd03b1480 2024-09-10T23:01:21.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.885+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efcec005220 con 0x562bd03b1480 2024-09-10T23:01:21.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.886+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcec005420 con 0x562bd03b1480 2024-09-10T23:01:21.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.886+0000 7efd03036740 10 client.28058 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:21.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.886+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562bd03cc310 con 0x562bd03b1480 2024-09-10T23:01:21.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.886+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcec00edf0 con 0x562bd03b1480 2024-09-10T23:01:21.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efcec02d070 con 0x562bd03b1480 2024-09-10T23:01:21.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 10 client.28058 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:21.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 20 client.28058 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd03cd770 0x562bd03cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 4 client.28058 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.887+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bd0329da0 con 0x562bd03cd770 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.888+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd03cd770 0x562bd03cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.888+0000 7efce1ffb640 10 client.28058 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:21.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.888+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd03cd770 0x562bd03cfb60 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21250ed78b844b6 server_cookie=7e5addef85d77f95 in_seq=0 out_seq=0 2024-09-10T23:01:21.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efce1ffb640 1 -- 172.21.15.96:0/3479421546 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bd0329da0 con 0x562bd03cd770 2024-09-10T23:01:21.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efce1ffb640 10 client.28058 handle_command_reply: tid=0 2024-09-10T23:01:21.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 client.28058 shutdown 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 msgr2=0x562bd03bd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03bd5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efcec004700 tx=0x7efcec004270 comp rx=0 tx=0).stop 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 shutdown_connections 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd03cd770 0x562bd03cfb60 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03bdd30 0x562bd03bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 --2- 172.21.15.96:0/3479421546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03b1480 0x562bd03bd5b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 >> 172.21.15.96:0/3479421546 conn(0x562bd03af2b0 msgr2=0x562bd03b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.889+0000 7efce1ffb640 0 client.28058 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:21.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.890+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 shutdown_connections 2024-09-10T23:01:21.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.890+0000 7efd03036740 1 -- 172.21.15.96:0/3479421546 wait complete. 2024-09-10T23:01:21.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.890+0000 7efd03036740 20 client.28058 trim_cache size 0 max 16384 2024-09-10T23:01:21.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 Processor -- start 2024-09-10T23:01:21.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 -- start start 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 0x562bd03c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03b8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00e3f0 con 0x562bd03b1da0 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00ff30 con 0x562bd03bdd30 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.915+0000 7efd03036740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00eac0 con 0x562bd03b1480 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcf9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 0x562bd03c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03b8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:39626/0 (socket says 172.21.15.96:39626) 2024-09-10T23:01:21.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfad9d640 1 -- 172.21.15.96:0/4259728804 learned_addr learned my addr 172.21.15.96:0/4259728804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:21.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 -- 172.21.15.96:0/4259728804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 msgr2=0x562bd03b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03b8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 -- 172.21.15.96:0/4259728804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 msgr2=0x562bd03b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcfa59c640 1 -- 172.21.15.96:0/4259728804 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd03afd30 con 0x562bd03b1480 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.916+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.917+0000 7efcfa59c640 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 0x562bd03c9c70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad1319eccdd121f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.917+0000 7efce3fff640 1 -- 172.21.15.96:0/4259728804 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec028070 con 0x562bd03b1480 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.917+0000 7efce3fff640 1 -- 172.21.15.96:0/4259728804 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcec00edd0 con 0x562bd03b1480 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.917+0000 7efce3fff640 1 -- 172.21.15.96:0/4259728804 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec033040 con 0x562bd03b1480 2024-09-10T23:01:21.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- 172.21.15.96:0/4259728804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 msgr2=0x562bd03c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 0x562bd03c9c70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec009810 comp rx=0 tx=0).stop 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- 172.21.15.96:0/4259728804 shutdown_connections 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1da0 0x562bd03b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 --2- 172.21.15.96:0/4259728804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1480 0x562bd03c9c70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- 172.21.15.96:0/4259728804 >> 172.21.15.96:0/4259728804 conn(0x562bd03af2b0 msgr2=0x562bd03b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- 172.21.15.96:0/4259728804 shutdown_connections 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- 172.21.15.96:0/4259728804 wait complete. 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 -- start start 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.918+0000 7efd03036740 1 Processor -- start 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 0x562bd03c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 0x562bd03c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00f860 con 0x562bd03b1480 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00f190 con 0x562bd03bdd30 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc00c3d0 con 0x562bd03b1da0 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efd03036740 10 client.0 _ll_register_callbacks cb 0x7efcec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efcfad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efcf9d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 0x562bd03c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efcf9d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 0x562bd03c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43448/0 (socket says 172.21.15.96:43448) 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.919+0000 7efcf9d9b640 1 -- 172.21.15.96:0/687620808 learned_addr learned my addr 172.21.15.96:0/687620808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 0x562bd03c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 -- 172.21.15.96:0/687620808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 msgr2=0x562bd03c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 0x562bd03c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 -- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 msgr2=0x562bd03c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:21.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfa59c640 1 -- 172.21.15.96:0/687620808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd0439570 con 0x562bd03b1480 2024-09-10T23:01:21.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.920+0000 7efcfad9d640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efcfa59c640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 0x562bd03c1d30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efcec0046d0 tx=0x7efcec024930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794cca17db9e779b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec028070 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcec00edd0 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efcec024aa0 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcec033040 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efd03036740 10 client.18447 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.921+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562bd043a7d0 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcec00edd0 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efcec02d040 con 0x562bd03b1480 2024-09-10T23:01:21.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 10 client.18447 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 20 client.18447 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd043ae20 0x562bd043d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 4 client.18447 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bd0325850 con 0x562bd043ae20 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.922+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd043ae20 0x562bd043d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.923+0000 7efce1ffb640 10 client.18447 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.923+0000 7efcf9d9b640 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd043ae20 0x562bd043d240 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe74485b0ead386 server_cookie=2f9c3120b04a8907 in_seq=0 out_seq=0 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.923+0000 7efce1ffb640 1 -- 172.21.15.96:0/687620808 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x562bd0325850 con 0x562bd043ae20 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.923+0000 7efce1ffb640 10 client.18447 handle_command_reply: tid=0 2024-09-10T23:01:21.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.923+0000 7efd03036740 1 client.18447 shutdown 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 msgr2=0x562bd03c1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 0x562bd03c1d30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efcec0046d0 tx=0x7efcec024930 comp rx=0 tx=0).stop 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 shutdown_connections 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562bd043ae20 0x562bd043d240 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bd03bdd30 0x562bd03c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562bd03b1da0 0x562bd03c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efce1ffb640 0 client.18447 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 --2- 172.21.15.96:0/687620808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562bd03b1480 0x562bd03c1d30 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 >> 172.21.15.96:0/687620808 conn(0x562bd03af2b0 msgr2=0x562bd03adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.924+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 shutdown_connections 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd03036740 1 -- 172.21.15.96:0/687620808 wait complete. 2024-09-10T23:01:21.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd03036740 20 client.18447 trim_cache size 0 max 16384 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:01:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:01:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:01:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:01:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:01:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:01:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:01:21.942 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:01:21.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 msgr2=0x7efce40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 0x7efce40b7820 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009430 tx=0x7efcf40095c0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 msgr2=0x7efcfc197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.925+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efcf000e9f0 tx=0x7efcf000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 shutdown_connections 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcfc1076c0 0x7efcfc197400 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efce40b5360 0x7efce40b7820 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcfc108040 0x7efcfc197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 --2- 172.21.15.96:0/1936155166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcfc101cb0 0x7efcfc196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 >> 172.21.15.96:0/1936155166 conn(0x7efcfc0fd700 msgr2=0x7efcfc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 shutdown_connections 2024-09-10T23:01:21.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:21.926+0000 7efd00c61640 1 -- 172.21.15.96:0/1936155166 wait complete. 2024-09-10T23:01:22.913 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:01:22.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:01:23.301 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:23.627 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:23 smithi096 ceph-mon[30869]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-10T23:01:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:23 smithi121 ceph-mon[35119]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-10T23:01:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:23 smithi117 ceph-mon[35081]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 MiB/s wr, 617 op/s 2024-09-10T23:01:24.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.048+0000 7f11bdd76640 1 -- 172.21.15.96:0/2409552197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 msgr2=0x7f11b810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:24.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.048+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2409552197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810de20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f11ac01a910 tx=0x7f11ac03b450 comp rx=0 tx=0).stop 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 -- 172.21.15.96:0/2409552197 shutdown_connections 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2409552197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810de20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2409552197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b8106ca0 0x7f11b8107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2409552197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11b81012a0 0x7f11b8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 -- 172.21.15.96:0/2409552197 >> 172.21.15.96:0/2409552197 conn(0x7f11b80fd2f0 msgr2=0x7f11b80ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 -- 172.21.15.96:0/2409552197 shutdown_connections 2024-09-10T23:01:24.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.049+0000 7f11bdd76640 1 -- 172.21.15.96:0/2409552197 wait complete. 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 Processor -- start 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 -- start start 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11b8106ca0 0x7f11b810a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11b806f3b0 con 0x7f11b8106ca0 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11b806f230 con 0x7f11b8107640 2024-09-10T23:01:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.050+0000 7f11bdd76640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f11b806f530 con 0x7f11b81012a0 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:43464/0 (socket says 172.21.15.96:43464) 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 -- 172.21.15.96:0/2294458654 learned_addr learned my addr 172.21.15.96:0/2294458654 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bd575640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 -- 172.21.15.96:0/2294458654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 msgr2=0x7f11b810ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 -- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11b8106ca0 msgr2=0x7f11b810a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:24.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11b8106ca0 0x7f11b810a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.051+0000 7f11bcd74640 1 -- 172.21.15.96:0/2294458654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b81bd9a0 con 0x7f11b81012a0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.052+0000 7f11bcd74640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11b0007c90 tx=0x7f11b000e420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.052+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0018070 con 0x7f11b81012a0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.052+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b806fbd0 con 0x7f11b81012a0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.053+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b8070190 con 0x7f11b81012a0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.053+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b000c2d0 con 0x7f11b81012a0 2024-09-10T23:01:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.054+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0013b60 con 0x7f11b81012a0 2024-09-10T23:01:24.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.058+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f11b000eb30 con 0x7f11b81012a0 2024-09-10T23:01:24.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.059+0000 7f11b5ffb640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 0x7f11a00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:24.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.060+0000 7f11b7fff640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 0x7f11a00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:24.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.060+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f11b00e9340 con 0x7f11b81012a0 2024-09-10T23:01:24.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.060+0000 7f11b7fff640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 0x7f11a00b78d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f11b810dca0 tx=0x7f11a8006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:24.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.061+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a40052c0 con 0x7f11b81012a0 2024-09-10T23:01:24.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.066+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f11b00b0200 con 0x7f11b81012a0 2024-09-10T23:01:24.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.400+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a40055b0 con 0x7f11b81012a0 2024-09-10T23:01:24.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.407+0000 7f11b5ffb640 1 -- 172.21.15.96:0/2294458654 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f11b00afb00 con 0x7f11b81012a0 2024-09-10T23:01:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:01:24.410 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:01:24.411 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:01:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.417+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 msgr2=0x7f11a00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.417+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 0x7f11a00b78d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f11b810dca0 tx=0x7f11a8006c40 comp rx=0 tx=0).stop 2024-09-10T23:01:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.417+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 msgr2=0x7f11b810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:24.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.417+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11b0007c90 tx=0x7f11b000e420 comp rx=0 tx=0).stop 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.425+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 shutdown_connections 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.426+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b8107640 0x7f11b810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.426+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f11a00b5410 0x7f11a00b78d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.426+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11b8106ca0 0x7f11b810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.426+0000 7f11bdd76640 1 --2- 172.21.15.96:0/2294458654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11b81012a0 0x7f11b810a190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.426+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 >> 172.21.15.96:0/2294458654 conn(0x7f11b80fd2f0 msgr2=0x7f11b806ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.437+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 shutdown_connections 2024-09-10T23:01:24.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:24.441+0000 7f11bdd76640 1 -- 172.21.15.96:0/2294458654 wait complete. 2024-09-10T23:01:25.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T23:01:25.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:25 smithi096 ceph-mon[30869]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-10T23:01:25.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:25 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2294458654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:25.585 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:25 smithi121 ceph-mon[35119]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-10T23:01:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:25 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2294458654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:25 smithi117 ceph-mon[35081]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 610 op/s 2024-09-10T23:01:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:25 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2294458654' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:01:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 830.0 s, 2092.8 tps, lat 47.504 ms stddev 56.115 2024-09-10T23:01:26.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 -- 172.21.15.96:0/344003555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 msgr2=0x7fda3c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 --2- 172.21.15.96:0/344003555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1040b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009860 tx=0x7fda2c02e750 comp rx=0 tx=0).stop 2024-09-10T23:01:26.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 -- 172.21.15.96:0/344003555 shutdown_connections 2024-09-10T23:01:26.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 --2- 172.21.15.96:0/344003555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 0x7fda3c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 --2- 172.21.15.96:0/344003555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 --2- 172.21.15.96:0/344003555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1040b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 -- 172.21.15.96:0/344003555 >> 172.21.15.96:0/344003555 conn(0x7fda3c0fd770 msgr2=0x7fda3c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 -- 172.21.15.96:0/344003555 shutdown_connections 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.167+0000 7fda416c8640 1 -- 172.21.15.96:0/344003555 wait complete. 2024-09-10T23:01:26.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.168+0000 7fda416c8640 1 Processor -- start 2024-09-10T23:01:26.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.168+0000 7fda416c8640 1 -- start start 2024-09-10T23:01:26.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 0x7fda3c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c110be0 con 0x7fda3c10d130 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c110a60 con 0x7fda3c1045f0 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda416c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c110d60 con 0x7fda3c103cd0 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38390/0 (socket says 172.21.15.96:38390) 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3affd640 1 -- 172.21.15.96:0/1027313024 learned_addr learned my addr 172.21.15.96:0/1027313024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda327fc640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 0x7fda3c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 -- 172.21.15.96:0/1027313024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 msgr2=0x7fda3c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 -- 172.21.15.96:0/1027313024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 msgr2=0x7fda3c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 0x7fda3c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 -- 172.21.15.96:0/1027313024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c19d160 con 0x7fda3c10d130 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c19c990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fda3400ef00 tx=0x7fda3400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.169+0000 7fda38ff9640 1 -- 172.21.15.96:0/1027313024 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34004b20 con 0x7fda3c10d130 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.170+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c19d450 con 0x7fda3c10d130 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.170+0000 7fda38ff9640 1 -- 172.21.15.96:0/1027313024 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda34003e20 con 0x7fda3c10d130 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.170+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a3b10 con 0x7fda3c10d130 2024-09-10T23:01:26.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.172+0000 7fda38ff9640 1 -- 172.21.15.96:0/1027313024 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3401a020 con 0x7fda3c10d130 2024-09-10T23:01:26.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.173+0000 7fda43a9d740 1 Processor -- start 2024-09-10T23:01:26.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.173+0000 7fda38ff9640 1 -- 172.21.15.96:0/1027313024 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fda3400b040 con 0x7fda3c10d130 2024-09-10T23:01:26.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.173+0000 7fda43a9d740 1 -- start start 2024-09-10T23:01:26.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 0x559228611840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559228611d80 0x559228614170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55922857bea0 con 0x559228605df0 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55922856b2d0 con 0x559228611d80 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda43a9d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55922856f0e0 con 0x5592286054d0 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 0x559228611840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38392/0 (socket says 172.21.15.96:38392) 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 -- 172.21.15.96:0/2826283689 learned_addr learned my addr 172.21.15.96:0/2826283689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559228611d80 0x559228614170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 -- 172.21.15.96:0/2826283689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559228611d80 msgr2=0x559228614170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559228611d80 0x559228614170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 -- 172.21.15.96:0/2826283689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 msgr2=0x559228611840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 0x559228611840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda327fc640 1 -- 172.21.15.96:0/2826283689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592282c9650 con 0x5592286054d0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.174+0000 7fda3affd640 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 0x559228611840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda327fc640 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda24012780 tx=0x7fda24004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94eed4a94411604 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda38ff9640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 0x7fda280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda317fa640 1 -- 172.21.15.96:0/2826283689 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24012d00 con 0x5592286054d0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda317fa640 1 -- 172.21.15.96:0/2826283689 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda24010770 con 0x5592286054d0 2024-09-10T23:01:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda317fa640 1 -- 172.21.15.96:0/2826283689 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240044e0 con 0x5592286054d0 2024-09-10T23:01:26.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda3affd640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 0x7fda280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.175+0000 7fda38ff9640 1 -- 172.21.15.96:0/1027313024 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fda340e5900 con 0x7fda3c10d130 2024-09-10T23:01:26.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 -- 172.21.15.96:0/2826283689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 msgr2=0x5592286058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda3affd640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 0x7fda280b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c036ff0 tx=0x7fda2c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda24012780 tx=0x7fda24004730 comp rx=0 tx=0).stop 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 -- 172.21.15.96:0/2826283689 shutdown_connections 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559228611d80 0x559228614170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228605df0 0x559228611840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 --2- 172.21.15.96:0/2826283689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592286054d0 0x5592286058b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.176+0000 7fda43a9d740 1 -- 172.21.15.96:0/2826283689 >> 172.21.15.96:0/2826283689 conn(0x559228603300 msgr2=0x5592286044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.177+0000 7fda43a9d740 1 -- 172.21.15.96:0/2826283689 shutdown_connections 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.177+0000 7fda43a9d740 1 -- 172.21.15.96:0/2826283689 wait complete. 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 -- start start 2024-09-10T23:01:26.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 Processor -- start 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 0x55922860ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228611d80 0x55922860f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5592283cc700 con 0x559228611d80 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559228506160 con 0x5592286054d0 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.178+0000 7fda43a9d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559228614a50 con 0x559228605df0 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda3affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda3affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37614/0 (socket says 172.21.15.96:37614) 2024-09-10T23:01:26.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda3affd640 1 -- 172.21.15.96:0/4000627991 learned_addr learned my addr 172.21.15.96:0/4000627991 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 0x55922860ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda43a9d740 10 client.0 _ll_register_callbacks cb 0x55922851f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228611d80 0x55922860f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 -- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 msgr2=0x559228611620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 -- 172.21.15.96:0/4000627991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228611d80 msgr2=0x55922860f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228611d80 0x55922860f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 -- 172.21.15.96:0/4000627991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559228533c90 con 0x559228605df0 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda3affd640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda327fc640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 0x55922860ea70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda24010740 tx=0x7fda24010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b024c8afbff6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda227fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24024070 con 0x559228605df0 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda24010770 con 0x559228605df0 2024-09-10T23:01:26.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.179+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fda24004450 con 0x559228605df0 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda24030050 con 0x559228605df0 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 10 client.28147 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559228620360 con 0x559228605df0 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda24010770 con 0x559228605df0 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fda2401f460 con 0x559228605df0 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 10 client.28147 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 10 client.28147 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 20 client.28147 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5592286219f0 0x559228623e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 4 client.28147 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55922857ddf0 con 0x5592286219f0 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.180+0000 7fda3affd640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5592286219f0 0x559228623e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda3affd640 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5592286219f0 0x559228623e80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49d9bb55c3ce3990 server_cookie=b695aaf55dc1d3f1 in_seq=0 out_seq=0 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda227fc640 10 client.28147 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda227fc640 1 -- 172.21.15.96:0/4000627991 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55922857ddf0 con 0x5592286219f0 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda227fc640 10 client.28147 handle_command_reply: tid=0 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda43a9d740 1 client.28147 shutdown 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 msgr2=0x55922860ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.181+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 0x55922860ea70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda24010740 tx=0x7fda24010bd0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 shutdown_connections 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x5592286219f0 0x559228623e80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559228611d80 0x55922860f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228605df0 0x55922860ea70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 --2- 172.21.15.96:0/4000627991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x559228611620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 >> 172.21.15.96:0/4000627991 conn(0x559228603300 msgr2=0x559228608ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda227fc640 0 client.28147 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:01:26.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 shutdown_connections 2024-09-10T23:01:26.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 1 -- 172.21.15.96:0/4000627991 wait complete. 2024-09-10T23:01:26.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.182+0000 7fda43a9d740 20 client.28147 trim_cache size 0 max 16384 2024-09-10T23:01:26.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 Processor -- start 2024-09-10T23:01:26.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 -- start start 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x5592286043c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 0x559228604900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ac090 con 0x5592286054d0 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1adbd0 con 0x55922860da00 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda43a9d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ac760 con 0x559228611d80 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34910/0 (socket says 172.21.15.96:34910) 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda3affd640 1 -- 172.21.15.96:0/3887729131 learned_addr learned my addr 172.21.15.96:0/3887729131 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:26.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 0x559228604900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.201+0000 7fda327fc640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x5592286043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 -- 172.21.15.96:0/3887729131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 msgr2=0x5592286043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x5592286043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 -- 172.21.15.96:0/3887729131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 msgr2=0x559228604900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 0x559228604900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 -- 172.21.15.96:0/3887729131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559228605c20 con 0x5592286054d0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 0x559228604900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda3affd640 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009830 tx=0x7fda2c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f1a8616f059840 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda30ff9640 1 -- 172.21.15.96:0/3887729131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c005550 con 0x5592286054d0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda30ff9640 1 -- 172.21.15.96:0/3887729131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c004b70 con 0x5592286054d0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.202+0000 7fda30ff9640 1 -- 172.21.15.96:0/3887729131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c057a50 con 0x5592286054d0 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.203+0000 7fda43a9d740 1 -- 172.21.15.96:0/3887729131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 msgr2=0x559228602c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.203+0000 7fda43a9d740 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009830 tx=0x7fda2c045000 comp rx=0 tx=0).stop 2024-09-10T23:01:26.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.203+0000 7fda43a9d740 1 -- 172.21.15.96:0/3887729131 shutdown_connections 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55922860da00 0x559228604900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x5592286043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 --2- 172.21.15.96:0/3887729131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592286054d0 0x559228602c10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 -- 172.21.15.96:0/3887729131 >> 172.21.15.96:0/3887729131 conn(0x559228603300 msgr2=0x55922860d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 -- 172.21.15.96:0/3887729131 shutdown_connections 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 -- 172.21.15.96:0/3887729131 wait complete. 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.204+0000 7fda43a9d740 1 -- start start 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 Processor -- start 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55922860da00 0x559228613290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x559228613a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ad500 con 0x55922860da00 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ace30 con 0x5592286054d0 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1aa090 con 0x559228611d80 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda43a9d740 10 client.0 _ll_register_callbacks cb 0x7fda2c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x559228613a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda327fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55922860da00 0x559228613290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37632/0 (socket says 172.21.15.96:37632) 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 -- 172.21.15.96:0/648951756 learned_addr learned my addr 172.21.15.96:0/648951756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 -- 172.21.15.96:0/648951756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 msgr2=0x559228613a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x559228613a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 -- 172.21.15.96:0/648951756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55922860da00 msgr2=0x559228613290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55922860da00 0x559228613290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3affd640 1 -- 172.21.15.96:0/648951756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559228619390 con 0x5592286054d0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.205+0000 7fda3b7fe640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x559228613a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda3affd640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0344f0 tx=0x7fda2c057590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aba327e01ab7cad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda227fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c069070 con 0x5592286054d0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fda2c004d20 con 0x5592286054d0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2c004b70 con 0x5592286054d0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda2c075050 con 0x5592286054d0 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda43a9d740 10 client.28088 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:26.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.206+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55922868ec20 con 0x5592286054d0 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.207+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c004b70 con 0x5592286054d0 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.207+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fda2c010270 con 0x5592286054d0 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.207+0000 7fda43a9d740 10 client.28088 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.207+0000 7fda43a9d740 10 client.28088 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.208+0000 7fda43a9d740 20 client.28088 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.208+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55922868f3d0 0x5592286917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.208+0000 7fda43a9d740 4 client.28088 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.208+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592285798a0 con 0x55922868f3d0 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.208+0000 7fda327fc640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55922868f3d0 0x5592286917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:26.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda327fc640 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55922868f3d0 0x5592286917f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=943d68c84661470f server_cookie=e60e503bd33561d0 in_seq=0 out_seq=0 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda227fc640 10 client.28088 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda227fc640 1 -- 172.21.15.96:0/648951756 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592285798a0 con 0x55922868f3d0 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda227fc640 10 client.28088 handle_command_reply: tid=0 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 client.28088 shutdown 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 msgr2=0x55922861a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0344f0 tx=0x7fda2c057590 comp rx=0 tx=0).stop 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 shutdown_connections 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x55922868f3d0 0x5592286917f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559228611d80 0x559228613a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda227fc640 0 client.28088 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:01:26.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55922860da00 0x559228613290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 --2- 172.21.15.96:0/648951756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592286054d0 0x55922861a730 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 >> 172.21.15.96:0/648951756 conn(0x559228603300 msgr2=0x5592286043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 shutdown_connections 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 1 -- 172.21.15.96:0/648951756 wait complete. 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.209+0000 7fda43a9d740 20 client.28088 trim_cache size 0 max 16384 2024-09-10T23:01:26.216 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:01:26.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 msgr2=0x7fda280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 0x7fda280b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda2c036ff0 tx=0x7fda2c0058d0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 msgr2=0x7fda3c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:26.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c19c990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fda3400ef00 tx=0x7fda3400c740 comp rx=0 tx=0).stop 2024-09-10T23:01:26.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 shutdown_connections 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3c1045f0 0x7fda3c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fda280b52e0 0x7fda280b77a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda3c10d130 0x7fda3c19c990 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 --2- 172.21.15.96:0/1027313024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda3c103cd0 0x7fda3c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 >> 172.21.15.96:0/1027313024 conn(0x7fda3c0fd770 msgr2=0x7fda3c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 shutdown_connections 2024-09-10T23:01:26.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:26.210+0000 7fda416c8640 1 -- 172.21.15.96:0/1027313024 wait complete. 2024-09-10T23:01:27.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T23:01:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:27 smithi096 ceph-mon[30869]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-09-10T23:01:27.650 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:27 smithi121 ceph-mon[35119]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-09-10T23:01:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:27 smithi117 ceph-mon[35081]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.6 MiB/s wr, 718 op/s 2024-09-10T23:01:27.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.996+0000 7ff52f916640 1 -- 172.21.15.96:0/1990101083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810aac0 msgr2=0x7ff528111420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.996+0000 7ff52f916640 1 --2- 172.21.15.96:0/1990101083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810aac0 0x7ff528111420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff520009900 tx=0x7ff52002a440 comp rx=0 tx=0).stop 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.998+0000 7ff52f916640 1 -- 172.21.15.96:0/1990101083 shutdown_connections 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.998+0000 7ff52f916640 1 --2- 172.21.15.96:0/1990101083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810aac0 0x7ff528111420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.998+0000 7ff52f916640 1 --2- 172.21.15.96:0/1990101083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52810a100 0x7ff52810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.998+0000 7ff52f916640 1 --2- 172.21.15.96:0/1990101083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff528109230 0x7ff528109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:27.998+0000 7ff52f916640 1 -- 172.21.15.96:0/1990101083 >> 172.21.15.96:0/1990101083 conn(0x7ff5280783d0 msgr2=0x7ff52807a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.002+0000 7ff52f916640 1 -- 172.21.15.96:0/1990101083 shutdown_connections 2024-09-10T23:01:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.008+0000 7ff52f916640 1 -- 172.21.15.96:0/1990101083 wait complete. 2024-09-10T23:01:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 Processor -- start 2024-09-10T23:01:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 -- start start 2024-09-10T23:01:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 0x7ff52819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810a100 0x7ff52819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 0x7ff5281a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff528114d00 con 0x7ff52810aac0 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff528114b80 con 0x7ff52810a100 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.009+0000 7ff52f916640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff528114e80 con 0x7ff528109230 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.010+0000 7ff52d6b2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 0x7ff52819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.010+0000 7ff52d6b2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 0x7ff52819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38452/0 (socket says 172.21.15.96:38452) 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.010+0000 7ff52d6b2640 1 -- 172.21.15.96:0/4218243429 learned_addr learned my addr 172.21.15.96:0/4218243429 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.010+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 0x7ff5281a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.017+0000 7ff52deb3640 1 -- 172.21.15.96:0/4218243429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 msgr2=0x7ff52819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.017+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 0x7ff52819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.017+0000 7ff52deb3640 1 -- 172.21.15.96:0/4218243429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810a100 msgr2=0x7ff52819fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.017+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810a100 0x7ff52819fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.017+0000 7ff52deb3640 1 -- 172.21.15.96:0/4218243429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281a0fc0 con 0x7ff52810aac0 2024-09-10T23:01:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.018+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 0x7ff5281a0700 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff520009850 tx=0x7ff520005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.022+0000 7ff51e7fc640 1 -- 172.21.15.96:0/4218243429 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52003f070 con 0x7ff52810aac0 2024-09-10T23:01:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.022+0000 7ff51e7fc640 1 -- 172.21.15.96:0/4218243429 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff520004e00 con 0x7ff52810aac0 2024-09-10T23:01:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.022+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281a1250 con 0x7ff52810aac0 2024-09-10T23:01:28.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.022+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52807bc00 con 0x7ff52810aac0 2024-09-10T23:01:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.025+0000 7ff51e7fc640 1 -- 172.21.15.96:0/4218243429 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520035030 con 0x7ff52810aac0 2024-09-10T23:01:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.028+0000 7ff532322740 1 Processor -- start 2024-09-10T23:01:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.031+0000 7ff51e7fc640 1 -- 172.21.15.96:0/4218243429 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff520031bd0 con 0x7ff52810aac0 2024-09-10T23:01:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff51e7fc640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 0x7ff5140b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff51e7fc640 1 -- 172.21.15.96:0/4218243429 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff5201065e0 con 0x7ff52810aac0 2024-09-10T23:01:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 -- start start 2024-09-10T23:01:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bda190 0x55f9a1bdc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1b496f0 con 0x55f9a1bda190 2024-09-10T23:01:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1b38b20 con 0x55f9a1bd7860 2024-09-10T23:01:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.032+0000 7ff532322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1b3c930 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.035+0000 7ff52deb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.035+0000 7ff52deb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38460/0 (socket says 172.21.15.96:38460) 2024-09-10T23:01:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.035+0000 7ff52deb3640 1 -- 172.21.15.96:0/4054915723 learned_addr learned my addr 172.21.15.96:0/4054915723 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.035+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.037+0000 7ff52deb3640 1 -- 172.21.15.96:0/4054915723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 msgr2=0x55f9a1bd9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.037+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.037+0000 7ff52deb3640 1 -- 172.21.15.96:0/4054915723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bda190 msgr2=0x55f9a1bdc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.037+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bda190 0x55f9a1bdc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.037+0000 7ff52deb3640 1 -- 172.21.15.96:0/4054915723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a1afe530 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.041+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 0x7ff5140b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.041+0000 7ff52deb3640 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff5200c9370 tx=0x7ff5200c8d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe823766b4018ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.044+0000 7ff5137fe640 1 -- 172.21.15.96:0/4054915723 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520049040 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.044+0000 7ff5137fe640 1 -- 172.21.15.96:0/4054915723 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5200cc8b0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.044+0000 7ff5137fe640 1 -- 172.21.15.96:0/4054915723 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200d5ea0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.045+0000 7ff532322740 1 -- 172.21.15.96:0/4054915723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 msgr2=0x55f9a1bdf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.045+0000 7ff532322740 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff5200c9370 tx=0x7ff5200c8d00 comp rx=0 tx=0).stop 2024-09-10T23:01:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.048+0000 7ff532322740 1 -- 172.21.15.96:0/4054915723 shutdown_connections 2024-09-10T23:01:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.048+0000 7ff532322740 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdf090 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.048+0000 7ff532322740 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bda190 0x55f9a1bdc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.048+0000 7ff532322740 1 --2- 172.21.15.96:0/4054915723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.048+0000 7ff532322740 1 -- 172.21.15.96:0/4054915723 >> 172.21.15.96:0/4054915723 conn(0x55f9a1bd29b0 msgr2=0x55f9a1bd3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.049+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 0x7ff5140b79b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff52400ca60 tx=0x7ff52400b250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.051+0000 7ff532322740 1 -- 172.21.15.96:0/4054915723 shutdown_connections 2024-09-10T23:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.051+0000 7ff532322740 1 -- 172.21.15.96:0/4054915723 wait complete. 2024-09-10T23:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 -- start start 2024-09-10T23:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 Processor -- start 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1be0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bda190 0x55f9a1be1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1a70ab0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1ade730 con 0x55f9a1bd7860 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f9a1bdf970 con 0x55f9a1bda190 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.052+0000 7ff532322740 10 client.0 _ll_register_callbacks cb 0x55f9a1b326b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34958/0 (socket says 172.21.15.96:34958) 2024-09-10T23:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 -- 172.21.15.96:0/705877487 learned_addr learned my addr 172.21.15.96:0/705877487 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 -- 172.21.15.96:0/705877487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bda190 msgr2=0x55f9a1be1230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1be0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bda190 0x55f9a1be1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.053+0000 7ff52deb3640 1 -- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 msgr2=0x55f9a1be0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff52deb3640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1be0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff52deb3640 1 -- 172.21.15.96:0/705877487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a189a790 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1be0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff52deb3640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff5200ca010 tx=0x7ff5200c8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b32cf015c9df87c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52007d020 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.054+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff5200d60b0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.055+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5200cc8b0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.059+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff520049040 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.063+0000 7ff532322740 10 client.18471 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.063+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9a1913620 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5200cc8b0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff5200c8890 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 10 client.18471 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 20 client.18471 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1bec370 0x55f9a1bee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 4 client.18471 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.064+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a1b4b640 con 0x55f9a1bec370 2024-09-10T23:01:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.065+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1bec370 0x55f9a1bee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.066+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1bec370 0x55f9a1bee760 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d162fc8c2e017c7 server_cookie=9f541dd2999606f8 in_seq=0 out_seq=0 2024-09-10T23:01:28.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.066+0000 7ff510ff9640 10 client.18471 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.067+0000 7ff510ff9640 1 -- 172.21.15.96:0/705877487 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9a1b4b640 con 0x55f9a1bec370 2024-09-10T23:01:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.067+0000 7ff510ff9640 10 client.18471 handle_command_reply: tid=0 2024-09-10T23:01:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.067+0000 7ff532322740 1 client.18471 shutdown 2024-09-10T23:01:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 msgr2=0x55f9a1be1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff5200ca010 tx=0x7ff5200c8d70 comp rx=0 tx=0).stop 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 shutdown_connections 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1bec370 0x55f9a1bee760 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be1e00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bda190 0x55f9a1be1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 --2- 172.21.15.96:0/705877487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bd7860 0x55f9a1be0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 >> 172.21.15.96:0/705877487 conn(0x55f9a1bd29b0 msgr2=0x55f9a1bd1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.069+0000 7ff510ff9640 0 client.18471 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.070+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 shutdown_connections 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.070+0000 7ff532322740 1 -- 172.21.15.96:0/705877487 wait complete. 2024-09-10T23:01:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.070+0000 7ff532322740 20 client.18471 trim_cache size 0 max 16384 2024-09-10T23:01:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.095+0000 7ff532322740 1 Processor -- start 2024-09-10T23:01:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.095+0000 7ff532322740 1 -- start start 2024-09-10T23:01:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1bd7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00e410 con 0x55f9a1bd7860 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00ff50 con 0x55f9a1bda190 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff532322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00eae0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52d6b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52d6b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34962/0 (socket says 172.21.15.96:34962) 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52d6b2640 1 -- 172.21.15.96:0/990469819 learned_addr learned my addr 172.21.15.96:0/990469819 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1bd7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52deb3640 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52d6b2640 1 -- 172.21.15.96:0/990469819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 msgr2=0x55f9a1bdfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.096+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff52d6b2640 1 -- 172.21.15.96:0/990469819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 msgr2=0x55f9a1bd7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1bd7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff52d6b2640 1 -- 172.21.15.96:0/990469819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a1bd6940 con 0x55f9a1bd7860 2024-09-10T23:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff518009380 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37b50ed3c46654 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff512ffd640 1 -- 172.21.15.96:0/990469819 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518016e60 con 0x55f9a1bd7860 2024-09-10T23:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff512ffd640 1 -- 172.21.15.96:0/990469819 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff518020030 con 0x55f9a1bd7860 2024-09-10T23:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.097+0000 7ff512ffd640 1 -- 172.21.15.96:0/990469819 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518014610 con 0x55f9a1bd7860 2024-09-10T23:01:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- 172.21.15.96:0/990469819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 msgr2=0x55f9a1bd6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff518009380 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-09-10T23:01:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- 172.21.15.96:0/990469819 shutdown_connections 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1bd7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1bdfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 --2- 172.21.15.96:0/990469819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1bd6cc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- 172.21.15.96:0/990469819 >> 172.21.15.96:0/990469819 conn(0x55f9a1bd29b0 msgr2=0x55f9a1bd5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- 172.21.15.96:0/990469819 shutdown_connections 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- 172.21.15.96:0/990469819 wait complete. 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 -- start start 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.098+0000 7ff532322740 1 Processor -- start 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1be5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00f880 con 0x55f9a1bd7860 2024-09-10T23:01:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52d6b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1be5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37676/0 (socket says 172.21.15.96:37676) 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52deb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 -- 172.21.15.96:0/4052477856 learned_addr learned my addr 172.21.15.96:0/4052477856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00f1b0 con 0x55f9a1bda190 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00c3f0 con 0x55f9a1bdcbd0 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 -- 172.21.15.96:0/4052477856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 msgr2=0x55f9a1be67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 -- 172.21.15.96:0/4052477856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 msgr2=0x55f9a1be5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1be5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.099+0000 7ff52ceb1640 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a1be7010 con 0x55f9a1bda190 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff52ceb1640 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff52401a750 tx=0x7ff524006c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f138d4f9cf841a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff510ff9640 1 -- 172.21.15.96:0/4052477856 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524027060 con 0x55f9a1bda190 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff510ff9640 1 -- 172.21.15.96:0/4052477856 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff52400b7b0 con 0x55f9a1bda190 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff532322740 10 client.0 _ll_register_callbacks cb 0x7ff5200d6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.100+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff518020030 con 0x55f9a1bda190 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.101+0000 7ff510ff9640 1 -- 172.21.15.96:0/4052477856 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff52401ae60 con 0x55f9a1bda190 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.101+0000 7ff532322740 10 client.28100 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.101+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55f9a1be8310 con 0x55f9a1bda190 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff510ff9640 1 -- 172.21.15.96:0/4052477856 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff524021030 con 0x55f9a1bda190 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 10 client.28100 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 10 client.28100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 20 client.28100 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1c59230 0x55f9a1c5b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 4 client.28100 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff518020030 con 0x55f9a1c59230 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.102+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1c59230 0x55f9a1c5b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.103+0000 7ff510ff9640 10 client.28100 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.103+0000 7ff52d6b2640 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1c59230 0x55f9a1c5b620 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3e3086b0cf8d1c server_cookie=4efc8ee49221932b in_seq=0 out_seq=0 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff510ff9640 1 -- 172.21.15.96:0/4052477856 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff518020030 con 0x55f9a1c59230 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff510ff9640 10 client.28100 handle_command_reply: tid=0 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 client.28100 shutdown 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 msgr2=0x55f9a1be5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff52401a750 tx=0x7ff524006c30 comp rx=0 tx=0).stop 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 shutdown_connections 2024-09-10T23:01:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55f9a1c59230 0x55f9a1c5b620 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9a1bda190 0x55f9a1be5c10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f9a1bdcbd0 0x55f9a1be67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 --2- 172.21.15.96:0/4052477856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f9a1bd7860 0x55f9a1be5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff510ff9640 0 client.28100 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 >> 172.21.15.96:0/4052477856 conn(0x55f9a1bd29b0 msgr2=0x55f9a1bd6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.117+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 shutdown_connections 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff532322740 1 -- 172.21.15.96:0/4052477856 wait complete. 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff532322740 20 client.28100 trim_cache size 0 max 16384 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 msgr2=0x7ff5140b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 0x7ff5140b79b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff52400ca60 tx=0x7ff52400b250 comp rx=0 tx=0).stop 2024-09-10T23:01:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 msgr2=0x7ff5281a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 0x7ff5281a0700 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff520009850 tx=0x7ff520005230 comp rx=0 tx=0).stop 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 shutdown_connections 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52810a100 0x7ff52819fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff5140b54f0 0x7ff5140b79b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff52810aac0 0x7ff5281a0700 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 --2- 172.21.15.96:0/4218243429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff528109230 0x7ff52819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.118+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 >> 172.21.15.96:0/4218243429 conn(0x7ff5280783d0 msgr2=0x7ff52810d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.119+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 shutdown_connections 2024-09-10T23:01:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:28.119+0000 7ff52f916640 1 -- 172.21.15.96:0/4218243429 wait complete. 2024-09-10T23:01:28.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.216+0000 7fac78e0e640 1 -- 172.21.15.96:0/961683472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410f750 msgr2=0x7fac74115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.216+0000 7fac78e0e640 1 --2- 172.21.15.96:0/961683472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410f750 0x7fac74115f50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009ca0 tx=0x7fac6c02a8f0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 -- 172.21.15.96:0/961683472 shutdown_connections 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 --2- 172.21.15.96:0/961683472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410f750 0x7fac74115f50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 --2- 172.21.15.96:0/961683472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac7410edd0 0x7fac7410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 --2- 172.21.15.96:0/961683472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac741093c0 0x7fac741097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 -- 172.21.15.96:0/961683472 >> 172.21.15.96:0/961683472 conn(0x7fac74078aa0 msgr2=0x7fac7407aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 -- 172.21.15.96:0/961683472 shutdown_connections 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.217+0000 7fac78e0e640 1 -- 172.21.15.96:0/961683472 wait complete. 2024-09-10T23:01:28.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.218+0000 7fac78e0e640 1 Processor -- start 2024-09-10T23:01:28.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.218+0000 7fac78e0e640 1 -- start start 2024-09-10T23:01:28.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410edd0 0x7fac74111010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac7259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac7259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38484/0 (socket says 172.21.15.96:38484) 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac7259c640 1 -- 172.21.15.96:0/2478507502 learned_addr learned my addr 172.21.15.96:0/2478507502 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7410f750 0x7fac74111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac7407d870 con 0x7fac7410f750 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac7407d6f0 con 0x7fac7410edd0 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.219+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac7407d9f0 con 0x7fac741093c0 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac6bfff640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410edd0 0x7fac74111010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 -- 172.21.15.96:0/2478507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410edd0 msgr2=0x7fac74111010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410edd0 0x7fac74111010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 -- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7410f750 msgr2=0x7fac74111670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7410f750 0x7fac74111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74073bb0 con 0x7fac741093c0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.220+0000 7fac7259c640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fac6400d950 tx=0x7fac6400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.222+0000 7fac6b7fe640 1 -- 172.21.15.96:0/2478507502 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64017070 con 0x7fac741093c0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.222+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac74073ea0 con 0x7fac741093c0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.223+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac74074320 con 0x7fac741093c0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.223+0000 7fac6b7fe640 1 -- 172.21.15.96:0/2478507502 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac64009c30 con 0x7fac741093c0 2024-09-10T23:01:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.223+0000 7fac6b7fe640 1 -- 172.21.15.96:0/2478507502 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400f620 con 0x7fac741093c0 2024-09-10T23:01:28.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.229+0000 7fac7b1e3740 1 Processor -- start 2024-09-10T23:01:28.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.230+0000 7fac6b7fe640 1 -- 172.21.15.96:0/2478507502 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fac64023060 con 0x7fac741093c0 2024-09-10T23:01:28.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 -- start start 2024-09-10T23:01:28.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e7bf13ea0 con 0x563e7bf9ddf0 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e7bf032d0 con 0x563e7bf9d4d0 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e7bf070e0 con 0x563e7bfa9d80 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac6b7fe640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 0x7fac600b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38498/0 (socket says 172.21.15.96:38498) 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.231+0000 7fac6bfff640 1 -- 172.21.15.96:0/2461993216 learned_addr learned my addr 172.21.15.96:0/2461993216 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac7259c640 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 -- 172.21.15.96:0/2461993216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 msgr2=0x563e7bf9d8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9d8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 -- 172.21.15.96:0/2461993216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 msgr2=0x563e7bfa9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 -- 172.21.15.96:0/2461993216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7bc61650 con 0x563e7bfa9d80 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6b7fe640 1 -- 172.21.15.96:0/2478507502 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac640e53c0 con 0x7fac741093c0 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.232+0000 7fac6bfff640 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009c90 tx=0x7fac5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955ab0d314b57abc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.233+0000 7fac68ff9640 1 -- 172.21.15.96:0/2461993216 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c005220 con 0x563e7bfa9d80 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.233+0000 7fac68ff9640 1 -- 172.21.15.96:0/2461993216 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c004c50 con 0x563e7bfa9d80 2024-09-10T23:01:28.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.233+0000 7fac68ff9640 1 -- 172.21.15.96:0/2461993216 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c019830 con 0x563e7bfa9d80 2024-09-10T23:01:28.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac72d9d640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 0x7fac600b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 -- 172.21.15.96:0/2461993216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 msgr2=0x563e7bfac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009c90 tx=0x7fac5c011810 comp rx=0 tx=0).stop 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 -- 172.21.15.96:0/2461993216 shutdown_connections 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bfa9d80 0x563e7bfac170 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/2461993216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.234+0000 7fac7b1e3740 1 -- 172.21.15.96:0/2461993216 >> 172.21.15.96:0/2461993216 conn(0x563e7bf9b300 msgr2=0x563e7bf9c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.235+0000 7fac7b1e3740 1 -- 172.21.15.96:0/2461993216 shutdown_connections 2024-09-10T23:01:28.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.235+0000 7fac7b1e3740 1 -- 172.21.15.96:0/2461993216 wait complete. 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.235+0000 7fac7b1e3740 1 -- start start 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.235+0000 7fac7b1e3740 1 Processor -- start 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfa6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563e7bd64700 con 0x563e7bfa9d80 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e7be9e160 con 0x563e7bf9d4d0 2024-09-10T23:01:28.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e7bfaca00 con 0x563e7bf9ddf0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7b1e3740 10 client.0 _ll_register_callbacks cb 0x563e7beb79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfa6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfa6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35010/0 (socket says 172.21.15.96:35010) 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac72d9d640 1 -- 172.21.15.96:0/1517101403 learned_addr learned my addr 172.21.15.96:0/1517101403 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac6bfff640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 -- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9ddf0 msgr2=0x563e7bfa6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 -- 172.21.15.96:0/1517101403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 msgr2=0x563e7bfa6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfa6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.236+0000 7fac7259c640 1 -- 172.21.15.96:0/1517101403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7becbc90 con 0x563e7bf9d4d0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac7259c640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa95f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x563e7bf9c1e0 tx=0x7fac6401f760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7865c8f4e380004 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac640ac170 con 0x563e7bf9d4d0 2024-09-10T23:01:28.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac640b5920 con 0x563e7bf9d4d0 2024-09-10T23:01:28.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5c004c50 con 0x563e7bf9d4d0 2024-09-10T23:01:28.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.237+0000 7fac72d9d640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 0x7fac600b79a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fac6c02aee0 tx=0x7fac6c02adf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac640abf90 con 0x563e7bf9d4d0 2024-09-10T23:01:28.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 10 client.28109 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:28.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e7bfb83a0 con 0x563e7bf9d4d0 2024-09-10T23:01:28.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c004c50 con 0x563e7bf9d4d0 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac6405c1f0 con 0x563e7bf9d4d0 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 10 client.28109 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 10 client.28109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 20 client.28109 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7bfb9860 0x563e7bfbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 4 client.28109 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e7bf15df0 con 0x563e7bfb9860 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.238+0000 7fac6bfff640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7bfb9860 0x563e7bfbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.239+0000 7fac6bfff640 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7bfb9860 0x563e7bfbbc50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4477ae0a88a8fb5d server_cookie=cd8bfcd70cdc7e7a in_seq=0 out_seq=0 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.239+0000 7fac59ffb640 10 client.28109 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac59ffb640 1 -- 172.21.15.96:0/1517101403 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e7bf15df0 con 0x563e7bfb9860 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac59ffb640 10 client.28109 handle_command_reply: tid=0 2024-09-10T23:01:28.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 client.28109 shutdown 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 msgr2=0x563e7bfa95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa95f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x563e7bf9c1e0 tx=0x7fac6401f760 comp rx=0 tx=0).stop 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 shutdown_connections 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7bfb9860 0x563e7bfbbc50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfa6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9ddf0 0x563e7bfa6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/1517101403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa95f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 >> 172.21.15.96:0/1517101403 conn(0x563e7bf9b300 msgr2=0x563e7bfa0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac59ffb640 0 client.28109 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 shutdown_connections 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 1 -- 172.21.15.96:0/1517101403 wait complete. 2024-09-10T23:01:28.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.240+0000 7fac7b1e3740 20 client.28109 trim_cache size 0 max 16384 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.256+0000 7fac7b1e3740 1 Processor -- start 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 -- start start 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400e3f0 con 0x563e7bfa9d80 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400ff30 con 0x563e7bf9ddf0 2024-09-10T23:01:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400eac0 con 0x563e7bf9d4d0 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac72d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35020/0 (socket says 172.21.15.96:35020) 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37738/0 (socket says 172.21.15.96:37738) 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.257+0000 7fac72d9d640 1 -- 172.21.15.96:0/330352979 learned_addr learned my addr 172.21.15.96:0/330352979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 -- 172.21.15.96:0/330352979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 msgr2=0x563e7bf9ac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 -- 172.21.15.96:0/330352979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 msgr2=0x563e7bfab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 -- 172.21.15.96:0/330352979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7bf9bfe0 con 0x563e7bf9ddf0 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac6bfff640 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0058c0 tx=0x7fac5c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ff777693fbdfb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac5bfff640 1 -- 172.21.15.96:0/330352979 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0035f0 con 0x563e7bf9ddf0 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.258+0000 7fac5bfff640 1 -- 172.21.15.96:0/330352979 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c003c40 con 0x563e7bf9ddf0 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.259+0000 7fac5bfff640 1 -- 172.21.15.96:0/330352979 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0199d0 con 0x563e7bf9ddf0 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.259+0000 7fac7b1e3740 1 -- 172.21.15.96:0/330352979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 msgr2=0x563e7bfab170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.259+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0058c0 tx=0x7fac5c000bc0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 -- 172.21.15.96:0/330352979 shutdown_connections 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bf9ddf0 0x563e7bfab170 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bfa9d80 0x563e7bfab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/330352979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bf9ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 -- 172.21.15.96:0/330352979 >> 172.21.15.96:0/330352979 conn(0x563e7bf9b300 msgr2=0x563e7bfa5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 -- 172.21.15.96:0/330352979 shutdown_connections 2024-09-10T23:01:28.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.260+0000 7fac7b1e3740 1 -- 172.21.15.96:0/330352979 wait complete. 2024-09-10T23:01:28.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 -- start start 2024-09-10T23:01:28.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 Processor -- start 2024-09-10T23:01:28.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bfa9d80 0x563e7bfb24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400f860 con 0x563e7bf9ddf0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400f190 con 0x563e7bfa9d80 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac5400c3d0 con 0x563e7bf9d4d0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7b1e3740 10 client.0 _ll_register_callbacks cb 0x7fac64005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35026/0 (socket says 172.21.15.96:35026) 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 -- 172.21.15.96:0/719269897 learned_addr learned my addr 172.21.15.96:0/719269897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac7259c640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac72d9d640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bfa9d80 0x563e7bfb24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 -- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 msgr2=0x563e7bfa4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 -- 172.21.15.96:0/719269897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bfa9d80 msgr2=0x563e7bfb24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.261+0000 7fac6bfff640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bfa9d80 0x563e7bfb24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.262+0000 7fac6bfff640 1 -- 172.21.15.96:0/719269897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7bfc1430 con 0x563e7bf9ddf0 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.262+0000 7fac6bfff640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009a50 tx=0x7fac5c0096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f8a9bce4bed8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.262+0000 7fac59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.263+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004040 con 0x563e7bf9ddf0 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.263+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fac5c0035f0 con 0x563e7bf9ddf0 2024-09-10T23:01:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.263+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5c003c40 con 0x563e7bf9ddf0 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac5c003830 con 0x563e7bf9ddf0 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 10 client.18492 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e7c026a10 con 0x563e7bf9ddf0 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5c003c40 con 0x563e7bf9ddf0 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fac5c03f570 con 0x563e7bf9ddf0 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 10 client.18492 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 10 client.18492 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 20 client.18492 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7c026e40 0x563e7c029260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 4 client.18492 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e7bf118a0 con 0x563e7c026e40 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.264+0000 7fac7259c640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7c026e40 0x563e7c029260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7259c640 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7c026e40 0x563e7c029260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=797fdc058c8e2ef6 server_cookie=e62117cef4c38d07 in_seq=0 out_seq=0 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac59ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac59ffb640 1 -- 172.21.15.96:0/719269897 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e7bf118a0 con 0x563e7c026e40 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac59ffb640 10 client.18492 handle_command_reply: tid=0 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 client.18492 shutdown 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 msgr2=0x563e7bfb1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009a50 tx=0x7fac5c0096b0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 shutdown_connections 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x563e7c026e40 0x563e7c029260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e7bfa9d80 0x563e7bfb24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563e7bf9ddf0 0x563e7bfb1d40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 --2- 172.21.15.96:0/719269897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e7bf9d4d0 0x563e7bfa4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 >> 172.21.15.96:0/719269897 conn(0x563e7bf9b300 msgr2=0x563e7bf9c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.265+0000 7fac59ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:01:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.266+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 shutdown_connections 2024-09-10T23:01:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.266+0000 7fac7b1e3740 1 -- 172.21.15.96:0/719269897 wait complete. 2024-09-10T23:01:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.266+0000 7fac7b1e3740 20 client.18492 trim_cache size 0 max 16384 2024-09-10T23:01:28.271 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 msgr2=0x7fac600b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 0x7fac600b79a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fac6c02aee0 tx=0x7fac6c02adf0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 msgr2=0x7fac74115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fac6400d950 tx=0x7fac6400de10 comp rx=0 tx=0).stop 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 shutdown_connections 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac7410edd0 0x7fac74111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fac600b54e0 0x7fac600b79a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac7410f750 0x7fac74111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 --2- 172.21.15.96:0/2478507502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac741093c0 0x7fac74115de0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:28.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 >> 172.21.15.96:0/2478507502 conn(0x7fac74078aa0 msgr2=0x7fac74108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:28.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 shutdown_connections 2024-09-10T23:01:28.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:28.267+0000 7fac78e0e640 1 -- 172.21.15.96:0/2478507502 wait complete. 2024-09-10T23:01:29.333 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T23:01:29.677 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:29.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:29 smithi096 ceph-mon[30869]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 564 op/s 2024-09-10T23:01:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:29 smithi121 ceph-mon[35119]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 564 op/s 2024-09-10T23:01:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:29 smithi117 ceph-mon[35081]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 564 op/s 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 -- 172.21.15.96:0/1801469106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d81087c0 msgr2=0x7f33d8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/1801469106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d81087c0 0x7f33d8108c00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009900 tx=0x7f33cc02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 -- 172.21.15.96:0/1801469106 shutdown_connections 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/1801469106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8109140 0x7f33d810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/1801469106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d81087c0 0x7f33d8108c00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/1801469106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d8102db0 0x7f33d8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 -- 172.21.15.96:0/1801469106 >> 172.21.15.96:0/1801469106 conn(0x7f33d80fe860 msgr2=0x7f33d8100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 -- 172.21.15.96:0/1801469106 shutdown_connections 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.268+0000 7f33dcd3b640 1 -- 172.21.15.96:0/1801469106 wait complete. 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 Processor -- start 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 -- start start 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8102db0 0x7f33d8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d81087c0 0x7f33d8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8112e00 con 0x7f33d8109140 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8112c80 con 0x7f33d81087c0 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33dcd3b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8112f80 con 0x7f33d8102db0 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35052/0 (socket says 172.21.15.96:35052) 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33d6d9d640 1 -- 172.21.15.96:0/2140423807 learned_addr learned my addr 172.21.15.96:0/2140423807 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.269+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d81087c0 0x7f33d8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d659c640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8102db0 0x7f33d8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 -- 172.21.15.96:0/2140423807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8102db0 msgr2=0x7f33d8199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8102db0 0x7f33d8199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 -- 172.21.15.96:0/2140423807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d81087c0 msgr2=0x7f33d8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d81087c0 0x7f33d8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 -- 172.21.15.96:0/2140423807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d819f2c0 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.270+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f33d000cce0 tx=0x7f33d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.272+0000 7f33c77fe640 1 -- 172.21.15.96:0/2140423807 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d000f450 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.272+0000 7f33c77fe640 1 -- 172.21.15.96:0/2140423807 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d000faa0 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.272+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d819f5b0 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.272+0000 7f33c77fe640 1 -- 172.21.15.96:0/2140423807 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0018b70 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.273+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d81a5c70 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.276+0000 7f33df110740 1 Processor -- start 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.276+0000 7f33c77fe640 1 -- 172.21.15.96:0/2140423807 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f33d000f5f0 con 0x7f33d8109140 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.276+0000 7f33df110740 1 -- start start 2024-09-10T23:01:30.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2b4a29ea0 con 0x55a2b4ac4190 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2b4a192d0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33df110740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2b4a1d0e0 con 0x55a2b4ab8200 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37794/0 (socket says 172.21.15.96:37794) 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 -- 172.21.15.96:0/4280864198 learned_addr learned my addr 172.21.15.96:0/4280864198 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d659c640 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 -- 172.21.15.96:0/4280864198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 msgr2=0x55a2b4ac3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 -- 172.21.15.96:0/4280864198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 msgr2=0x55a2b4ac6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.277+0000 7f33d5d9b640 1 -- 172.21.15.96:0/4280864198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b4777650 con 0x55a2b4ab78e0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f33cc032fd0 tx=0x7f33cc005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8d69dce81024a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33c77fe640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 0x7f33c00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33c4ff9640 1 -- 172.21.15.96:0/4280864198 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc035070 con 0x55a2b4ab78e0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33c4ff9640 1 -- 172.21.15.96:0/4280864198 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc0035f0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33c4ff9640 1 -- 172.21.15.96:0/4280864198 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0044e0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33d659c640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 0x7f33c00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.278+0000 7f33c77fe640 1 -- 172.21.15.96:0/2140423807 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f33d00e61f0 con 0x7f33d8109140 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33d659c640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 0x7f33c00b7820 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a2b4ab65f0 tx=0x7f33c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 -- 172.21.15.96:0/4280864198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 msgr2=0x55a2b4ab7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f33cc032fd0 tx=0x7f33cc005db0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 -- 172.21.15.96:0/4280864198 shutdown_connections 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 --2- 172.21.15.96:0/4280864198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ab7cc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 -- 172.21.15.96:0/4280864198 >> 172.21.15.96:0/4280864198 conn(0x55a2b4ab5710 msgr2=0x55a2b4ab68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 -- 172.21.15.96:0/4280864198 shutdown_connections 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.279+0000 7f33df110740 1 -- 172.21.15.96:0/4280864198 wait complete. 2024-09-10T23:01:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 -- start start 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 Processor -- start 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2b487a700 con 0x55a2b4ac4190 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a2b49b4160 con 0x55a2b4ab78e0 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2b4ac6db0 con 0x55a2b4ab8200 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35066/0 (socket says 172.21.15.96:35066) 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d6d9d640 1 -- 172.21.15.96:0/721890697 learned_addr learned my addr 172.21.15.96:0/721890697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33df110740 10 client.0 _ll_register_callbacks cb 0x55a2b49cd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.280+0000 7f33d659c640 1 -- 172.21.15.96:0/721890697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 msgr2=0x55a2b4ac0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33d659c640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33d659c640 1 -- 172.21.15.96:0/721890697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 msgr2=0x55a2b4ac15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33d659c640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33d659c640 1 -- 172.21.15.96:0/721890697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b49e1c90 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33d659c640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac3b00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33c8012cb0 tx=0x7f33c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea75f4fe227cdcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8003d00 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.281+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f33c802a420 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.282+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc0035f0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.283+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33c802a620 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.283+0000 7f33df110740 10 client.28127 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.283+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2b4ad27c0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.283+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc0035f0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f33c80205f0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 10 client.28127 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 10 client.28127 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 20 client.28127 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4ad4070 0x55a2b4ad6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 4 client.28127 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2b4a2bdf0 con 0x55a2b4ad4070 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4ad4070 0x55a2b4ad6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33bdffb640 10 client.28127 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.284+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4ad4070 0x55a2b4ad6510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7f89352c14086d5 server_cookie=70d6dd53915f70a0 in_seq=0 out_seq=0 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.285+0000 7f33bdffb640 1 -- 172.21.15.96:0/721890697 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2b4a2bdf0 con 0x55a2b4ad4070 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.285+0000 7f33bdffb640 10 client.28127 handle_command_reply: tid=0 2024-09-10T23:01:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.285+0000 7f33df110740 1 client.28127 shutdown 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.285+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 msgr2=0x55a2b4ac3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.285+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac3b00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33c8012cb0 tx=0x7f33c8004730 comp rx=0 tx=0).stop 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 shutdown_connections 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4ad4070 0x55a2b4ad6510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ac4190 0x55a2b4ac15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4ac0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 --2- 172.21.15.96:0/721890697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac3b00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 >> 172.21.15.96:0/721890697 conn(0x55a2b4ab5710 msgr2=0x55a2b4ab0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33bdffb640 0 client.28127 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 shutdown_connections 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 1 -- 172.21.15.96:0/721890697 wait complete. 2024-09-10T23:01:30.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.286+0000 7f33df110740 20 client.28127 trim_cache size 0 max 16384 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 Processor -- start 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 -- start start 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4abc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81ac030 con 0x55a2b4ab8200 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81adb70 con 0x55a2b4ac4190 2024-09-10T23:01:30.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33df110740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81ac700 con 0x55a2b4ab78e0 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33d5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33d5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35072/0 (socket says 172.21.15.96:35072) 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33d5d9b640 1 -- 172.21.15.96:0/1971938674 learned_addr learned my addr 172.21.15.96:0/1971938674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4abc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.302+0000 7f33d659c640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 -- 172.21.15.96:0/1971938674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab78e0 msgr2=0x55a2b4ac0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 -- 172.21.15.96:0/1971938674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 msgr2=0x55a2b4abc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4abc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 -- 172.21.15.96:0/1971938674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b4ac1940 con 0x55a2b4ab8200 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4abc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.303+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0096c0 tx=0x7f33cc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3679238e3c36edde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.305+0000 7f33bffff640 1 -- 172.21.15.96:0/1971938674 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc035040 con 0x55a2b4ab8200 2024-09-10T23:01:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.305+0000 7f33bffff640 1 -- 172.21.15.96:0/1971938674 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc009780 con 0x55a2b4ab8200 2024-09-10T23:01:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.305+0000 7f33bffff640 1 -- 172.21.15.96:0/1971938674 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0049b0 con 0x55a2b4ab8200 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 -- 172.21.15.96:0/1971938674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 msgr2=0x55a2b4abc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0096c0 tx=0x7f33cc031400 comp rx=0 tx=0).stop 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 -- 172.21.15.96:0/1971938674 shutdown_connections 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4abc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab8200 0x55a2b4abc160 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 --2- 172.21.15.96:0/1971938674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab78e0 0x55a2b4ac0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 -- 172.21.15.96:0/1971938674 >> 172.21.15.96:0/1971938674 conn(0x55a2b4ab5710 msgr2=0x55a2b4ab67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 -- 172.21.15.96:0/1971938674 shutdown_connections 2024-09-10T23:01:30.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.306+0000 7f33df110740 1 -- 172.21.15.96:0/1971938674 wait complete. 2024-09-10T23:01:30.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 -- start start 2024-09-10T23:01:30.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 Processor -- start 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4adb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4adbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81ad4a0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81acdd0 con 0x55a2b4ac4190 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81aa030 con 0x55a2b4ab8200 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33df110740 10 client.0 _ll_register_callbacks cb 0x7f33c8009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.307+0000 7f33d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35086/0 (socket says 172.21.15.96:35086) 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 -- 172.21.15.96:0/1155651328 learned_addr learned my addr 172.21.15.96:0/1155651328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:30.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4adb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d6d9d640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4adbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 -- 172.21.15.96:0/1155651328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 msgr2=0x55a2b4adb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4adb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 -- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 msgr2=0x55a2b4adbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4adbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 -- 172.21.15.96:0/1155651328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b4adc4a0 con 0x55a2b4ab78e0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33d659c640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f33c8000c00 tx=0x7f33c8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a78baf06e93924 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c802a420 con 0x55a2b4ab78e0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f33c8020410 con 0x55a2b4ab78e0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.308+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc009780 con 0x55a2b4ab78e0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33c8020610 con 0x55a2b4ab78e0 2024-09-10T23:01:30.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 10 client.18510 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2b4b41020 con 0x55a2b4ab78e0 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc009780 con 0x55a2b4ab78e0 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f33c8048070 con 0x55a2b4ab78e0 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 10 client.18510 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 10 client.18510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 20 client.18510 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4b416b0 0x55a2b4b43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 4 client.18510 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2b4a278a0 con 0x55a2b4b416b0 2024-09-10T23:01:30.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.309+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4b416b0 0x55a2b4b43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33d5d9b640 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4b416b0 0x55a2b4b43ad0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6565c3de62d2c25 server_cookie=90947772c40cc017 in_seq=0 out_seq=0 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33bdffb640 10 client.18510 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33bdffb640 1 -- 172.21.15.96:0/1155651328 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2b4a278a0 con 0x55a2b4b416b0 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33bdffb640 10 client.18510 handle_command_reply: tid=0 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 client.18510 shutdown 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 msgr2=0x55a2b4acc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f33c8000c00 tx=0x7f33c8009f80 comp rx=0 tx=0).stop 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 shutdown_connections 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55a2b4b416b0 0x55a2b4b43ad0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a2b4ac4190 0x55a2b4adbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b4ab8200 0x55a2b4adb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 --2- 172.21.15.96:0/1155651328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b4ab78e0 0x55a2b4acc6a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33bdffb640 0 client.18510 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.310+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 >> 172.21.15.96:0/1155651328 conn(0x55a2b4ab5710 msgr2=0x55a2b4abc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 shutdown_connections 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33df110740 1 -- 172.21.15.96:0/1155651328 wait complete. 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33df110740 20 client.18510 trim_cache size 0 max 16384 2024-09-10T23:01:30.316 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 msgr2=0x7f33c00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 0x7f33c00b7820 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a2b4ab65f0 tx=0x7f33c8005f90 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 msgr2=0x7f33d819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.311+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f33d000cce0 tx=0x7f33d000a620 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 shutdown_connections 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d81087c0 0x7f33d8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f33c00b5360 0x7f33c00b7820 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33d8109140 0x7f33d819eaf0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 --2- 172.21.15.96:0/2140423807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33d8102db0 0x7f33d8199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:30.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 >> 172.21.15.96:0/2140423807 conn(0x7f33d80fe860 msgr2=0x7f33d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 shutdown_connections 2024-09-10T23:01:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:30.312+0000 7f33dcd3b640 1 -- 172.21.15.96:0/2140423807 wait complete. 2024-09-10T23:01:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 835.0 s, 2122.2 tps, lat 47.076 ms stddev 57.070 2024-09-10T23:01:31.118 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T23:01:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:31 smithi096 ceph-mon[30869]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 762 op/s 2024-09-10T23:01:31.524 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:31 smithi121 ceph-mon[35119]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 762 op/s 2024-09-10T23:01:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:31 smithi117 ceph-mon[35081]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 762 op/s 2024-09-10T23:01:32.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.164+0000 7f1636e66640 1 -- 172.21.15.96:0/688397438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 msgr2=0x7f1630108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.164+0000 7f1636e66640 1 --2- 172.21.15.96:0/688397438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630108c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f161c009900 tx=0x7f161c02a410 comp rx=0 tx=0).stop 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 -- 172.21.15.96:0/688397438 shutdown_connections 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 --2- 172.21.15.96:0/688397438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 --2- 172.21.15.96:0/688397438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630108c30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 --2- 172.21.15.96:0/688397438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f16301031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 -- 172.21.15.96:0/688397438 >> 172.21.15.96:0/688397438 conn(0x7f16300fe870 msgr2=0x7f1630100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 -- 172.21.15.96:0/688397438 shutdown_connections 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.165+0000 7f1636e66640 1 -- 172.21.15.96:0/688397438 wait complete. 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 Processor -- start 2024-09-10T23:01:32.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 -- start start 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f1630074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f163006d430 con 0x7f1630109170 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f163006d2b0 con 0x7f1630102de0 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1636e66640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f163006d5b0 con 0x7f16301087f0 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.166+0000 7f1635403640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f1630074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1635403640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35106/0 (socket says 172.21.15.96:35106) 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1635403640 1 -- 172.21.15.96:0/917007228 learned_addr learned my addr 172.21.15.96:0/917007228 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f162ffff640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 -- 172.21.15.96:0/917007228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 msgr2=0x7f1630074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 -- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 msgr2=0x7f163006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 -- 172.21.15.96:0/917007228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163006f360 con 0x7f1630102de0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1635403640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1634c02640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f1630074760 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9f0 tx=0x7f162400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f162dffb640 1 -- 172.21.15.96:0/917007228 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400ccc0 con 0x7f1630102de0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163006f650 con 0x7f1630102de0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f162dffb640 1 -- 172.21.15.96:0/917007228 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16240108e0 con 0x7f1630102de0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.167+0000 7f162dffb640 1 -- 172.21.15.96:0/917007228 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624018880 con 0x7f1630102de0 2024-09-10T23:01:32.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.168+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301ae530 con 0x7f1630102de0 2024-09-10T23:01:32.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.170+0000 7f163923b740 1 Processor -- start 2024-09-10T23:01:32.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.170+0000 7f163923b740 1 -- start start 2024-09-10T23:01:32.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.170+0000 7f163923b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f163923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d34460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f163923b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f163923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f43c9eea0 con 0x556f43d28a10 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f163923b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f43c8e2d0 con 0x556f43d280f0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f163923b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f43c920e0 con 0x556f43d349a0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f162dffb640 1 -- 172.21.15.96:0/917007228 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1624010430 con 0x7f1630102de0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f162ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37856/0 (socket says 172.21.15.96:37856) 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 -- 172.21.15.96:0/3393690150 learned_addr learned my addr 172.21.15.96:0/3393690150 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1634c02640 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d34460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 -- 172.21.15.96:0/3393690150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 msgr2=0x556f43d36d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d36d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 -- 172.21.15.96:0/3393690150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 msgr2=0x556f43d34460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d34460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 -- 172.21.15.96:0/3393690150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f439ec650 con 0x556f43d280f0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1634c02640 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d34460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.171+0000 7f1635403640 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f162800e830 tx=0x7f162800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483ce0d888a918c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f161affd640 1 -- 172.21.15.96:0/3393690150 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628004d20 con 0x556f43d280f0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f161affd640 1 -- 172.21.15.96:0/3393690150 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1628005020 con 0x556f43d280f0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f161affd640 1 -- 172.21.15.96:0/3393690150 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801a5e0 con 0x556f43d280f0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f162dffb640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 0x7f16200b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f1634c02640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 0x7f16200b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f162dffb640 1 -- 172.21.15.96:0/917007228 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f16240e5e30 con 0x7f1630102de0 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 -- 172.21.15.96:0/3393690150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 msgr2=0x556f43d284d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f162800e830 tx=0x7f162800ecf0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f1634c02640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 0x7f16200b77e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16240ac6d0 tx=0x7f1624009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 -- 172.21.15.96:0/3393690150 shutdown_connections 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d36d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d34460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 --2- 172.21.15.96:0/3393690150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d284d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.172+0000 7f163923b740 1 -- 172.21.15.96:0/3393690150 >> 172.21.15.96:0/3393690150 conn(0x556f43d25f20 msgr2=0x556f43d270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- 172.21.15.96:0/3393690150 shutdown_connections 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- 172.21.15.96:0/3393690150 wait complete. 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- start start 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 Processor -- start 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f43aef700 con 0x556f43d28a10 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f43c29160 con 0x556f43d280f0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f43d3ba30 con 0x556f43d349a0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.173+0000 7f163923b740 10 client.0 _ll_register_callbacks cb 0x556f43c429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37864/0 (socket says 172.21.15.96:37864) 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 -- 172.21.15.96:0/2000646003 learned_addr learned my addr 172.21.15.96:0/2000646003 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f162ffff640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 -- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 msgr2=0x556f43d31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 -- 172.21.15.96:0/2000646003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 msgr2=0x556f43d316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 -- 172.21.15.96:0/2000646003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f43c56c90 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f1635403640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1628011fd0 tx=0x7f16280059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad81c5610f867034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f160ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628020040 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1628004a40 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1628005020 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1628004c40 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f163923b740 10 client.28145 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.174+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f43d472f0 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1628005020 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1628026070 con 0x556f43d280f0 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 10 client.28145 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 10 client.28145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 20 client.28145 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d48750 0x556f43d4ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 4 client.28145 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f43ca0df0 con 0x556f43d48750 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f162ffff640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d48750 0x556f43d4ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f162ffff640 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d48750 0x556f43d4ab40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47d8ad49254fc265 server_cookie=c01c2b57f255a356 in_seq=0 out_seq=0 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.175+0000 7f160ffff640 10 client.28145 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f160ffff640 1 -- 172.21.15.96:0/2000646003 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f43ca0df0 con 0x556f43d48750 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f160ffff640 10 client.28145 handle_command_reply: tid=0 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 client.28145 shutdown 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 msgr2=0x556f43d342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1628011fd0 tx=0x7f16280059e0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 shutdown_connections 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d48750 0x556f43d4ab40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 --2- 172.21.15.96:0/2000646003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d280f0 0x556f43d342a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f160ffff640 0 client.28145 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.176+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 >> 172.21.15.96:0/2000646003 conn(0x556f43d25f20 msgr2=0x556f43d2bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.177+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 shutdown_connections 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.177+0000 7f163923b740 1 -- 172.21.15.96:0/2000646003 wait complete. 2024-09-10T23:01:32.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.177+0000 7f163923b740 20 client.28145 trim_cache size 0 max 16384 2024-09-10T23:01:32.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 Processor -- start 2024-09-10T23:01:32.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 -- start start 2024-09-10T23:01:32.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d28a10 0x556f43d44c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b48c0 con 0x556f43d280f0 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b6400 con 0x556f43d28a10 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f163923b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b4f90 con 0x556f43d349a0 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f1635403640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f1635403640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35124/0 (socket says 172.21.15.96:35124) 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f1635403640 1 -- 172.21.15.96:0/775113994 learned_addr learned my addr 172.21.15.96:0/775113994 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.194+0000 7f162ffff640 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d28a10 0x556f43d44c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 -- 172.21.15.96:0/775113994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 msgr2=0x556f43d210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 -- 172.21.15.96:0/775113994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d28a10 msgr2=0x556f43d44c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d28a10 0x556f43d44c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 -- 172.21.15.96:0/775113994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f43d32070 con 0x556f43d280f0 2024-09-10T23:01:32.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f1635403640 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1628011810 tx=0x7f16280296b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d5e6b8bce6c76e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f161a7fc640 1 -- 172.21.15.96:0/775113994 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628038070 con 0x556f43d280f0 2024-09-10T23:01:32.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f161a7fc640 1 -- 172.21.15.96:0/775113994 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1628009820 con 0x556f43d280f0 2024-09-10T23:01:32.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.195+0000 7f161a7fc640 1 -- 172.21.15.96:0/775113994 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628003bd0 con 0x556f43d280f0 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- 172.21.15.96:0/775113994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 msgr2=0x556f43d27760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1628011810 tx=0x7f16280296b0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- 172.21.15.96:0/775113994 shutdown_connections 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d28a10 0x556f43d44c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d349a0 0x556f43d210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 --2- 172.21.15.96:0/775113994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d280f0 0x556f43d27760 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- 172.21.15.96:0/775113994 >> 172.21.15.96:0/775113994 conn(0x556f43d25f20 msgr2=0x556f43d27050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- 172.21.15.96:0/775113994 shutdown_connections 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- 172.21.15.96:0/775113994 wait complete. 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 -- start start 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.196+0000 7f163923b740 1 Processor -- start 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d280f0 0x556f43d2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b5d30 con 0x556f43d28a10 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b5660 con 0x556f43d349a0 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b28c0 con 0x556f43d280f0 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f163923b740 10 client.0 _ll_register_callbacks cb 0x7f1624005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f1635403640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:37884/0 (socket says 172.21.15.96:37884) 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f1635403640 1 -- 172.21.15.96:0/2571307241 learned_addr learned my addr 172.21.15.96:0/2571307241 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f1635403640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d280f0 0x556f43d2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 -- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d280f0 msgr2=0x556f43d2f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d280f0 0x556f43d2f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 -- 172.21.15.96:0/2571307241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 msgr2=0x556f43d41490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f162ffff640 1 -- 172.21.15.96:0/2571307241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f43d50270 con 0x556f43d28a10 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.197+0000 7f1635403640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f162ffff640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d40d00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f161c02aa00 tx=0x7f161c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768533ec6c14c9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f160ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c035040 con 0x556f43d28a10 2024-09-10T23:01:32.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f161c005450 con 0x556f43d28a10 2024-09-10T23:01:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1628009820 con 0x556f43d28a10 2024-09-10T23:01:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f161c004780 con 0x556f43d28a10 2024-09-10T23:01:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f163923b740 10 client.18525 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.198+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556f43d2dd30 con 0x556f43d28a10 2024-09-10T23:01:32.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.199+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1628009820 con 0x556f43d28a10 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.199+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f161c058070 con 0x556f43d28a10 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 10 client.18525 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 10 client.18525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 20 client.18525 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d2e440 0x556f43db84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 4 client.18525 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.200+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f43c9c8a0 con 0x556f43d2e440 2024-09-10T23:01:32.219 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.206+0000 7f1635403640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d2e440 0x556f43db84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.207+0000 7f1635403640 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d2e440 0x556f43db84a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=784e4f275925db91 server_cookie=854b339c0328135d in_seq=0 out_seq=0 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.209+0000 7f160ffff640 10 client.18525 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.211+0000 7f160ffff640 1 -- 172.21.15.96:0/2571307241 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f43c9c8a0 con 0x556f43d2e440 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.211+0000 7f160ffff640 10 client.18525 handle_command_reply: tid=0 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.211+0000 7f163923b740 1 client.18525 shutdown 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.211+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 msgr2=0x556f43d40d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.211+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d40d00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f161c02aa00 tx=0x7f161c004520 comp rx=0 tx=0).stop 2024-09-10T23:01:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 shutdown_connections 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x556f43d2e440 0x556f43db84a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f43d349a0 0x556f43d41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f43d28a10 0x556f43d40d00 secure :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f161c02aa00 tx=0x7f161c004520 comp rx=0 tx=0).stop 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 --2- 172.21.15.96:0/2571307241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f43d280f0 0x556f43d2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f160ffff640 0 client.18525 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.213+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 >> 172.21.15.96:0/2571307241 conn(0x556f43d25f20 msgr2=0x556f43d24750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.217+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 shutdown_connections 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.217+0000 7f163923b740 1 -- 172.21.15.96:0/2571307241 wait complete. 2024-09-10T23:01:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.217+0000 7f163923b740 20 client.18525 trim_cache size 0 max 16384 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.226+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 msgr2=0x7f16200b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.226+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 0x7f16200b77e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16240ac6d0 tx=0x7f1624009700 comp rx=0 tx=0).stop 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.226+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 msgr2=0x7f1630074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.226+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f1630074760 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9f0 tx=0x7f162400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 shutdown_connections 2024-09-10T23:01:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f16200b5320 0x7f16200b77e0 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16240ac6d0 tx=0x7f1624009700 comp rx=0 tx=0).stop 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1630109170 0x7f163006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16301087f0 0x7f1630074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 --2- 172.21.15.96:0/917007228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1630102de0 0x7f1630074760 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162400e9f0 tx=0x7f162400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.227+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 >> 172.21.15.96:0/917007228 conn(0x7f16300fe870 msgr2=0x7f1630100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.229+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 shutdown_connections 2024-09-10T23:01:32.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:32.229+0000 7f1636e66640 1 -- 172.21.15.96:0/917007228 wait complete. 2024-09-10T23:01:33.087 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:01:33.449 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:33.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:33 smithi096 ceph-mon[30869]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-10T23:01:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:33 smithi121 ceph-mon[35119]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-10T23:01:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:33 smithi117 ceph-mon[35081]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 MiB/s wr, 678 op/s 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 -- 172.21.15.96:0/1017572146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 msgr2=0x7fd83c116c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 --2- 172.21.15.96:0/1017572146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c116c60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd834009ca0 tx=0x7fd83402a950 comp rx=0 tx=0).stop 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 -- 172.21.15.96:0/1017572146 shutdown_connections 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 --2- 172.21.15.96:0/1017572146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10bb80 0x7fd83c1142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 --2- 172.21.15.96:0/1017572146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c116c60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 --2- 172.21.15.96:0/1017572146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10b260 0x7fd83c10b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.116+0000 7fd8435e3640 1 -- 172.21.15.96:0/1017572146 >> 172.21.15.96:0/1017572146 conn(0x7fd83c06e2e0 msgr2=0x7fd83c06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.119+0000 7fd8435e3640 1 -- 172.21.15.96:0/1017572146 shutdown_connections 2024-09-10T23:01:34.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.119+0000 7fd8435e3640 1 -- 172.21.15.96:0/1017572146 wait complete. 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.120+0000 7fd8435e3640 1 Processor -- start 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.120+0000 7fd8435e3640 1 -- start start 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.120+0000 7fd8435e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 0x7fd83c11aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.120+0000 7fd8435e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10bb80 0x7fd83c11b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.120+0000 7fd8435e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c1203c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.121+0000 7fd8435e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c119340 con 0x7fd83c114810 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.121+0000 7fd8435e3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1191c0 con 0x7fd83c10b260 2024-09-10T23:01:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.121+0000 7fd8435e3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1194c0 con 0x7fd83c10bb80 2024-09-10T23:01:34.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.123+0000 7fd841b80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c1203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.123+0000 7fd841b80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c1203c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35146/0 (socket says 172.21.15.96:35146) 2024-09-10T23:01:34.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.123+0000 7fd841b80640 1 -- 172.21.15.96:0/4130390272 learned_addr learned my addr 172.21.15.96:0/4130390272 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:34.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.125+0000 7fd840b7e640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10bb80 0x7fd83c11b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.127+0000 7fd84137f640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 0x7fd83c11aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.132+0000 7fd84137f640 1 -- 172.21.15.96:0/4130390272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10bb80 msgr2=0x7fd83c11b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.132+0000 7fd84137f640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10bb80 0x7fd83c11b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.132+0000 7fd84137f640 1 -- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 msgr2=0x7fd83c1203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.132+0000 7fd84137f640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c1203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.132+0000 7fd84137f640 1 -- 172.21.15.96:0/4130390272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c120b40 con 0x7fd83c10b260 2024-09-10T23:01:34.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.142+0000 7fd84137f640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 0x7fd83c11aae0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.149+0000 7fd8327fc640 1 -- 172.21.15.96:0/4130390272 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800ccc0 con 0x7fd83c10b260 2024-09-10T23:01:34.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.149+0000 7fd8327fc640 1 -- 172.21.15.96:0/4130390272 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd838005020 con 0x7fd83c10b260 2024-09-10T23:01:34.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.149+0000 7fd8327fc640 1 -- 172.21.15.96:0/4130390272 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004a00 con 0x7fd83c10b260 2024-09-10T23:01:34.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.150+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c120e30 con 0x7fd83c10b260 2024-09-10T23:01:34.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.150+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c125550 con 0x7fd83c10b260 2024-09-10T23:01:34.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.154+0000 7fd8459b8740 1 Processor -- start 2024-09-10T23:01:34.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.155+0000 7fd8327fc640 1 -- 172.21.15.96:0/4130390272 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fd838004ba0 con 0x7fd83c10b260 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.155+0000 7fd8459b8740 1 -- start start 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b45eea0 con 0x55dc2b4e84d0 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b44e2d0 con 0x55dc2b4f4d80 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd8459b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b4520e0 con 0x55dc2b4e8df0 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd84137f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd84137f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38624/0 (socket says 172.21.15.96:38624) 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd84137f640 1 -- 172.21.15.96:0/3940831715 learned_addr learned my addr 172.21.15.96:0/3940831715 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:34.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.156+0000 7fd840b7e640 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 -- 172.21.15.96:0/3940831715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 msgr2=0x55dc2b4f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 -- 172.21.15.96:0/3940831715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 msgr2=0x55dc2b4e88b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:34.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e88b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 -- 172.21.15.96:0/3940831715 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2b1ac650 con 0x55dc2b4e8df0 2024-09-10T23:01:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.157+0000 7fd84137f640 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8380ebc20 tx=0x7fd8380ebd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21efccef85344aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.158+0000 7fd8327fc640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 0x7fd8280b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.158+0000 7fd8327fc640 1 -- 172.21.15.96:0/4130390272 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd8380e5b10 con 0x7fd83c10b260 2024-09-10T23:01:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.159+0000 7fd8277fe640 1 -- 172.21.15.96:0/3940831715 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8380f4070 con 0x55dc2b4e8df0 2024-09-10T23:01:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.159+0000 7fd8277fe640 1 -- 172.21.15.96:0/3940831715 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8380edc90 con 0x55dc2b4e8df0 2024-09-10T23:01:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 -- 172.21.15.96:0/3940831715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 msgr2=0x55dc2b4f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8380ebc20 tx=0x7fd8380ebd10 comp rx=0 tx=0).stop 2024-09-10T23:01:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd841b80640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 0x7fd8280b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 -- 172.21.15.96:0/3940831715 shutdown_connections 2024-09-10T23:01:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f4840 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 --2- 172.21.15.96:0/3940831715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.160+0000 7fd8459b8740 1 -- 172.21.15.96:0/3940831715 >> 172.21.15.96:0/3940831715 conn(0x55dc2b4e6300 msgr2=0x55dc2b4e74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.161+0000 7fd841b80640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 0x7fd8280b79b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd834005e30 tx=0x7fd83402ae50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.161+0000 7fd8459b8740 1 -- 172.21.15.96:0/3940831715 shutdown_connections 2024-09-10T23:01:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.161+0000 7fd8459b8740 1 -- 172.21.15.96:0/3940831715 wait complete. 2024-09-10T23:01:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 -- start start 2024-09-10T23:01:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 Processor -- start 2024-09-10T23:01:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b2af700 con 0x55dc2b4e84d0 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.162+0000 7fd8459b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b3e9160 con 0x55dc2b4e8df0 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd8459b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dc2b4f7a50 con 0x55dc2b4f4d80 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38632/0 (socket says 172.21.15.96:38632) 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 -- 172.21.15.96:0/896191775 learned_addr learned my addr 172.21.15.96:0/896191775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd840b7e640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd8459b8740 10 client.0 _ll_register_callbacks cb 0x55dc2b4029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:34.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd84137f640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 -- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e8df0 msgr2=0x55dc2b4f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 -- 172.21.15.96:0/896191775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 msgr2=0x55dc2b4f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 -- 172.21.15.96:0/896191775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2b416c90 con 0x55dc2b4f4d80 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd841b80640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd834003a90 tx=0x7fd8340095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c57439cf8d499c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.163+0000 7fd824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.164+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834034040 con 0x55dc2b4f4d80 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.164+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd83402fc40 con 0x55dc2b4f4d80 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.164+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8380edc90 con 0x55dc2b4f4d80 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.164+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd83402fe40 con 0x55dc2b4f4d80 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 10 client.28222 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc2b503360 con 0x55dc2b4f4d80 2024-09-10T23:01:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83406e070 con 0x55dc2b4f4d80 2024-09-10T23:01:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd834073840 con 0x55dc2b4f4d80 2024-09-10T23:01:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 10 client.28222 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 10 client.28222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 20 client.28222 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b504870 0x55dc2b506cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 4 client.28222 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.165+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd8380edc90 con 0x55dc2b504870 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.166+0000 7fd84137f640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b504870 0x55dc2b506cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.166+0000 7fd824ff9640 10 client.28222 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.167+0000 7fd84137f640 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b504870 0x55dc2b506cd0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680e7ea67f01829c server_cookie=e49ab05cbcfa0cdd in_seq=0 out_seq=0 2024-09-10T23:01:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.168+0000 7fd824ff9640 1 -- 172.21.15.96:0/896191775 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fd8380edc90 con 0x55dc2b504870 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.168+0000 7fd824ff9640 10 client.28222 handle_command_reply: tid=0 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.168+0000 7fd8459b8740 1 client.28222 shutdown 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 msgr2=0x55dc2b4f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd834003a90 tx=0x7fd8340095c0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 shutdown_connections 2024-09-10T23:01:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b504870 0x55dc2b506cd0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f1fc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 --2- 172.21.15.96:0/896191775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 >> 172.21.15.96:0/896191775 conn(0x55dc2b4e6300 msgr2=0x55dc2b4ebea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd824ff9640 0 client.28222 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 shutdown_connections 2024-09-10T23:01:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.169+0000 7fd8459b8740 1 -- 172.21.15.96:0/896191775 wait complete. 2024-09-10T23:01:34.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.170+0000 7fd8459b8740 20 client.28222 trim_cache size 0 max 16384 2024-09-10T23:01:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 Processor -- start 2024-09-10T23:01:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 -- start start 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b4770 con 0x55dc2b4e8df0 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b62b0 con 0x55dc2b4e84d0 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.200+0000 7fd8459b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b4e40 con 0x55dc2b4f4d80 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd840b7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:38642/0 (socket says 172.21.15.96:38642) 2024-09-10T23:01:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 -- 172.21.15.96:0/781215456 learned_addr learned my addr 172.21.15.96:0/781215456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd84137f640 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 -- 172.21.15.96:0/781215456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e84d0 msgr2=0x55dc2b4e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 -- 172.21.15.96:0/781215456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 msgr2=0x55dc2b4f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 -- 172.21.15.96:0/781215456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2b4e6fe0 con 0x55dc2b4f4d80 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.201+0000 7fd841b80640 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd83402af40 tx=0x7fd83402fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3388ba57dd1777d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.202+0000 7fd826ffd640 1 -- 172.21.15.96:0/781215456 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834034030 con 0x55dc2b4f4d80 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.202+0000 7fd826ffd640 1 -- 172.21.15.96:0/781215456 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83406e070 con 0x55dc2b4f4d80 2024-09-10T23:01:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.202+0000 7fd826ffd640 1 -- 172.21.15.96:0/781215456 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340041b0 con 0x55dc2b4f4d80 2024-09-10T23:01:34.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- 172.21.15.96:0/781215456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 msgr2=0x55dc2b4f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd83402af40 tx=0x7fd83402fdd0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- 172.21.15.96:0/781215456 shutdown_connections 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f6170 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 --2- 172.21.15.96:0/781215456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- 172.21.15.96:0/781215456 >> 172.21.15.96:0/781215456 conn(0x55dc2b4e6300 msgr2=0x55dc2b4f3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- 172.21.15.96:0/781215456 shutdown_connections 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- 172.21.15.96:0/781215456 wait complete. 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 -- start start 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.203+0000 7fd8459b8740 1 Processor -- start 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd8459b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd840b7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd84137f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd840b7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35174/0 (socket says 172.21.15.96:35174) 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd840b7e640 1 -- 172.21.15.96:0/1147125429 learned_addr learned my addr 172.21.15.96:0/1147125429 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.204+0000 7fd841b80640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b5be0 con 0x55dc2b4e8df0 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b5510 con 0x55dc2b4f4d80 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c1b2750 con 0x55dc2b4e84d0 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd8459b8740 10 client.0 _ll_register_callbacks cb 0x7fd838005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd84137f640 1 -- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 msgr2=0x55dc2b4f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd84137f640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd84137f640 1 -- 172.21.15.96:0/1147125429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 msgr2=0x55dc2b4fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd84137f640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd84137f640 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2b4f8550 con 0x55dc2b4e84d0 2024-09-10T23:01:34.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.205+0000 7fd840b7e640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:34.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd84137f640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4fcf50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c6a0 tx=0x7fd83800e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa68bd56db30934 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83801c040 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fd8380d30f0 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83406e070 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd8380f4040 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd8459b8740 10 client.28228 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc2b572240 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8380c1e50 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fd8380a8540 con 0x55dc2b4e84d0 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.206+0000 7fd8459b8740 10 client.28228 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd8459b8740 10 client.28228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd8459b8740 20 client.28228 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b572580 0x55dc2b5749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd8459b8740 4 client.28228 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc2b460df0 con 0x55dc2b572580 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd840b7e640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b572580 0x55dc2b5749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.207+0000 7fd840b7e640 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b572580 0x55dc2b5749a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b610189ededc07 server_cookie=f86b6930c5e44c1e in_seq=0 out_seq=0 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd824ff9640 10 client.28228 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd824ff9640 1 -- 172.21.15.96:0/1147125429 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc2b460df0 con 0x55dc2b572580 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd824ff9640 10 client.28228 handle_command_reply: tid=0 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 client.28228 shutdown 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 msgr2=0x55dc2b4fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4fcf50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c6a0 tx=0x7fd83800e940 comp rx=0 tx=0).stop 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 shutdown_connections 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55dc2b572580 0x55dc2b5749a0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd824ff9640 0 client.28228 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc2b4f4d80 0x55dc2b4f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dc2b4e8df0 0x55dc2b4fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 --2- 172.21.15.96:0/1147125429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc2b4e84d0 0x55dc2b4fcf50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.208+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 >> 172.21.15.96:0/1147125429 conn(0x55dc2b4e6300 msgr2=0x55dc2b4f3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 shutdown_connections 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8459b8740 1 -- 172.21.15.96:0/1147125429 wait complete. 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8459b8740 20 client.28228 trim_cache size 0 max 16384 2024-09-10T23:01:34.213 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:01:34.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 msgr2=0x7fd8280b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 0x7fd8280b79b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd834005e30 tx=0x7fd83402ae50 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 msgr2=0x7fd83c11aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 0x7fd83c11aae0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.209+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 shutdown_connections 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fd8280b54f0 0x7fd8280b79b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd83c114810 0x7fd83c1203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd83c10bb80 0x7fd83c11b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 --2- 172.21.15.96:0/4130390272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd83c10b260 0x7fd83c11aae0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 >> 172.21.15.96:0/4130390272 conn(0x7fd83c06e2e0 msgr2=0x7fd83c072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 shutdown_connections 2024-09-10T23:01:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:34.210+0000 7fd8435e3640 1 -- 172.21.15.96:0/4130390272 wait complete. 2024-09-10T23:01:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:35 smithi096 ceph-mon[30869]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-09-10T23:01:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:35 smithi121 ceph-mon[35119]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-09-10T23:01:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:35 smithi117 ceph-mon[35081]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-09-10T23:01:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 840.0 s, 1457.8 tps, lat 68.974 ms stddev 72.421 2024-09-10T23:01:35.953 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:01:35.953 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:01:36.318 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 -- 172.21.15.96:0/3883926480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 msgr2=0x7fb2ec108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 --2- 172.21.15.96:0/3883926480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec108bf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009860 tx=0x7fb2d402a340 comp rx=0 tx=0).stop 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 -- 172.21.15.96:0/3883926480 shutdown_connections 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 --2- 172.21.15.96:0/3883926480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec108bf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 --2- 172.21.15.96:0/3883926480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 --2- 172.21.15.96:0/3883926480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 -- 172.21.15.96:0/3883926480 >> 172.21.15.96:0/3883926480 conn(0x7fb2ec0fe850 msgr2=0x7fb2ec100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.938+0000 7fb2f2406640 1 -- 172.21.15.96:0/3883926480 shutdown_connections 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 -- 172.21.15.96:0/3883926480 wait complete. 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 Processor -- start 2024-09-10T23:01:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 -- start start 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec112d80 con 0x7fb2ec109130 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec112c00 con 0x7fb2ec1087b0 2024-09-10T23:01:36.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.939+0000 7fb2f2406640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec112f00 con 0x7fb2ec102da0 2024-09-10T23:01:36.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47796/0 (socket says 172.21.15.96:47796) 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53312/0 (socket says 172.21.15.96:53312) 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2f09a3640 1 -- 172.21.15.96:0/2127217269 learned_addr learned my addr 172.21.15.96:0/2127217269 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 -- 172.21.15.96:0/2127217269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 msgr2=0x7fb2ec1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 -- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 msgr2=0x7fb2ec19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2e3fff640 1 -- 172.21.15.96:0/2127217269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec19f3a0 con 0x7fb2ec1087b0 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.940+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec199830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4035040 tx=0x7fb2d4032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2e9ffb640 1 -- 172.21.15.96:0/2127217269 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4047070 con 0x7fb2ec1087b0 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec19f630 con 0x7fb2ec1087b0 2024-09-10T23:01:36.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2e9ffb640 1 -- 172.21.15.96:0/2127217269 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4038040 con 0x7fb2ec1087b0 2024-09-10T23:01:36.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec075380 con 0x7fb2ec1087b0 2024-09-10T23:01:36.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.941+0000 7fb2e9ffb640 1 -- 172.21.15.96:0/2127217269 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4037050 con 0x7fb2ec1087b0 2024-09-10T23:01:36.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.943+0000 7fb2f47db740 1 Processor -- start 2024-09-10T23:01:36.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.943+0000 7fb2f47db740 1 -- start start 2024-09-10T23:01:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2e9ffb640 1 -- 172.21.15.96:0/2127217269 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fb2d4030450 con 0x7fb2ec1087b0 2024-09-10T23:01:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a2562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a264b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56215a1cce60 con 0x56215a262790 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56215a1bc290 con 0x56215a255ee0 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2f47db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56215a1c00a0 con 0x56215a256800 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53328/0 (socket says 172.21.15.96:53328) 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2ebfff640 1 -- 172.21.15.96:0/2553496760 learned_addr learned my addr 172.21.15.96:0/2553496760 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.944+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a264b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a2562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2ebfff640 1 -- 172.21.15.96:0/2553496760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 msgr2=0x56215a2562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a2562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2ebfff640 1 -- 172.21.15.96:0/2553496760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 msgr2=0x56215a264b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a264b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2ebfff640 1 -- 172.21.15.96:0/2553496760 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562159f1a930 con 0x56215a256800 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.946+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a264b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc010500 tx=0x7fb2dc010900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1c41087f73de1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2e27fc640 1 -- 172.21.15.96:0/2553496760 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc011760 con 0x56215a256800 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2e27fc640 1 -- 172.21.15.96:0/2553496760 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc011db0 con 0x56215a256800 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2e27fc640 1 -- 172.21.15.96:0/2553496760 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc019dd0 con 0x56215a256800 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2e9ffb640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 0x7fb2d80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 0x7fb2d80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.947+0000 7fb2e9ffb640 1 -- 172.21.15.96:0/2127217269 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb2d410dff0 con 0x7fb2ec1087b0 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- 172.21.15.96:0/2553496760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 msgr2=0x56215a262250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc010500 tx=0x7fb2dc010900 comp rx=0 tx=0).stop 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 0x7fb2d80b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40049d0 tx=0x7fb2e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- 172.21.15.96:0/2553496760 shutdown_connections 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a264b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a262250 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 --2- 172.21.15.96:0/2553496760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a2562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- 172.21.15.96:0/2553496760 >> 172.21.15.96:0/2553496760 conn(0x56215a253d10 msgr2=0x56215a254eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- 172.21.15.96:0/2553496760 shutdown_connections 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- 172.21.15.96:0/2553496760 wait complete. 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 -- start start 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.948+0000 7fb2f47db740 1 Processor -- start 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a25fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56215a01d890 con 0x56215a262790 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56215a156d40 con 0x56215a256800 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56215a265440 con 0x56215a255ee0 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f47db740 10 client.0 _ll_register_callbacks cb 0x56215a1bac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a25fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47804/0 (socket says 172.21.15.96:47804) 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53344/0 (socket says 172.21.15.96:53344) 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3334103207 learned_addr learned my addr 172.21.15.96:0/3334103207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3334103207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 msgr2=0x56215a25f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 msgr2=0x56215a25fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a25fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3334103207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215a26e870 con 0x56215a262790 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a25fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400fb00 tx=0x7fb2e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3428686dc11f3299 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.949+0000 7fb2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e402e060 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb2e4018030 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc011db0 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2e4004b20 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2f47db740 10 client.18552 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56215a271280 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.950+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc011db0 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb2e40050b0 con 0x56215a262790 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 10 client.18552 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 20 client.18552 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a272790 0x56215a274ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 4 client.18552 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56215a1cedb0 con 0x56215a272790 2024-09-10T23:01:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a272790 0x56215a274ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.951+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a272790 0x56215a274ba0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3b24adc2de124b server_cookie=df67551bf46f63bd in_seq=0 out_seq=0 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2cb7fe640 10 client.18552 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/3334103207 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56215a1cedb0 con 0x56215a272790 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2cb7fe640 10 client.18552 handle_command_reply: tid=0 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2f47db740 1 client.18552 shutdown 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 msgr2=0x56215a260200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400fb00 tx=0x7fb2e4004ab0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 shutdown_connections 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.952+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a272790 0x56215a274ba0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a25fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a260200 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2cb7fe640 0 client.18552 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3334103207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a25f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 >> 172.21.15.96:0/3334103207 conn(0x56215a253d10 msgr2=0x56215a252f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 shutdown_connections 2024-09-10T23:01:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 1 -- 172.21.15.96:0/3334103207 wait complete. 2024-09-10T23:01:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.953+0000 7fb2f47db740 20 client.18552 trim_cache size 0 max 16384 2024-09-10T23:01:36.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.973+0000 7fb2f47db740 1 Processor -- start 2024-09-10T23:01:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 -- start start 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a255220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a2614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1ac070 con 0x56215a262790 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1adbb0 con 0x56215a256800 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f47db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1ac740 con 0x56215a255ee0 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47806/0 (socket says 172.21.15.96:47806) 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3056076745 learned_addr learned my addr 172.21.15.96:0/3056076745 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a2614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.975+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a255220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3056076745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 msgr2=0x56215a255220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a255220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3056076745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 msgr2=0x56215a2614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a2614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 -- 172.21.15.96:0/3056076745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215a254790 con 0x56215a262790 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009f80 tx=0x7fb2e400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4476ffa7af39e9be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2e1ffb640 1 -- 172.21.15.96:0/3056076745 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4038070 con 0x56215a262790 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2e1ffb640 1 -- 172.21.15.96:0/3056076745 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e4005170 con 0x56215a262790 2024-09-10T23:01:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.976+0000 7fb2e1ffb640 1 -- 172.21.15.96:0/3056076745 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e401e7e0 con 0x56215a262790 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- 172.21.15.96:0/3056076745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 msgr2=0x56215a25c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009f80 tx=0x7fb2e400ff00 comp rx=0 tx=0).stop 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- 172.21.15.96:0/3056076745 shutdown_connections 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a256800 0x56215a2614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a25c990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 --2- 172.21.15.96:0/3056076745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a255ee0 0x56215a255220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- 172.21.15.96:0/3056076745 >> 172.21.15.96:0/3056076745 conn(0x56215a253d10 msgr2=0x56215a260db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- 172.21.15.96:0/3056076745 shutdown_connections 2024-09-10T23:01:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- 172.21.15.96:0/3056076745 wait complete. 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 -- start start 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.977+0000 7fb2f47db740 1 Processor -- start 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a266b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a267040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1ad4e0 con 0x56215a262790 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1ace10 con 0x56215a255ee0 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1aa050 con 0x56215a256800 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f47db740 10 client.0 _ll_register_callbacks cb 0x7fb2e4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47808/0 (socket says 172.21.15.96:47808) 2024-09-10T23:01:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 -- 172.21.15.96:0/53346654 learned_addr learned my addr 172.21.15.96:0/53346654 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a266b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2e3fff640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a267040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 -- 172.21.15.96:0/53346654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 msgr2=0x56215a267040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a267040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 -- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 msgr2=0x56215a266b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a266b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 -- 172.21.15.96:0/53346654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215a26e6a0 con 0x56215a262790 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a266b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2f09a3640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4019040 tx=0x7fb2e40059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a4c5eb5aec7a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.978+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4038070 con 0x56215a262790 2024-09-10T23:01:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fb2e4004ed0 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e4005170 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2e402e020 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 10 client.18558 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56215a2dfcd0 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e4005170 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb2e403d4f0 con 0x56215a262790 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 10 client.18558 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 10 client.18558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 20 client.18558 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a2e0330 0x56215a2e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 4 client.18558 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56215a1ca860 con 0x56215a2e0330 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.979+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a2e0330 0x56215a2e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:36.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.980+0000 7fb2cb7fe640 10 client.18558 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:36.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.980+0000 7fb2ebfff640 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a2e0330 0x56215a2e2750 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a38020d7d3e3f6b server_cookie=6f2e6838e2c105c4 in_seq=0 out_seq=0 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2cb7fe640 1 -- 172.21.15.96:0/53346654 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56215a1ca860 con 0x56215a2e0330 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2cb7fe640 10 client.18558 handle_command_reply: tid=0 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 client.18558 shutdown 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 msgr2=0x56215a2de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4019040 tx=0x7fb2e40059e0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 shutdown_connections 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x56215a2e0330 0x56215a2e2750 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56215a262790 0x56215a2de670 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56215a256800 0x56215a267040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 --2- 172.21.15.96:0/53346654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56215a255ee0 0x56215a266b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2cb7fe640 0 client.18558 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 >> 172.21.15.96:0/53346654 conn(0x56215a253d10 msgr2=0x56215a252750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 shutdown_connections 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 1 -- 172.21.15.96:0/53346654 wait complete. 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.982+0000 7fb2f47db740 20 client.18558 trim_cache size 0 max 16384 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "2df1ad6c-0476-4e75-81a3-b22370ae9b18", 2024-09-10T23:01:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 msgr2=0x7fb2d80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 0x7fb2d80b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40049d0 tx=0x7fb2e4009500 comp rx=0 tx=0).stop 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 msgr2=0x7fb2ec199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec199830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4035040 tx=0x7fb2d4032dc0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 shutdown_connections 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2ec1087b0 0x7fb2ec199830 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fb2d80b5360 0x7fb2d80b7820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2ec109130 0x7fb2ec19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 --2- 172.21.15.96:0/2127217269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2ec102da0 0x7fb2ec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:36.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 >> 172.21.15.96:0/2127217269 conn(0x7fb2ec0fe850 msgr2=0x7fb2ec100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:36.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 shutdown_connections 2024-09-10T23:01:36.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:36.983+0000 7fb2f2406640 1 -- 172.21.15.96:0/2127217269 wait complete. 2024-09-10T23:01:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:37 smithi096 ceph-mon[30869]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 776 op/s 2024-09-10T23:01:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:37 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:01:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:37 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:37 smithi121 ceph-mon[35119]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 776 op/s 2024-09-10T23:01:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:37 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:01:37.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:37 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:37 smithi117 ceph-mon[35081]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 776 op/s 2024-09-10T23:01:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:37 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:01:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:37 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:37.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:01:38.330 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:01:38.668 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:38 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T23:01:38.668 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:38 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:38.668 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:38 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:01:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:38 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T23:01:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:38 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:38.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:38 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:01:38.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:38 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T23:01:38.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:38 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:38.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:38 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:01:38.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- 172.21.15.96:0/2871574043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4100df0 msgr2=0x7fc0e41011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 --2- 172.21.15.96:0/2871574043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4100df0 0x7fc0e41011d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e002a410 comp rx=0 tx=0).stop 2024-09-10T23:01:38.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- 172.21.15.96:0/2871574043 shutdown_connections 2024-09-10T23:01:38.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 --2- 172.21.15.96:0/2871574043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0e4109200 0x7fc0e410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 --2- 172.21.15.96:0/2871574043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4108860 0x7fc0e4108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 --2- 172.21.15.96:0/2871574043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4100df0 0x7fc0e41011d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- 172.21.15.96:0/2871574043 >> 172.21.15.96:0/2871574043 conn(0x7fc0e40fca20 msgr2=0x7fc0e40fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- 172.21.15.96:0/2871574043 shutdown_connections 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- 172.21.15.96:0/2871574043 wait complete. 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 Processor -- start 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.901+0000 7fc0eba02640 1 -- start start 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0e4100df0 0x7fc0e41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 0x7fc0e419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4112d80 con 0x7fc0e4108860 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4112c00 con 0x7fc0e4100df0 2024-09-10T23:01:38.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0eba02640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4112f00 con 0x7fc0e4109200 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47830/0 (socket says 172.21.15.96:47830) 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/1361138200 learned_addr learned my addr 172.21.15.96:0/1361138200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 0x7fc0e419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/1361138200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 msgr2=0x7fc0e419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 0x7fc0e419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/1361138200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0e4100df0 msgr2=0x7fc0e41993b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0e4100df0 0x7fc0e41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/1361138200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e419f460 con 0x7fc0e4108860 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 0x7fc0e419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.902+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.903+0000 7fc0da7fc640 1 -- 172.21.15.96:0/1361138200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400cd80 con 0x7fc0e4108860 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.903+0000 7fc0da7fc640 1 -- 172.21.15.96:0/1361138200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40054d0 con 0x7fc0e4108860 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.903+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e419f750 con 0x7fc0e4108860 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.903+0000 7fc0da7fc640 1 -- 172.21.15.96:0/1361138200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010690 con 0x7fc0e4108860 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.903+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e41a5e10 con 0x7fc0e4108860 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.906+0000 7fc0eddd7740 1 Processor -- start 2024-09-10T23:01:38.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.906+0000 7fc0da7fc640 1 -- 172.21.15.96:0/1361138200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fc0d4004e90 con 0x7fc0e4108860 2024-09-10T23:01:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 -- start start 2024-09-10T23:01:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2642c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653a25abe50 con 0x5653a26431a0 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5653a259b280 con 0x5653a2642820 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.907+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5653a259f090 con 0x5653a2632f00 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53380/0 (socket says 172.21.15.96:53380) 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e979e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2642c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 -- 172.21.15.96:0/3427636842 learned_addr learned my addr 172.21.15.96:0/3427636842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0da7fc640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 0x7fc0d00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 -- 172.21.15.96:0/3427636842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 msgr2=0x5653a2642c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2642c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 -- 172.21.15.96:0/3427636842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 msgr2=0x5653a264cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 -- 172.21.15.96:0/3427636842 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653a22f9650 con 0x5653a2632f00 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 0x7fc0d00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0da7fc640 1 -- 172.21.15.96:0/1361138200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0d40e6090 con 0x7fc0e4108860 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.908+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc011790 tx=0x7fc0dc011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50a8c111abcecfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0cf7fe640 1 -- 172.21.15.96:0/3427636842 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00f9a0 con 0x5653a2632f00 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0cf7fe640 1 -- 172.21.15.96:0/3427636842 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc00fc80 con 0x5653a2632f00 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0cf7fe640 1 -- 172.21.15.96:0/3427636842 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc01cab0 con 0x5653a2632f00 2024-09-10T23:01:38.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3427636842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 msgr2=0x5653a26332e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 0x7fc0d00b7820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc020000 tx=0x7fc0dc021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.909+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc011790 tx=0x7fc0dc011c50 comp rx=0 tx=0).stop 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3427636842 shutdown_connections 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2642c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3427636842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a26332e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3427636842 >> 172.21.15.96:0/3427636842 conn(0x5653a2635420 msgr2=0x5653a2635800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3427636842 shutdown_connections 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3427636842 wait complete. 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- start start 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 Processor -- start 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a264c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2649e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653a23fc700 con 0x5653a26431a0 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5653a2536110 con 0x5653a2642820 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.910+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5653a264d280 con 0x5653a2632f00 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0eddd7740 10 client.0 _ll_register_callbacks cb 0x5653a254f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:38.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a264c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e9f9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e9f9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47846/0 (socket says 172.21.15.96:47846) 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e9f9f640 1 -- 172.21.15.96:0/3081942472 learned_addr learned my addr 172.21.15.96:0/3081942472 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2649e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 -- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 msgr2=0x5653a2649e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2649e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 -- 172.21.15.96:0/3081942472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 msgr2=0x5653a264a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 -- 172.21.15.96:0/3081942472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653a2563c40 con 0x5653a2632f00 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0e979e640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a264c960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005c20 tx=0x7fc0e0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ab781dd80a8720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.911+0000 7fc0ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:38.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0040070 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0dc00fc80 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0e00318b0 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0035040 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 10 client.28258 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5653a2658bc0 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc00fc80 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc0e00046c0 con 0x5653a2632f00 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 10 client.28258 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 10 client.28258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 20 client.28258 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a265a3b0 0x5653a265c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 4 client.28258 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.912+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653a25adda0 con 0x5653a265a3b0 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.913+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a265a3b0 0x5653a265c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.913+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a265a3b0 0x5653a265c810 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd1bb9c71a3be2a server_cookie=52e92d15c97aa601 in_seq=0 out_seq=0 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.913+0000 7fc0ccff9640 10 client.28258 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0ccff9640 1 -- 172.21.15.96:0/3081942472 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653a25adda0 con 0x5653a265a3b0 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0ccff9640 10 client.28258 handle_command_reply: tid=0 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 client.28258 shutdown 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 msgr2=0x5653a264c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a264c960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005c20 tx=0x7fc0e0032d50 comp rx=0 tx=0).stop 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 shutdown_connections 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a265a3b0 0x5653a265c810 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2649e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a26431a0 0x5653a264a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0ccff9640 0 client.28258 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3081942472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2632f00 0x5653a264c960 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 >> 172.21.15.96:0/3081942472 conn(0x5653a2635420 msgr2=0x5653a2645150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 shutdown_connections 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3081942472 wait complete. 2024-09-10T23:01:38.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.914+0000 7fc0eddd7740 20 client.28258 trim_cache size 0 max 16384 2024-09-10T23:01:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.936+0000 7fc0eddd7740 1 Processor -- start 2024-09-10T23:01:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.936+0000 7fc0eddd7740 1 -- start start 2024-09-10T23:01:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.936+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a2645100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.936+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.936+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a26431a0 0x5653a2631bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41ac190 con 0x5653a2632f00 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41adcd0 con 0x5653a26431a0 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41ac860 con 0x5653a2642820 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e979e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a2645100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53404/0 (socket says 172.21.15.96:53404) 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/3263522057 learned_addr learned my addr 172.21.15.96:0/3263522057 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a26431a0 0x5653a2631bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/3263522057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a26431a0 msgr2=0x5653a2631bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a26431a0 0x5653a2631bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/3263522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 msgr2=0x5653a2645100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a2645100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e8f9d640 1 -- 172.21.15.96:0/3263522057 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653a2649bb0 con 0x5653a2642820 2024-09-10T23:01:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.937+0000 7fc0e979e640 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a2645100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.938+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40ac550 tx=0x7fc0d40a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6def299ca96ed5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.938+0000 7fc0ceffd640 1 -- 172.21.15.96:0/3263522057 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40abfc0 con 0x5653a2642820 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.938+0000 7fc0ceffd640 1 -- 172.21.15.96:0/3263522057 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40ac360 con 0x5653a2642820 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.938+0000 7fc0ceffd640 1 -- 172.21.15.96:0/3263522057 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40b5de0 con 0x5653a2642820 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3263522057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 msgr2=0x5653a2645d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40ac550 tx=0x7fc0d40a8040 comp rx=0 tx=0).stop 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3263522057 shutdown_connections 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a26431a0 0x5653a2631bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a2642820 0x5653a2645d10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/3263522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a2645100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3263522057 >> 172.21.15.96:0/3263522057 conn(0x5653a2635420 msgr2=0x5653a26489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3263522057 shutdown_connections 2024-09-10T23:01:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- 172.21.15.96:0/3263522057 wait complete. 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 -- start start 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.939+0000 7fc0eddd7740 1 Processor -- start 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2652d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a26431a0 0x5653a264f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41ad600 con 0x5653a2632f00 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41acf30 con 0x5653a2642820 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41aa170 con 0x5653a26431a0 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0eddd7740 10 client.0 _ll_register_callbacks cb 0x7fc0d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0e979e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0e979e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47874/0 (socket says 172.21.15.96:47874) 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0e979e640 1 -- 172.21.15.96:0/221763393 learned_addr learned my addr 172.21.15.96:0/221763393 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2652d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.940+0000 7fc0e9f9f640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a26431a0 0x5653a264f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 -- 172.21.15.96:0/221763393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a26431a0 msgr2=0x5653a264f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a26431a0 0x5653a264f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 -- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 msgr2=0x5653a2652d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2652d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 -- 172.21.15.96:0/221763393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653a264fd30 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2652d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0e979e640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e0004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2839493ef10b15c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0040040 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fc0e0005490 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d40ac360 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0060050 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0eddd7740 10 client.18576 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.941+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5653a26c7e60 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40ac360 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc0e00322e0 con 0x5653a2632f00 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 10 client.18576 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:38.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 20 client.18576 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a26c85b0 0x5653a26ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 4 client.18576 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653a25a9850 con 0x5653a26c85b0 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.942+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a26c85b0 0x5653a26ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0ccff9640 10 client.18576 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0e8f9d640 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a26c85b0 0x5653a26ca9d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873337a04a474a79 server_cookie=acc134794db61604 in_seq=0 out_seq=0 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0ccff9640 1 -- 172.21.15.96:0/221763393 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x5653a25a9850 con 0x5653a26c85b0 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0ccff9640 10 client.18576 handle_command_reply: tid=0 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 client.18576 shutdown 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 msgr2=0x5653a26527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0000c00 tx=0x7fc0e0004f30 comp rx=0 tx=0).stop 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 shutdown_connections 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x5653a26c85b0 0x5653a26ca9d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653a2642820 0x5653a2652d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0ccff9640 0 client.18576 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653a26431a0 0x5653a264f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 --2- 172.21.15.96:0/221763393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653a2632f00 0x5653a26527c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.943+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 >> 172.21.15.96:0/221763393 conn(0x5653a2635420 msgr2=0x5653a2645d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.945+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 shutdown_connections 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.945+0000 7fc0eddd7740 1 -- 172.21.15.96:0/221763393 wait complete. 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.945+0000 7fc0eddd7740 20 client.18576 trim_cache size 0 max 16384 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "2df1ad6c-0476-4e75-81a3-b22370ae9b18": { 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "2df1ad6c-0476-4e75-81a3-b22370ae9b18", 2024-09-10T23:01:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "2df1ad6c-0476-4e75-81a3-b22370ae9b18": { 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:01:36.980195+0000", 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:01:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:01:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:01:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 msgr2=0x7fc0d00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 0x7fc0d00b7820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc020000 tx=0x7fc0dc021000 comp rx=0 tx=0).stop 2024-09-10T23:01:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 msgr2=0x7fc0e41998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400edf0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 shutdown_connections 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fc0d00b5360 0x7fc0d00b7820 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0e4109200 0x7fc0e419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e4108860 0x7fc0e41998f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 --2- 172.21.15.96:0/1361138200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0e4100df0 0x7fc0e41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 >> 172.21.15.96:0/1361138200 conn(0x7fc0e40fca20 msgr2=0x7fc0e40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 shutdown_connections 2024-09-10T23:01:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:01:38.947+0000 7fc0eba02640 1 -- 172.21.15.96:0/1361138200 wait complete. 2024-09-10T23:01:39.654 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:39 smithi096 ceph-mon[30869]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-09-10T23:01:39.654 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:39 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:39 smithi121 ceph-mon[35119]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-09-10T23:01:39.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:39 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:39 smithi117 ceph-mon[35081]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-09-10T23:01:39.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:39 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:01:40.096 INFO:tasks.cephfs.filesystem:scrub status for tag:2df1ad6c-0476-4e75-81a3-b22370ae9b18 - {'path': '/', 'tag': '2df1ad6c-0476-4e75-81a3-b22370ae9b18', 'options': 'recursive,force'} 2024-09-10T23:01:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 845.0 s, 1572.2 tps, lat 62.803 ms stddev 67.374 2024-09-10T23:01:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 -- 172.21.15.96:0/3554034675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb0109230 msgr2=0x7f0eb0109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/3554034675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb0109230 0x7f0eb0109630 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009930 tx=0x7f0ea002a420 comp rx=0 tx=0).stop 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 -- 172.21.15.96:0/3554034675 shutdown_connections 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/3554034675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010a100 0x7f0eb010a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/3554034675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb010aac0 0x7f0eb0111420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/3554034675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb0109230 0x7f0eb0109630 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 -- 172.21.15.96:0/3554034675 >> 172.21.15.96:0/3554034675 conn(0x7f0eb00783d0 msgr2=0x7f0eb007a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 -- 172.21.15.96:0/3554034675 shutdown_connections 2024-09-10T23:01:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.225+0000 7f0eb4df3640 1 -- 172.21.15.96:0/3554034675 wait complete. 2024-09-10T23:01:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 Processor -- start 2024-09-10T23:01:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 -- start start 2024-09-10T23:01:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb010a100 0x7f0eb019f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 0x7f0eb01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0114be0 con 0x7f0eb010a100 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0114a60 con 0x7f0eb010aac0 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eb4df3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0114d60 con 0x7f0eb0109230 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb010a100 0x7f0eb019f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53432/0 (socket says 172.21.15.96:53432) 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 -- 172.21.15.96:0/396177585 learned_addr learned my addr 172.21.15.96:0/396177585 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 0x7f0eb01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 -- 172.21.15.96:0/396177585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 msgr2=0x7f0eb01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 0x7f0eb01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 -- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb010a100 msgr2=0x7f0eb019f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb010a100 0x7f0eb019f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eae59c640 1 -- 172.21.15.96:0/396177585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb01a0ed0 con 0x7f0eb0109230 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.226+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 0x7f0eb01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0eae59c640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009a50 tx=0x7f0ea00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0e9f7fe640 1 -- 172.21.15.96:0/396177585 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea003f030 con 0x7f0eb0109230 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0e9f7fe640 1 -- 172.21.15.96:0/396177585 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea0031910 con 0x7f0eb0109230 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb01a1160 con 0x7f0eb0109230 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0e9f7fe640 1 -- 172.21.15.96:0/396177585 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0035070 con 0x7f0eb0109230 2024-09-10T23:01:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.227+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb01a7890 con 0x7f0eb0109230 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.229+0000 7f0eb7bc4740 1 Processor -- start 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.229+0000 7f0e9f7fe640 1 -- 172.21.15.96:0/396177585 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0ea003b9d0 con 0x7f0eb0109230 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 -- start start 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 0x562a6881b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a687886f0 con 0x562a68816860 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a68777b20 con 0x562a68819190 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562a6877b930 con 0x562a6881bbd0 2024-09-10T23:01:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 0x562a6881b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47890/0 (socket says 172.21.15.96:47890) 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2608834539 learned_addr learned my addr 172.21.15.96:0/2608834539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2608834539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 msgr2=0x562a6881e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2608834539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 msgr2=0x562a6881b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 0x562a6881b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2608834539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6873d530 con 0x562a68816860 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eae59c640 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 0x562a6881b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80117d0 tx=0x7f0ea8011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fc6b8f1e040818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0e9f7fe640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 0x7f0e980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.230+0000 7f0e9cff9640 1 -- 172.21.15.96:0/2608834539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8011220 con 0x562a68816860 2024-09-10T23:01:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eae59c640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 0x7f0e980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0e9cff9640 1 -- 172.21.15.96:0/2608834539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea8003970 con 0x562a68816860 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0e9cff9640 1 -- 172.21.15.96:0/2608834539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80044a0 con 0x562a68816860 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0e9f7fe640 1 -- 172.21.15.96:0/396177585 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0ea01063e0 con 0x7f0eb0109230 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2608834539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 msgr2=0x562a68818c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80117d0 tx=0x7f0ea8011c90 comp rx=0 tx=0).stop 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eae59c640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 0x7f0e980b77a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00cc8a0 tx=0x7f0ea0109e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2608834539 shutdown_connections 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68819190 0x562a6881b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.231+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2608834539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a68818c50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.232+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2608834539 >> 172.21.15.96:0/2608834539 conn(0x562a688119b0 msgr2=0x562a68812740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.232+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2608834539 shutdown_connections 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.232+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2608834539 wait complete. 2024-09-10T23:01:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.232+0000 7f0eb7bc4740 1 -- start start 2024-09-10T23:01:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.232+0000 7f0eb7bc4740 1 Processor -- start 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a6881fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a6881bbd0 0x562a688210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a686afab0 con 0x562a68816860 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a6871d730 con 0x562a6881bbd0 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562a6881ebb0 con 0x562a68819190 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.233+0000 7f0eb7bc4740 10 client.0 _ll_register_callbacks cb 0x562a687716b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:53446/0 (socket says 172.21.15.96:53446) 2024-09-10T23:01:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 -- 172.21.15.96:0/2946787132 learned_addr learned my addr 172.21.15.96:0/2946787132 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a6881fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 -- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a6881bbd0 msgr2=0x562a688210e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a6881bbd0 0x562a688210e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.234+0000 7f0eadd9b640 1 -- 172.21.15.96:0/2946787132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 msgr2=0x562a6881fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a6881fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0eadd9b640 1 -- 172.21.15.96:0/2946787132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a684d9790 con 0x562a68819190 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a6881fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4012740 tx=0x7f0ea4012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4321b311a68a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400f920 con 0x562a68819190 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0ea400fac0 con 0x562a68819190 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.235+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea8003970 con 0x562a68819190 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ea400fcc0 con 0x562a68819190 2024-09-10T23:01:41.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 10 client.28276 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a68552620 con 0x562a68819190 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea8003970 con 0x562a68819190 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea401c420 con 0x562a68819190 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 10 client.28276 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 10 client.28276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 20 client.28276 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a6883c800 0x562a6883ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 4 client.28276 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a6878a640 con 0x562a6883c800 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.236+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a6883c800 0x562a6883ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.237+0000 7f0e95ffb640 10 client.28276 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.237+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a6883c800 0x562a6883ec50 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a023649ae6b8d59c server_cookie=14b19bbf5e0a5abc in_seq=0 out_seq=0 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.237+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2946787132 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a6878a640 con 0x562a6883c800 2024-09-10T23:01:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.237+0000 7f0e95ffb640 10 client.28276 handle_command_reply: tid=0 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.237+0000 7f0eb7bc4740 1 client.28276 shutdown 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 msgr2=0x562a68820510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4012740 tx=0x7f0ea4012c00 comp rx=0 tx=0).stop 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 shutdown_connections 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a6883c800 0x562a6883ec50 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a6881bbd0 0x562a688210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68820510 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0e95ffb640 0 client.28276 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2946787132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68816860 0x562a6881fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 >> 172.21.15.96:0/2946787132 conn(0x562a688119b0 msgr2=0x562a68810bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 shutdown_connections 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2946787132 wait complete. 2024-09-10T23:01:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.238+0000 7f0eb7bc4740 20 client.28276 trim_cache size 0 max 16384 2024-09-10T23:01:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.256+0000 7f0eb7bc4740 1 Processor -- start 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.256+0000 7f0eb7bc4740 1 -- start start 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a6880cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01a7b30 con 0x562a68819190 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01a9650 con 0x562a68816860 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01a81e0 con 0x562a6881bbd0 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47914/0 (socket says 172.21.15.96:47914) 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 -- 172.21.15.96:0/103555840 learned_addr learned my addr 172.21.15.96:0/103555840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eaed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a6880cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 -- 172.21.15.96:0/103555840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 msgr2=0x562a6881c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 -- 172.21.15.96:0/103555840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 msgr2=0x562a6880cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a6880cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 -- 172.21.15.96:0/103555840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a68818fc0 con 0x562a68819190 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400f810 tx=0x7f0ea400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45822458b03918fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0e97fff640 1 -- 172.21.15.96:0/103555840 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400f920 con 0x562a68819190 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0e97fff640 1 -- 172.21.15.96:0/103555840 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4012e50 con 0x562a68819190 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.257+0000 7f0e97fff640 1 -- 172.21.15.96:0/103555840 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003850 con 0x562a68819190 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/103555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 msgr2=0x562a6881bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400f810 tx=0x7f0ea400f840 comp rx=0 tx=0).stop 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/103555840 shutdown_connections 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a6881bbd0 0x562a6881c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a68819190 0x562a6881bfb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/103555840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a6880cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/103555840 >> 172.21.15.96:0/103555840 conn(0x562a688119b0 msgr2=0x562a68815cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/103555840 shutdown_connections 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/103555840 wait complete. 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.258+0000 7f0eb7bc4740 1 -- start start 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 Processor -- start 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a68827570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68827c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01a8f80 con 0x562a6881bbd0 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01a88b0 con 0x562a68816860 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0073430 con 0x562a68819190 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eb7bc4740 10 client.0 _ll_register_callbacks cb 0x7f0ea4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47922/0 (socket says 172.21.15.96:47922) 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2250411286 learned_addr learned my addr 172.21.15.96:0/2250411286 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a68827570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68827c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2250411286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 msgr2=0x562a68827c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68827c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 msgr2=0x562a68827570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a68827570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 -- 172.21.15.96:0/2250411286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a68826c50 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0eaed9d640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80115e0 tx=0x7f0ea802bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bfcb162c5e4794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea801d040 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.259+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0ea8025440 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea4012e50 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ea803f050 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 10 client.18594 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a68825ae0 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4012e50 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea8023040 con 0x562a6881bbd0 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 10 client.18594 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 10 client.18594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 20 client.18594 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a688a9ba0 0x562a688abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 4 client.18594 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a687860f0 con 0x562a688a9ba0 2024-09-10T23:01:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.260+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a688a9ba0 0x562a688abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.261+0000 7f0eadd9b640 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a688a9ba0 0x562a688abf90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58b90f34587fcd5 server_cookie=c514ec682d08e8b5 in_seq=0 out_seq=0 2024-09-10T23:01:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.261+0000 7f0e95ffb640 10 client.18594 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0e95ffb640 1 -- 172.21.15.96:0/2250411286 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a687860f0 con 0x562a688a9ba0 2024-09-10T23:01:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0e95ffb640 10 client.18594 handle_command_reply: tid=0 2024-09-10T23:01:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 client.18594 shutdown 2024-09-10T23:01:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 msgr2=0x562a68826420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80115e0 tx=0x7f0ea802bf90 comp rx=0 tx=0).stop 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 shutdown_connections 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x562a688a9ba0 0x562a688abf90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a6881bbd0 0x562a68826420 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a68819190 0x562a68827c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0e95ffb640 0 client.18594 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 --2- 172.21.15.96:0/2250411286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a68816860 0x562a68827570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 >> 172.21.15.96:0/2250411286 conn(0x562a688119b0 msgr2=0x562a68815c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 shutdown_connections 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 1 -- 172.21.15.96:0/2250411286 wait complete. 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.376+0000 7f0eb7bc4740 20 client.18594 trim_cache size 0 max 16384 2024-09-10T23:01:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:01:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:01:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:01:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:01:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 msgr2=0x7f0e980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 0x7f0e980b77a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00cc8a0 tx=0x7f0ea0109e20 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 msgr2=0x7f0eb019f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009a50 tx=0x7f0ea00042f0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 shutdown_connections 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb010aac0 0x7f0eb01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0e980b52e0 0x7f0e980b77a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0eb010a100 0x7f0eb019f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 --2- 172.21.15.96:0/396177585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb0109230 0x7f0eb019f390 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 >> 172.21.15.96:0/396177585 conn(0x7f0eb00783d0 msgr2=0x7f0eb010d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 shutdown_connections 2024-09-10T23:01:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:41.377+0000 7f0eb4df3640 1 -- 172.21.15.96:0/396177585 wait complete. 2024-09-10T23:01:41.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:41 smithi096 ceph-mon[30869]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.6 MiB/s wr, 796 op/s 2024-09-10T23:01:41.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:41 smithi121 ceph-mon[35119]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.6 MiB/s wr, 796 op/s 2024-09-10T23:01:41.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:41 smithi117 ceph-mon[35081]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.6 MiB/s wr, 796 op/s 2024-09-10T23:01:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:43 smithi096 ceph-mon[30869]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.6 MiB/s wr, 676 op/s 2024-09-10T23:01:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:01:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:43 smithi121 ceph-mon[35119]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.6 MiB/s wr, 676 op/s 2024-09-10T23:01:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:01:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:43 smithi117 ceph-mon[35081]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.6 MiB/s wr, 676 op/s 2024-09-10T23:01:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:01:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:01:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:01:43.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:01:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:45 smithi096 ceph-mon[30869]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.6 MiB/s wr, 680 op/s 2024-09-10T23:01:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:45 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:01:45.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:45 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:01:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:45 smithi121 ceph-mon[35119]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.6 MiB/s wr, 680 op/s 2024-09-10T23:01:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:45 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:01:45.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:45 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:01:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:45 smithi117 ceph-mon[35081]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.6 MiB/s wr, 680 op/s 2024-09-10T23:01:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:45 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:01:45.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:45 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:01:45.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 850.0 s, 2566.4 tps, lat 39.474 ms stddev 46.547 2024-09-10T23:01:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:47 smithi096 ceph-mon[30869]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 755 op/s 2024-09-10T23:01:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:47 smithi121 ceph-mon[35119]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 755 op/s 2024-09-10T23:01:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:47 smithi117 ceph-mon[35081]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.5 MiB/s wr, 755 op/s 2024-09-10T23:01:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:49 smithi096 ceph-mon[30869]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 626 op/s 2024-09-10T23:01:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:49 smithi121 ceph-mon[35119]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 626 op/s 2024-09-10T23:01:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:49 smithi117 ceph-mon[35081]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 626 op/s 2024-09-10T23:01:50.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 855.0 s, 1514.2 tps, lat 65.627 ms stddev 70.007 2024-09-10T23:01:51.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:51 smithi096 ceph-mon[30869]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.4 MiB/s wr, 809 op/s 2024-09-10T23:01:51.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:51 smithi121 ceph-mon[35119]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.4 MiB/s wr, 809 op/s 2024-09-10T23:01:51.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:51 smithi117 ceph-mon[35081]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.4 MiB/s wr, 809 op/s 2024-09-10T23:01:53.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:53 smithi096 ceph-mon[30869]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-09-10T23:01:53.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:53 smithi121 ceph-mon[35119]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-09-10T23:01:53.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:53 smithi117 ceph-mon[35081]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-09-10T23:01:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.505+0000 7f9a6855d640 1 -- 172.21.15.96:0/492055032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 msgr2=0x7f9a60074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.505+0000 7f9a6855d640 1 --2- 172.21.15.96:0/492055032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60074690 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009900 tx=0x7f9a5802a440 comp rx=0 tx=0).stop 2024-09-10T23:01:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.517+0000 7f9a6855d640 1 -- 172.21.15.96:0/492055032 shutdown_connections 2024-09-10T23:01:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.517+0000 7f9a6855d640 1 --2- 172.21.15.96:0/492055032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a6010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.517+0000 7f9a6855d640 1 --2- 172.21.15.96:0/492055032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a6011a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.517+0000 7f9a6855d640 1 --2- 172.21.15.96:0/492055032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60074690 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.517+0000 7f9a6855d640 1 -- 172.21.15.96:0/492055032 >> 172.21.15.96:0/492055032 conn(0x7f9a6006fa40 msgr2=0x7f9a60071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.522+0000 7f9a6855d640 1 -- 172.21.15.96:0/492055032 shutdown_connections 2024-09-10T23:01:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.523+0000 7f9a6855d640 1 -- 172.21.15.96:0/492055032 wait complete. 2024-09-10T23:01:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.524+0000 7f9a6855d640 1 Processor -- start 2024-09-10T23:01:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.524+0000 7f9a6855d640 1 -- start start 2024-09-10T23:01:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60119660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a60119be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60112180 con 0x7f9a60074290 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60112000 con 0x7f9a6010a150 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40700/0 (socket says 172.21.15.96:40700) 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a66afa640 1 -- 172.21.15.96:0/936290345 learned_addr learned my addr 172.21.15.96:0/936290345 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.525+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60112300 con 0x7f9a6010ab10 2024-09-10T23:01:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.528+0000 7f9a662f9640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60119660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.531+0000 7f9a65af8640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a60119be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.532+0000 7f9a66afa640 1 -- 172.21.15.96:0/936290345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 msgr2=0x7f9a60119be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.532+0000 7f9a66afa640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a60119be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.532+0000 7f9a66afa640 1 -- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 msgr2=0x7f9a60119660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.532+0000 7f9a66afa640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60119660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.532+0000 7f9a66afa640 1 -- 172.21.15.96:0/936290345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a60118700 con 0x7f9a6010ab10 2024-09-10T23:01:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.536+0000 7f9a65af8640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a60119be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.536+0000 7f9a66afa640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b970 tx=0x7f9a5c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.541+0000 7f9a577fe640 1 -- 172.21.15.96:0/936290345 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c019070 con 0x7f9a6010ab10 2024-09-10T23:01:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.541+0000 7f9a577fe640 1 -- 172.21.15.96:0/936290345 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c014440 con 0x7f9a6010ab10 2024-09-10T23:01:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.541+0000 7f9a577fe640 1 -- 172.21.15.96:0/936290345 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c007630 con 0x7f9a6010ab10 2024-09-10T23:01:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.541+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a601189f0 con 0x7f9a6010ab10 2024-09-10T23:01:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.541+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601b8980 con 0x7f9a6010ab10 2024-09-10T23:01:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.545+0000 7f9a6b40b740 1 Processor -- start 2024-09-10T23:01:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.547+0000 7f9a6b40b740 1 -- start start 2024-09-10T23:01:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563edf2366f0 con 0x563edf2c7190 2024-09-10T23:01:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563edf225b20 con 0x563edf2c4860 2024-09-10T23:01:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.548+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563edf229930 con 0x563edf2c9bd0 2024-09-10T23:01:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.549+0000 7f9a577fe640 1 -- 172.21.15.96:0/936290345 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9a5c00cb40 con 0x7f9a6010ab10 2024-09-10T23:01:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.550+0000 7f9a65af8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.550+0000 7f9a65af8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40712/0 (socket says 172.21.15.96:40712) 2024-09-10T23:01:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.550+0000 7f9a65af8640 1 -- 172.21.15.96:0/3152122000 learned_addr learned my addr 172.21.15.96:0/3152122000 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.550+0000 7f9a65af8640 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.552+0000 7f9a65af8640 1 -- 172.21.15.96:0/3152122000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 msgr2=0x563edf2c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.552+0000 7f9a65af8640 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.552+0000 7f9a65af8640 1 -- 172.21.15.96:0/3152122000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 msgr2=0x563edf2c9580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:01:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.552+0000 7f9a65af8640 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2c9580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.552+0000 7f9a65af8640 1 -- 172.21.15.96:0/3152122000 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edf1eb530 con 0x563edf2c9bd0 2024-09-10T23:01:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.553+0000 7f9a577fe640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 0x7f9a4c0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.554+0000 7f9a577fe640 1 -- 172.21.15.96:0/936290345 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9a5c0e5330 con 0x7f9a6010ab10 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.555+0000 7f9a65af8640 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000cfc0 tx=0x7f9a5000c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f350e08fb84e6558 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.555+0000 7f9a54bfe640 1 -- 172.21.15.96:0/3152122000 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000f040 con 0x563edf2c9bd0 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.555+0000 7f9a54bfe640 1 -- 172.21.15.96:0/3152122000 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50003690 con 0x563edf2c9bd0 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.556+0000 7f9a6b40b740 1 -- 172.21.15.96:0/3152122000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 msgr2=0x563edf2cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.556+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000cfc0 tx=0x7f9a5000c1d0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.557+0000 7f9a662f9640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 0x7f9a4c0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.559+0000 7f9a6b40b740 1 -- 172.21.15.96:0/3152122000 shutdown_connections 2024-09-10T23:01:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.559+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cc090 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.559+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.559+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/3152122000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.559+0000 7f9a6b40b740 1 -- 172.21.15.96:0/3152122000 >> 172.21.15.96:0/3152122000 conn(0x563edf2bf9b0 msgr2=0x563edf2c0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.561+0000 7f9a662f9640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 0x7f9a4c0b79b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a580095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.562+0000 7f9a6b40b740 1 -- 172.21.15.96:0/3152122000 shutdown_connections 2024-09-10T23:01:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.564+0000 7f9a6b40b740 1 -- 172.21.15.96:0/3152122000 wait complete. 2024-09-10T23:01:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.564+0000 7f9a6b40b740 1 -- start start 2024-09-10T23:01:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.564+0000 7f9a6b40b740 1 Processor -- start 2024-09-10T23:01:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563edf15dab0 con 0x563edf2c7190 2024-09-10T23:01:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563edf1cb730 con 0x563edf2c4860 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563edf2ccbf0 con 0x563edf2c9bd0 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.565+0000 7f9a6b40b740 10 client.0 _ll_register_callbacks cb 0x563edf21f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:40728/0 (socket says 172.21.15.96:40728) 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 -- 172.21.15.96:0/2354488153 learned_addr learned my addr 172.21.15.96:0/2354488153 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a65af8640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a662f9640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 -- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 msgr2=0x563edf2cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 -- 172.21.15.96:0/2354488153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 msgr2=0x563edf2ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.566+0000 7f9a66afa640 1 -- 172.21.15.96:0/2354488153 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edef87790 con 0x563edf2c9bd0 2024-09-10T23:01:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a662f9640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a66afa640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ab7f0 tx=0x7f9a5c0a8270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c9018c45e7e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00ce60 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9a5c0b00d0 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a50003690 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a5c0c6050 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a6b40b740 10 client.28291 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.567+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563edf000620 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50003690 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9a5c0a7440 con 0x563edf2c9bd0 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 10 client.28291 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 10 client.28291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 20 client.28291 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf2d9bc0 0x563edf2dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 4 client.28291 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:01:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563edf238640 con 0x563edf2d9bc0 2024-09-10T23:01:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.568+0000 7f9a662f9640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf2d9bc0 0x563edf2dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.569+0000 7f9a662f9640 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf2d9bc0 0x563edf2dc060 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c860de257dc507a server_cookie=546b23c13a45b7a0 in_seq=0 out_seq=0 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.569+0000 7f9a49ffb640 10 client.28291 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.570+0000 7f9a49ffb640 1 -- 172.21.15.96:0/2354488153 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563edf238640 con 0x563edf2d9bc0 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.570+0000 7f9a49ffb640 10 client.28291 handle_command_reply: tid=0 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.570+0000 7f9a6b40b740 1 client.28291 shutdown 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.570+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 msgr2=0x563edf2cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.570+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ab7f0 tx=0x7f9a5c0a8270 comp rx=0 tx=0).stop 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 shutdown_connections 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf2d9bc0 0x563edf2dc060 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c9bd0 0x563edf2cf0d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c7190 0x563edf2ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/2354488153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 >> 172.21.15.96:0/2354488153 conn(0x563edf2bf9b0 msgr2=0x563edf2bebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a49ffb640 0 client.28291 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 shutdown_connections 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 1 -- 172.21.15.96:0/2354488153 wait complete. 2024-09-10T23:01:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.572+0000 7f9a6b40b740 20 client.28291 trim_cache size 0 max 16384 2024-09-10T23:01:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 Processor -- start 2024-09-10T23:01:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 -- start start 2024-09-10T23:01:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601c15f0 con 0x563edf2c9bd0 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601c3130 con 0x563edf2c4860 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601c1cc0 con 0x563edf2c7190 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37968/0 (socket says 172.21.15.96:37968) 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 -- 172.21.15.96:0/4119564615 learned_addr learned my addr 172.21.15.96:0/4119564615 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a65af8640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a662f9640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 -- 172.21.15.96:0/4119564615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 msgr2=0x563edf2c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 -- 172.21.15.96:0/4119564615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 msgr2=0x563edf2cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 -- 172.21.15.96:0/4119564615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edf2cd800 con 0x563edf2c9bd0 2024-09-10T23:01:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a662f9640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:01:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.593+0000 7f9a66afa640 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0a80c0 tx=0x7f9a5c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babbf4f9a8b38b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a4bfff640 1 -- 172.21.15.96:0/4119564615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0ca070 con 0x563edf2c9bd0 2024-09-10T23:01:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a4bfff640 1 -- 172.21.15.96:0/4119564615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c0ab600 con 0x563edf2c9bd0 2024-09-10T23:01:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a4bfff640 1 -- 172.21.15.96:0/4119564615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b05d0 con 0x563edf2c9bd0 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4119564615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 msgr2=0x563edf2c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0a80c0 tx=0x7f9a5c0098b0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4119564615 shutdown_connections 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2c2f40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4119564615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.594+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4119564615 >> 172.21.15.96:0/4119564615 conn(0x563edf2bf9b0 msgr2=0x563edf2baaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4119564615 shutdown_connections 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4119564615 wait complete. 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- start start 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 Processor -- start 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a662f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a662f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:58596/0 (socket says 172.21.15.96:58596) 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a66afa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601c2a60 con 0x563edf2c9bd0 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a65af8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a662f9640 1 -- 172.21.15.96:0/4050697867 learned_addr learned my addr 172.21.15.96:0/4050697867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601c2390 con 0x563edf2c4860 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601bf5d0 con 0x563edf2c7190 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.595+0000 7f9a6b40b740 10 client.0 _ll_register_callbacks cb 0x7f9a5c014910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 -- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 msgr2=0x563edf2d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 -- 172.21.15.96:0/4050697867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 msgr2=0x563edf2d4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2d4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edf2d51d0 con 0x563edf2c7190 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a662f9640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a66afa640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a65af8640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2d4550 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006be0 tx=0x7f9a50018aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139755e71ab45f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50028070 con 0x563edf2c7190 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9a5001e040 con 0x563edf2c7190 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a5c0ab600 con 0x563edf2c7190 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a50034050 con 0x563edf2c7190 2024-09-10T23:01:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a6b40b740 10 client.28297 fetch_fsmap learned FSMap version 32 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563edf346b70 con 0x563edf2c7190 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.596+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c0ab600 con 0x563edf2c7190 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9a50015040 con 0x563edf2c7190 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 10 client.28297 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 10 client.28297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 20 client.28297 populate_metadata read hostname 'smithi096' 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf346fe0 0x563edf3493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 4 client.28297 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563edf2340f0 con 0x563edf346fe0 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.597+0000 7f9a662f9640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf346fe0 0x563edf3493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:01:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.598+0000 7f9a662f9640 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf346fe0 0x563edf3493d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962cb6f6ca8fadfc server_cookie=eb62a8af86da54d0 in_seq=0 out_seq=0 2024-09-10T23:01:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.598+0000 7f9a49ffb640 10 client.28297 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a49ffb640 1 -- 172.21.15.96:0/4050697867 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563edf2340f0 con 0x563edf346fe0 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a49ffb640 10 client.28297 handle_command_reply: tid=0 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 client.28297 shutdown 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 msgr2=0x563edf2d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2d4550 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006be0 tx=0x7f9a50018aa0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 shutdown_connections 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563edf346fe0 0x563edf3493d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edf2c9bd0 0x563edf2d4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563edf2c7190 0x563edf2d4550 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 --2- 172.21.15.96:0/4050697867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563edf2c4860 0x563edf2d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a49ffb640 0 client.28297 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 >> 172.21.15.96:0/4050697867 conn(0x563edf2bf9b0 msgr2=0x563edf2c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 shutdown_connections 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 1 -- 172.21.15.96:0/4050697867 wait complete. 2024-09-10T23:01:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.613+0000 7f9a6b40b740 20 client.28297 trim_cache size 0 max 16384 2024-09-10T23:01:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:01:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:01:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:01:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 msgr2=0x7f9a4c0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 0x7f9a4c0b79b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a580095f0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 msgr2=0x7f9a60117f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b970 tx=0x7f9a5c00be30 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 shutdown_connections 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a6010a150 0x7f9a60119be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9a4c0b54f0 0x7f9a4c0b79b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a6010ab10 0x7f9a60117f60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 --2- 172.21.15.96:0/936290345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a60074290 0x7f9a60119660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 >> 172.21.15.96:0/936290345 conn(0x7f9a6006fa40 msgr2=0x7f9a6010b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 shutdown_connections 2024-09-10T23:01:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:01:54.614+0000 7f9a6855d640 1 -- 172.21.15.96:0/936290345 wait complete. 2024-09-10T23:01:55.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:55 smithi096 ceph-mon[30869]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 649 op/s 2024-09-10T23:01:55.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:55 smithi121 ceph-mon[35119]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 649 op/s 2024-09-10T23:01:55.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:55 smithi117 ceph-mon[35081]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 649 op/s 2024-09-10T23:01:55.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 860.0 s, 1531.6 tps, lat 65.767 ms stddev 74.295 2024-09-10T23:01:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:57 smithi096 ceph-mon[30869]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 MiB/s wr, 713 op/s 2024-09-10T23:01:57.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:57 smithi121 ceph-mon[35119]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 MiB/s wr, 713 op/s 2024-09-10T23:01:57.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:57 smithi117 ceph-mon[35081]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 MiB/s wr, 713 op/s 2024-09-10T23:01:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:01:59 smithi096 ceph-mon[30869]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 MiB/s wr, 579 op/s 2024-09-10T23:01:59.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:01:59 smithi121 ceph-mon[35119]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 MiB/s wr, 579 op/s 2024-09-10T23:01:59.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:01:59 smithi117 ceph-mon[35081]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 MiB/s wr, 579 op/s 2024-09-10T23:02:00.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 865.0 s, 1402.8 tps, lat 70.881 ms stddev 77.975 2024-09-10T23:02:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:01 smithi096 ceph-mon[30869]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-09-10T23:02:01.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:01 smithi121 ceph-mon[35119]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-09-10T23:02:01.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:01 smithi117 ceph-mon[35081]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-09-10T23:02:03.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:03 smithi096 ceph-mon[30869]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:02:03.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:03 smithi121 ceph-mon[35119]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:02:03.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:03 smithi117 ceph-mon[35081]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-10T23:02:05.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:05 smithi096 ceph-mon[30869]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:02:05.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:05 smithi121 ceph-mon[35119]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:02:05.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:05 smithi117 ceph-mon[35081]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.2 MiB/s wr, 606 op/s 2024-09-10T23:02:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 870.0 s, 1656.4 tps, lat 60.310 ms stddev 63.418 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 -- 172.21.15.96:0/3071121087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 msgr2=0x7f1670102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 --2- 172.21.15.96:0/3071121087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670102e30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1658009930 tx=0x7f165802a420 comp rx=0 tx=0).stop 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 -- 172.21.15.96:0/3071121087 shutdown_connections 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 --2- 172.21.15.96:0/3071121087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 0x7f1670109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 --2- 172.21.15.96:0/3071121087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670102e30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 --2- 172.21.15.96:0/3071121087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 -- 172.21.15.96:0/3071121087 >> 172.21.15.96:0/3071121087 conn(0x7f16700fd140 msgr2=0x7f16700ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 -- 172.21.15.96:0/3071121087 shutdown_connections 2024-09-10T23:02:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.748+0000 7f1675ec6640 1 -- 172.21.15.96:0/3071121087 wait complete. 2024-09-10T23:02:07.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:07 smithi096 ceph-mon[30869]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 Processor -- start 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 -- start start 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 0x7f1670197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f167010c3f0 con 0x7f16701029b0 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f167010c270 con 0x7f1670101860 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f1675ec6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f167010c570 con 0x7f1670103370 2024-09-10T23:02:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.749+0000 7f166effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50122/0 (socket says 172.21.15.96:50122) 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 -- 172.21.15.96:0/2428668988 learned_addr learned my addr 172.21.15.96:0/2428668988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166ffff640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 0x7f1670197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 -- 172.21.15.96:0/2428668988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 msgr2=0x7f1670197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 0x7f1670197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 -- 172.21.15.96:0/2428668988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 msgr2=0x7f1670196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 -- 172.21.15.96:0/2428668988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1670198670 con 0x7f16701029b0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166f7fe640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166effd640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1658005d90 tx=0x7f1658004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166cff9640 1 -- 172.21.15.96:0/2428668988 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658040070 con 0x7f16701029b0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1670198900 con 0x7f16701029b0 2024-09-10T23:02:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166cff9640 1 -- 172.21.15.96:0/2428668988 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1658004c60 con 0x7f16701029b0 2024-09-10T23:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f166cff9640 1 -- 172.21.15.96:0/2428668988 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658035020 con 0x7f16701029b0 2024-09-10T23:02:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.750+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167019f030 con 0x7f16701029b0 2024-09-10T23:02:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.753+0000 7f166cff9640 1 -- 172.21.15.96:0/2428668988 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f165803a040 con 0x7f16701029b0 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.753+0000 7f1678c98740 1 Processor -- start 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f166cff9640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 0x7f165c0b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f166ffff640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 0x7f165c0b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f166cff9640 1 -- 172.21.15.96:0/2428668988 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1658003810 con 0x7f16701029b0 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f1678c98740 1 -- start start 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f166ffff640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 0x7f165c0b9bb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f16701978b0 tx=0x7f1668009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.754+0000 7f1678c98740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 0x55e3b8e38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f1678c98740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e3b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f1678c98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f166ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f166ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50136/0 (socket says 172.21.15.96:50136) 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f166ffff640 1 -- 172.21.15.96:0/1539396343 learned_addr learned my addr 172.21.15.96:0/1539396343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f166f7fe640 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 0x55e3b8e38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8da86f0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8d97b20 con 0x55e3b8e39190 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f166effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e3b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.755+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8d9b930 con 0x55e3b8e36860 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 -- 172.21.15.96:0/1539396343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 msgr2=0x55e3b8e3b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e3b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 -- 172.21.15.96:0/1539396343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 msgr2=0x55e3b8e3e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 -- 172.21.15.96:0/1539396343 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8d5d530 con 0x55e3b8e36860 2024-09-10T23:02:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.756+0000 7f166f7fe640 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 0x55e3b8e38c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1660009fd0 tx=0x7f16600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ec4bde2f20b6403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.757+0000 7f1665ffb640 1 -- 172.21.15.96:0/1539396343 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166001e070 con 0x55e3b8e36860 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.757+0000 7f1665ffb640 1 -- 172.21.15.96:0/1539396343 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16600035f0 con 0x55e3b8e36860 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.757+0000 7f1665ffb640 1 -- 172.21.15.96:0/1539396343 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000f800 con 0x55e3b8e36860 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 msgr2=0x55e3b8e38c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 0x55e3b8e38c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1660009fd0 tx=0x7f16600118d0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 shutdown_connections 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e3b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 --2- 172.21.15.96:0/1539396343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e36860 0x55e3b8e38c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 >> 172.21.15.96:0/1539396343 conn(0x55e3b8e319b0 msgr2=0x55e3b8e32740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 shutdown_connections 2024-09-10T23:02:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- 172.21.15.96:0/1539396343 wait complete. 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.758+0000 7f1678c98740 1 -- start start 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 Processor -- start 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8ccfab0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8d3d730 con 0x55e3b8e36860 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e3b8e3ebb0 con 0x55e3b8e39190 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f1678c98740 10 client.0 _ll_register_callbacks cb 0x55e3b8d916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57694/0 (socket says 172.21.15.96:57694) 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 -- 172.21.15.96:0/4113508575 learned_addr learned my addr 172.21.15.96:0/4113508575 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166ffff640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166effd640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 -- 172.21.15.96:0/4113508575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 msgr2=0x55e3b8e405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 -- 172.21.15.96:0/4113508575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 msgr2=0x55e3b8e411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 -- 172.21.15.96:0/4113508575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8af9790 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f166f7fe640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1660003cc0 tx=0x7f1660003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33397e773a8c88c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.759+0000 7f164effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166001e070 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1660005250 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16600035f0 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1660004330 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 10 client.28235 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3b8b72620 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1660028070 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f166000f6c0 con 0x55e3b8e36860 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 10 client.28235 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 10 client.28235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 20 client.28235 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8e4b490 0x55e3b8e4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 4 client.28235 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.760+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f16600035f0 con 0x55e3b8e4b490 2024-09-10T23:02:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.761+0000 7f166effd640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8e4b490 0x55e3b8e4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.766 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:07 smithi121 ceph-mon[35119]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-09-10T23:02:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.762+0000 7f164effd640 10 client.28235 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.762+0000 7f166effd640 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8e4b490 0x55e3b8e4d8f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db84875ec598301f server_cookie=fb3960d77b28189f in_seq=0 out_seq=0 2024-09-10T23:02:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.763+0000 7f164effd640 1 -- 172.21.15.96:0/4113508575 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f16600035f0 con 0x55e3b8e4b490 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.763+0000 7f164effd640 10 client.28235 handle_command_reply: tid=0 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.763+0000 7f1678c98740 1 client.28235 shutdown 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.763+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 msgr2=0x55e3b8e3ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.763+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1660003cc0 tx=0x7f1660003e30 comp rx=0 tx=0).stop 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 shutdown_connections 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8e4b490 0x55e3b8e4d8f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 --2- 172.21.15.96:0/4113508575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3ff50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f164effd640 0 client.28235 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 >> 172.21.15.96:0/4113508575 conn(0x55e3b8e319b0 msgr2=0x55e3b8e30bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 shutdown_connections 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 1 -- 172.21.15.96:0/4113508575 wait complete. 2024-09-10T23:02:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.764+0000 7f1678c98740 20 client.28235 trim_cache size 0 max 16384 2024-09-10T23:02:07.775 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:07 smithi117 ceph-mon[35081]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 Processor -- start 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 -- start start 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a7be0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a9720 con 0x55e3b8e36860 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f1678c98740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a82b0 con 0x55e3b8e39190 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f166ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f166effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f166ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50148/0 (socket says 172.21.15.96:50148) 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f166ffff640 1 -- 172.21.15.96:0/3979820770 learned_addr learned my addr 172.21.15.96:0/3979820770 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.782+0000 7f166f7fe640 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 -- 172.21.15.96:0/3979820770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 msgr2=0x55e3b8e3c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e3c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 -- 172.21.15.96:0/3979820770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 msgr2=0x55e3b8e3bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 -- 172.21.15.96:0/3979820770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8e3b8f0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f166ffff640 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f166800e720 tx=0x7f1668014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d020fafb0a05ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f16657fa640 1 -- 172.21.15.96:0/3979820770 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668004b20 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f16657fa640 1 -- 172.21.15.96:0/3979820770 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1668004cc0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.783+0000 7f16657fa640 1 -- 172.21.15.96:0/3979820770 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668004710 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 -- 172.21.15.96:0/3979820770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 msgr2=0x55e3b8e3cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f166800e720 tx=0x7f1668014000 comp rx=0 tx=0).stop 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 -- 172.21.15.96:0/3979820770 shutdown_connections 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e3cc30 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e39190 0x55e3b8e3c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 --2- 172.21.15.96:0/3979820770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e36860 0x55e3b8e3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.784+0000 7f1678c98740 1 -- 172.21.15.96:0/3979820770 >> 172.21.15.96:0/3979820770 conn(0x55e3b8e319b0 msgr2=0x55e3b8e35d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.785+0000 7f1678c98740 1 -- 172.21.15.96:0/3979820770 shutdown_connections 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.785+0000 7f1678c98740 1 -- 172.21.15.96:0/3979820770 wait complete. 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.785+0000 7f1678c98740 1 -- start start 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 Processor -- start 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 0x55e3b8e44b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a9050 con 0x55e3b8e36860 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a8980 con 0x55e3b8e39190 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a5bc0 con 0x55e3b8e3bbd0 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f1678c98740 10 client.0 _ll_register_callbacks cb 0x7f166001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f166ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f166effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f166ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59138/0 (socket says 172.21.15.96:59138) 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f166ffff640 1 -- 172.21.15.96:0/2420367322 learned_addr learned my addr 172.21.15.96:0/2420367322 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.786+0000 7f166f7fe640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 0x55e3b8e44b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 -- 172.21.15.96:0/2420367322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 msgr2=0x55e3b8e45dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 -- 172.21.15.96:0/2420367322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 msgr2=0x55e3b8e44b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 0x55e3b8e44b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 -- 172.21.15.96:0/2420367322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8e465f0 con 0x55e3b8e39190 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166f7fe640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 0x55e3b8e44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166ffff640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f166effd640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e451f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1658009900 tx=0x7f16580c92e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1db3fa8faf69511 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f164effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658035040 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f16580c8f00 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1668004cc0 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f165804a040 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f1678c98740 10 client.28241 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.787+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3b8e479d0 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1668004cc0 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1658003a10 con 0x55e3b8e39190 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 10 client.28241 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 10 client.28241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 20 client.28241 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8eb84d0 0x55e3b8eba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 4 client.28241 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3b8daa640 con 0x55e3b8eb84d0 2024-09-10T23:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f166f7fe640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8eb84d0 0x55e3b8eba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f164effd640 10 client.28241 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.788+0000 7f166f7fe640 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8eb84d0 0x55e3b8eba8c0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca8a936375a49e7 server_cookie=dc861f3ce0c4b153 in_seq=0 out_seq=0 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f164effd640 1 -- 172.21.15.96:0/2420367322 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3b8daa640 con 0x55e3b8eb84d0 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f164effd640 10 client.28241 handle_command_reply: tid=0 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 client.28241 shutdown 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 msgr2=0x55e3b8e451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e451f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1658009900 tx=0x7f16580c92e0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 shutdown_connections 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e3b8eb84d0 0x55e3b8eba8c0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3b8e39190 0x55e3b8e451f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f164effd640 0 client.28241 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e3b8e3bbd0 0x55e3b8e45dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 --2- 172.21.15.96:0/2420367322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e3b8e36860 0x55e3b8e44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 >> 172.21.15.96:0/2420367322 conn(0x55e3b8e319b0 msgr2=0x55e3b8e36130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 shutdown_connections 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 1 -- 172.21.15.96:0/2420367322 wait complete. 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.805+0000 7f1678c98740 20 client.28241 trim_cache size 0 max 16384 2024-09-10T23:02:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:02:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:02:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:02:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 msgr2=0x7f165c0b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 0x7f165c0b9bb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f16701978b0 tx=0x7f1668009500 comp rx=0 tx=0).stop 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 msgr2=0x7f1670197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1658005d90 tx=0x7f1658004570 comp rx=0 tx=0).stop 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 shutdown_connections 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f165c0b76f0 0x7f165c0b9bb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1670103370 0x7f1670197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16701029b0 0x7f1670197180 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 --2- 172.21.15.96:0/2428668988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1670101860 0x7f1670196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 >> 172.21.15.96:0/2428668988 conn(0x7f16700fd140 msgr2=0x7f1670105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 shutdown_connections 2024-09-10T23:02:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:07.806+0000 7f1675ec6640 1 -- 172.21.15.96:0/2428668988 wait complete. 2024-09-10T23:02:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:09 smithi096 ceph-mon[30869]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T23:02:09.764 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:09 smithi121 ceph-mon[35119]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T23:02:09.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:09 smithi117 ceph-mon[35081]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-09-10T23:02:10.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:02:10.481 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:10.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 875.0 s, 2036.8 tps, lat 49.328 ms stddev 54.470 2024-09-10T23:02:11.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- 172.21.15.96:0/616314457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce81087b0 msgr2=0x7f9ce8108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 --2- 172.21.15.96:0/616314457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce81087b0 0x7f9ce8108bf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009900 tx=0x7f9cd002a410 comp rx=0 tx=0).stop 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- 172.21.15.96:0/616314457 shutdown_connections 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 --2- 172.21.15.96:0/616314457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce8109130 0x7f9ce810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 --2- 172.21.15.96:0/616314457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce81087b0 0x7f9ce8108bf0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 --2- 172.21.15.96:0/616314457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- 172.21.15.96:0/616314457 >> 172.21.15.96:0/616314457 conn(0x7f9ce80fe850 msgr2=0x7f9ce8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- 172.21.15.96:0/616314457 shutdown_connections 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- 172.21.15.96:0/616314457 wait complete. 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 Processor -- start 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.987+0000 7f9cedaab640 1 -- start start 2024-09-10T23:02:11.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce810c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce81087b0 0x7f9ce810c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8076640 con 0x7f9ce8109130 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce80764c0 con 0x7f9ce81087b0 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce80767c0 con 0x7f9ce8102da0 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50176/0 (socket says 172.21.15.96:50176) 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 -- 172.21.15.96:0/374370474 learned_addr learned my addr 172.21.15.96:0/374370474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce810c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cdffff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce81087b0 0x7f9ce810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 -- 172.21.15.96:0/374370474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 msgr2=0x7f9ce810c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce810c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 -- 172.21.15.96:0/374370474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce81087b0 msgr2=0x7f9ce810c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce81087b0 0x7f9ce810c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 -- 172.21.15.96:0/374370474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce81abab0 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce7fff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eaa0 tx=0x7f9ce000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce57fa640 1 -- 172.21.15.96:0/374370474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000cdb0 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81abda0 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce57fa640 1 -- 172.21.15.96:0/374370474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce0005020 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9ce57fa640 1 -- 172.21.15.96:0/374370474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00116d0 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.988+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81ac260 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce57fa640 1 -- 172.21.15.96:0/374370474 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9ce000f040 con 0x7f9ce8109130 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 Processor -- start 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 -- start start 2024-09-10T23:02:11.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 0x55b419f7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b419ee4e50 con 0x55b419f6eda0 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b419ed4280 con 0x55b419f7ad30 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cefe80740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b419ed8090 con 0x55b419f6e480 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50178/0 (socket says 172.21.15.96:50178) 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1397626135 learned_addr learned my addr 172.21.15.96:0/1397626135 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 0x55b419f7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1397626135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 msgr2=0x55b419f6e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f6e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1397626135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 msgr2=0x55b419f7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 0x55b419f7d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1397626135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b419c32650 con 0x55b419f6eda0 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 0x55b419f7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce57fa640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 0x7f9cd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.992+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8010580 tx=0x7f9cd8010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefa4b45a8e11343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cdffff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 0x7f9cd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cddffb640 1 -- 172.21.15.96:0/1397626135 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8011760 con 0x55b419f6eda0 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9ce57fa640 1 -- 172.21.15.96:0/374370474 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9ce00e58a0 con 0x7f9ce8109130 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cddffb640 1 -- 172.21.15.96:0/1397626135 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8011db0 con 0x55b419f6eda0 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cddffb640 1 -- 172.21.15.96:0/1397626135 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800e430 con 0x55b419f6eda0 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cefe80740 1 -- 172.21.15.96:0/1397626135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 msgr2=0x55b419f7a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cdffff640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 0x7f9cd40b77a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0031440 tx=0x7f9cd0005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.993+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8010580 tx=0x7f9cd8010a40 comp rx=0 tx=0).stop 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 -- 172.21.15.96:0/1397626135 shutdown_connections 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f7ad30 0x55b419f7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f7a7f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1397626135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f6e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 -- 172.21.15.96:0/1397626135 >> 172.21.15.96:0/1397626135 conn(0x55b419f6c2b0 msgr2=0x55b419f6d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 -- 172.21.15.96:0/1397626135 shutdown_connections 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 -- 172.21.15.96:0/1397626135 wait complete. 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.994+0000 7f9cefe80740 1 -- start start 2024-09-10T23:02:11.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 Processor -- start 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6eda0 0x55b419f77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b419d35700 con 0x55b419f7ad30 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b419e6f110 con 0x55b419f6eda0 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.995+0000 7f9cefe80740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b419f7d9b0 con 0x55b419f6e480 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9cefe80740 10 client.0 _ll_register_callbacks cb 0x55b419e88990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59168/0 (socket says 172.21.15.96:59168) 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1670866917 learned_addr learned my addr 172.21.15.96:0/1670866917 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6eda0 0x55b419f77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce7fff640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6eda0 msgr2=0x55b419f77a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6eda0 0x55b419f77a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1670866917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 msgr2=0x55b419f780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 -- 172.21.15.96:0/1670866917 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b419e9cc40 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9cd801ca20 tx=0x7f9cd801dee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb087b136782066 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.996+0000 7f9cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8011560 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd8011db0 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9cd8011700 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cd8011900 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 10 client.28327 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b419f892a0 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8011db0 con 0x55b419f6e480 2024-09-10T23:02:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9cd802f030 con 0x55b419f6e480 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 10 client.28327 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 10 client.28327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 20 client.28327 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419f8aaf0 0x55b419f8cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 4 client.28327 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.997+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b419ee6da0 con 0x55b419f8aaf0 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.998+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419f8aaf0 0x55b419f8cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.998+0000 7f9cdffff640 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419f8aaf0 0x55b419f8cfb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dbde82686e8721 server_cookie=897fdfeb2e8e5383 in_seq=0 out_seq=0 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.998+0000 7f9cc6ffd640 10 client.28327 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.998+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/1670866917 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b419ee6da0 con 0x55b419f8aaf0 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.998+0000 7f9cc6ffd640 10 client.28327 handle_command_reply: tid=0 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 client.28327 shutdown 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 msgr2=0x55b419f7a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9cd801ca20 tx=0x7f9cd801dee0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 shutdown_connections 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419f8aaf0 0x55b419f8cfb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6eda0 0x55b419f77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 --2- 172.21.15.96:0/1670866917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f6e480 0x55b419f7a620 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 >> 172.21.15.96:0/1670866917 conn(0x55b419f6c2b0 msgr2=0x55b419f71e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cc6ffd640 0 client.28327 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:10.999+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 shutdown_connections 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.002+0000 7f9cefe80740 1 -- 172.21.15.96:0/1670866917 wait complete. 2024-09-10T23:02:11.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.007+0000 7f9cefe80740 20 client.28327 trim_cache size 0 max 16384 2024-09-10T23:02:11.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 Processor -- start 2024-09-10T23:02:11.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 -- start start 2024-09-10T23:02:11.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 0x55b419f6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.044+0000 7f9cefe80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b4850 con 0x55b419f6eda0 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cefe80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b6390 con 0x55b419f6e480 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cefe80740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b4f20 con 0x55b419f7ad30 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 0x55b419f6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 0x55b419f6d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:59182/0 (socket says 172.21.15.96:59182) 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9ce7fff640 1 -- 172.21.15.96:0/2980586561 learned_addr learned my addr 172.21.15.96:0/2980586561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 -- 172.21.15.96:0/2980586561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 msgr2=0x55b419f6d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 0x55b419f6d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 -- 172.21.15.96:0/2980586561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 msgr2=0x55b419f8cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f8cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.045+0000 7f9cdffff640 1 -- 172.21.15.96:0/2980586561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b419f769d0 con 0x55b419f6eda0 2024-09-10T23:02:11.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.047+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f6bbc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009230 tx=0x7f9cd003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b5e4c96ea89f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.047+0000 7f9cdd7fa640 1 -- 172.21.15.96:0/2980586561 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00041c0 con 0x55b419f6eda0 2024-09-10T23:02:11.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.047+0000 7f9cdd7fa640 1 -- 172.21.15.96:0/2980586561 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd0004360 con 0x55b419f6eda0 2024-09-10T23:02:11.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.047+0000 7f9cdd7fa640 1 -- 172.21.15.96:0/2980586561 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd003d690 con 0x55b419f6eda0 2024-09-10T23:02:11.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 -- 172.21.15.96:0/2980586561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 msgr2=0x55b419f6bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f6bbc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009230 tx=0x7f9cd003f040 comp rx=0 tx=0).stop 2024-09-10T23:02:11.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 -- 172.21.15.96:0/2980586561 shutdown_connections 2024-09-10T23:02:11.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f7ad30 0x55b419f6d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f6eda0 0x55b419f6bbc0 secure :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009230 tx=0x7f9cd003f040 comp rx=0 tx=0).stop 2024-09-10T23:02:11.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2980586561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f8cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.049+0000 7f9cefe80740 1 -- 172.21.15.96:0/2980586561 >> 172.21.15.96:0/2980586561 conn(0x55b419f6c2b0 msgr2=0x55b419f6cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.052+0000 7f9cefe80740 1 -- 172.21.15.96:0/2980586561 shutdown_connections 2024-09-10T23:02:11.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.052+0000 7f9cefe80740 1 -- 172.21.15.96:0/2980586561 wait complete. 2024-09-10T23:02:11.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.053+0000 7f9cefe80740 1 -- start start 2024-09-10T23:02:11.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.053+0000 7f9cefe80740 1 Processor -- start 2024-09-10T23:02:11.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f92780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9cefe80740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f92e70 0x55b419f822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57764/0 (socket says 172.21.15.96:57764) 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.054+0000 7f9ce77fe640 1 -- 172.21.15.96:0/2525691066 learned_addr learned my addr 172.21.15.96:0/2525691066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f92780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9ce7fff640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f92e70 0x55b419f822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b5cc0 con 0x55b419f7ad30 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b55f0 con 0x55b419f6e480 2024-09-10T23:02:11.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b2830 con 0x55b419f92e70 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.055+0000 7f9cefe80740 10 client.0 _ll_register_callbacks cb 0x7f9cd80103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.056+0000 7f9ce77fe640 1 -- 172.21.15.96:0/2525691066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f92e70 msgr2=0x55b419f822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.056+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f92e70 0x55b419f822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.056+0000 7f9ce77fe640 1 -- 172.21.15.96:0/2525691066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 msgr2=0x55b419f92780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.056+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f92780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.056+0000 7f9ce77fe640 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b419f6ebd0 con 0x55b419f6e480 2024-09-10T23:02:11.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.057+0000 7f9ce77fe640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005d00 tx=0x7f9cd800bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c70c918ef824b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:11.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.067+0000 7f9cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:11.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.067+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd803f070 con 0x55b419f6e480 2024-09-10T23:02:11.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.067+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9cd8011aa0 con 0x55b419f6e480 2024-09-10T23:02:11.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.067+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd0004360 con 0x55b419f6e480 2024-09-10T23:02:11.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.078+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9cd8011760 con 0x55b419f6e480 2024-09-10T23:02:11.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.079+0000 7f9cefe80740 10 client.28256 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:11.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.079+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b419ff8020 con 0x55b419f6e480 2024-09-10T23:02:11.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.083+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd0004360 con 0x55b419f6e480 2024-09-10T23:02:11.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.083+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9cd802f040 con 0x55b419f6e480 2024-09-10T23:02:11.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 10 client.28256 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:11.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 10 client.28256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:11.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 20 client.28256 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:11.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419ff8750 0x55b419ffab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:11.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 4 client.28256 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:02:11.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b419ee2850 con 0x55b419ff8750 2024-09-10T23:02:11.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.084+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419ff8750 0x55b419ffab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:11.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cdffff640 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419ff8750 0x55b419ffab70 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea67293a8c052e1 server_cookie=633bc59a45669a8c in_seq=0 out_seq=0 2024-09-10T23:02:11.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cc6ffd640 10 client.28256 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:11.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cc6ffd640 1 -- 172.21.15.96:0/2525691066 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55b419ee2850 con 0x55b419ff8750 2024-09-10T23:02:11.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cc6ffd640 10 client.28256 handle_command_reply: tid=0 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cefe80740 1 client.28256 shutdown 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 msgr2=0x55b419f83680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.089+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005d00 tx=0x7f9cd800bb50 comp rx=0 tx=0).stop 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 shutdown_connections 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55b419ff8750 0x55b419ffab70 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b419f92e70 0x55b419f822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b419f7ad30 0x55b419f92780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 --2- 172.21.15.96:0/2525691066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b419f6e480 0x55b419f83680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 >> 172.21.15.96:0/2525691066 conn(0x55b419f6c2b0 msgr2=0x55b419f798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cc6ffd640 0 client.28256 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 shutdown_connections 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 1 -- 172.21.15.96:0/2525691066 wait complete. 2024-09-10T23:02:11.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.090+0000 7f9cefe80740 20 client.28256 trim_cache size 0 max 16384 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "2df1ad6c-0476-4e75-81a3-b22370ae9b18": { 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:01:36.980195+0000", 2024-09-10T23:02:11.093 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.095 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.096 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:02:11.097 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.099 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.100 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:02:11.101 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:11.103 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:11.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:02:11.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:02:11.104 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:02:11.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 msgr2=0x7f9cd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 0x7f9cd40b77a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0031440 tx=0x7f9cd0005780 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 msgr2=0x7f9ce81ab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eaa0 tx=0x7f9ce000ef60 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 shutdown_connections 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ce81087b0 0x7f9ce810c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9cd40b52e0 0x7f9cd40b77a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ce8109130 0x7f9ce81ab440 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 --2- 172.21.15.96:0/374370474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ce8102da0 0x7f9ce810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.092+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 >> 172.21.15.96:0/374370474 conn(0x7f9ce80fe850 msgr2=0x7f9ce8101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.093+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 shutdown_connections 2024-09-10T23:02:11.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:11.094+0000 7f9cedaab640 1 -- 172.21.15.96:0/374370474 wait complete. 2024-09-10T23:02:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:11 smithi096 ceph-mon[30869]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-10T23:02:11.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:11 smithi121 ceph-mon[35119]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-10T23:02:11.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:11 smithi117 ceph-mon[35081]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-09-10T23:02:13.159 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:02:13.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:02:13.513 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:13.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:13 smithi096 ceph-mon[30869]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-10T23:02:13.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:13 smithi121 ceph-mon[35119]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-10T23:02:13.775 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:13 smithi117 ceph-mon[35081]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-10T23:02:14.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 -- 172.21.15.96:0/4107662272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc107e90 msgr2=0x7faedc10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 --2- 172.21.15.96:0/4107662272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc107e90 0x7faedc10e690 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7faed4009be0 tx=0x7faed402a6c0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 -- 172.21.15.96:0/4107662272 shutdown_connections 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 --2- 172.21.15.96:0/4107662272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc107e90 0x7faedc10e690 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 --2- 172.21.15.96:0/4107662272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107510 0x7faedc107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 --2- 172.21.15.96:0/4107662272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc101b00 0x7faedc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 -- 172.21.15.96:0/4107662272 >> 172.21.15.96:0/4107662272 conn(0x7faedc0fd5b0 msgr2=0x7faedc0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 -- 172.21.15.96:0/4107662272 shutdown_connections 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.100+0000 7faee16b3640 1 -- 172.21.15.96:0/4107662272 wait complete. 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.101+0000 7faee16b3640 1 Processor -- start 2024-09-10T23:02:14.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.101+0000 7faee16b3640 1 -- start start 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc101b00 0x7faedc199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 0x7faedc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faedc112c50 con 0x7faedc101b00 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faedc112ad0 con 0x7faedc107510 2024-09-10T23:02:14.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee16b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faedc112dd0 con 0x7faedc107e90 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:57788/0 (socket says 172.21.15.96:57788) 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 -- 172.21.15.96:0/2510112861 learned_addr learned my addr 172.21.15.96:0/2510112861 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee0eb2640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 0x7faedc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 -- 172.21.15.96:0/2510112861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 msgr2=0x7faedc19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 0x7faedc19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 -- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc101b00 msgr2=0x7faedc199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc101b00 0x7faedc199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faedb7fe640 1 -- 172.21.15.96:0/2510112861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc19f130 con 0x7faedc107510 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.102+0000 7faee0eb2640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 0x7faedc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:14.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.103+0000 7faedb7fe640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faed000e9f0 tx=0x7faed000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.103+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed000cd80 con 0x7faedc107510 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.103+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed00054d0 con 0x7faedc107510 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.103+0000 7faee16b3640 1 -- 172.21.15.96:0/2510112861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc19f420 con 0x7faedc107510 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.104+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0010690 con 0x7faedc107510 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.104+0000 7faee16b3640 1 -- 172.21.15.96:0/2510112861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1a5ae0 con 0x7faedc107510 2024-09-10T23:02:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.105+0000 7faee16b3640 1 -- 172.21.15.96:0/2510112861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc101ee0 con 0x7faedc107510 2024-09-10T23:02:14.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.106+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7faed0004e90 con 0x7faedc107510 2024-09-10T23:02:14.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.106+0000 7faed97fa640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 0x7faec40b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:14.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.107+0000 7faedbfff640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 0x7faec40b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:14.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.107+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faed00e5f40 con 0x7faedc107510 2024-09-10T23:02:14.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.107+0000 7faedbfff640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 0x7faec40b79a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faecc0058c0 tx=0x7faecc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:14.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.111+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faed00ace00 con 0x7faedc107510 2024-09-10T23:02:14.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.439+0000 7faee16b3640 1 -- 172.21.15.96:0/2510112861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faedc19a1b0 con 0x7faedc107510 2024-09-10T23:02:14.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.441+0000 7faed97fa640 1 -- 172.21.15.96:0/2510112861 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7faedc19a1b0 con 0x7faedc107510 2024-09-10T23:02:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:14.442 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:02:14.443 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:02:14.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.445+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 msgr2=0x7faec40b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:14.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.445+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 0x7faec40b79a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faecc0058c0 tx=0x7faecc005850 comp rx=0 tx=0).stop 2024-09-10T23:02:14.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.445+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 msgr2=0x7faedc1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:14.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.445+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faed000e9f0 tx=0x7faed000edf0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 shutdown_connections 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faedc107510 0x7faedc1995c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faec40b54e0 0x7faec40b79a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faedc107e90 0x7faedc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 --2- 172.21.15.96:0/2510112861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faedc101b00 0x7faedc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 >> 172.21.15.96:0/2510112861 conn(0x7faedc0fd5b0 msgr2=0x7faedc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 shutdown_connections 2024-09-10T23:02:14.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:14.446+0000 7faec2ffd640 1 -- 172.21.15.96:0/2510112861 wait complete. 2024-09-10T23:02:14.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:14 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2510112861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:14.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:14 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2510112861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:14.775 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:14 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2510112861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:02:15.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T23:02:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:15 smithi096 ceph-mon[30869]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 604 op/s 2024-09-10T23:02:15.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:15 smithi121 ceph-mon[35119]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 604 op/s 2024-09-10T23:02:15.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:15 smithi117 ceph-mon[35081]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 604 op/s 2024-09-10T23:02:15.953 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 880.0 s, 1697.8 tps, lat 57.154 ms stddev 68.447 2024-09-10T23:02:15.977 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d1d9b640 1 -- 172.21.15.96:0/79926898 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c402f310 con 0x7ff3d4101b30 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 -- 172.21.15.96:0/79926898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4101b30 msgr2=0x7ff3d4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 --2- 172.21.15.96:0/79926898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4101b30 0x7ff3d4101f10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009ca0 tx=0x7ff3c402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 -- 172.21.15.96:0/79926898 shutdown_connections 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 --2- 172.21.15.96:0/79926898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4102450 0x7ff3d4106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 --2- 172.21.15.96:0/79926898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 --2- 172.21.15.96:0/79926898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4101b30 0x7ff3d4101f10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 -- 172.21.15.96:0/79926898 >> 172.21.15.96:0/79926898 conn(0x7ff3d40fd7e0 msgr2=0x7ff3d40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 -- 172.21.15.96:0/79926898 shutdown_connections 2024-09-10T23:02:16.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 -- 172.21.15.96:0/79926898 wait complete. 2024-09-10T23:02:16.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.496+0000 7ff3d90be640 1 Processor -- start 2024-09-10T23:02:16.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 -- start start 2024-09-10T23:02:16.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4101b30 0x7ff3d419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4119430 con 0x7ff3d4106f70 2024-09-10T23:02:16.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41192b0 con 0x7ff3d4101b30 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d90be640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41195b0 con 0x7ff3d4102450 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51198/0 (socket says 172.21.15.96:51198) 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d259c640 1 -- 172.21.15.96:0/4207535376 learned_addr learned my addr 172.21.15.96:0/4207535376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4101b30 0x7ff3d419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.497+0000 7ff3d359e640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 -- 172.21.15.96:0/4207535376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4101b30 msgr2=0x7ff3d419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4101b30 0x7ff3d419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 -- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 msgr2=0x7ff3d41a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 -- 172.21.15.96:0/4207535376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d41a59f0 con 0x7ff3d4102450 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d359e640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:16.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d259c640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d740 tx=0x7ff3c800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3bbfff640 1 -- 172.21.15.96:0/4207535376 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8017020 con 0x7ff3d4102450 2024-09-10T23:02:16.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3bbfff640 1 -- 172.21.15.96:0/4207535376 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3c800de50 con 0x7ff3d4102450 2024-09-10T23:02:16.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3bbfff640 1 -- 172.21.15.96:0/4207535376 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800f620 con 0x7ff3d4102450 2024-09-10T23:02:16.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d41a5ce0 con 0x7ff3d4102450 2024-09-10T23:02:16.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.498+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d41ac3a0 con 0x7ff3d4102450 2024-09-10T23:02:16.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3bbfff640 1 -- 172.21.15.96:0/4207535376 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff3c800f7c0 con 0x7ff3d4102450 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 Processor -- start 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 -- start start 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe5df0 0x557b48ff1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48ff1d80 0x557b48ff4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b48f5bea0 con 0x557b48fe54d0 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b48f4b2d0 con 0x557b48ff1d80 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3db493740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557b48f4f0e0 con 0x557b48fe5df0 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3bbfff640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 0x7ff3bc0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3d359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3d359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36452/0 (socket says 172.21.15.96:36452) 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3d359e640 1 -- 172.21.15.96:0/1418661143 learned_addr learned my addr 172.21.15.96:0/1418661143 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.503+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe5df0 0x557b48ff1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d259c640 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48ff1d80 0x557b48ff4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 0x7ff3bc0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3bbfff640 1 -- 172.21.15.96:0/4207535376 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff3c80e5c40 con 0x7ff3d4102450 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 -- 172.21.15.96:0/1418661143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe5df0 msgr2=0x557b48ff1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe5df0 0x557b48ff1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 -- 172.21.15.96:0/1418661143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48ff1d80 msgr2=0x557b48ff4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48ff1d80 0x557b48ff4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 -- 172.21.15.96:0/1418661143 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b48ca9650 con 0x557b48fe54d0 2024-09-10T23:02:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3d359e640 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc013850 tx=0x7ff3cc013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0d06aebc31420d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3b97fa640 1 -- 172.21.15.96:0/1418661143 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc023030 con 0x557b48fe54d0 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3b97fa640 1 -- 172.21.15.96:0/1418661143 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc009c20 con 0x557b48fe54d0 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.504+0000 7ff3b97fa640 1 -- 172.21.15.96:0/1418661143 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc01f480 con 0x557b48fe54d0 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- 172.21.15.96:0/1418661143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 msgr2=0x557b48fe58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3d359e640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 0x7ff3bc0b7970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc023000 tx=0x7ff3cc022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc013850 tx=0x7ff3cc013d10 comp rx=0 tx=0).stop 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- 172.21.15.96:0/1418661143 shutdown_connections 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48ff1d80 0x557b48ff4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe5df0 0x557b48ff1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 --2- 172.21.15.96:0/1418661143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48fe54d0 0x557b48fe58b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- 172.21.15.96:0/1418661143 >> 172.21.15.96:0/1418661143 conn(0x557b48fe3300 msgr2=0x557b48fe44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- 172.21.15.96:0/1418661143 shutdown_connections 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- 172.21.15.96:0/1418661143 wait complete. 2024-09-10T23:02:16.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.505+0000 7ff3db493740 1 -- start start 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 Processor -- start 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ff16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48feeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b48dac700 con 0x557b48ff1d80 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b48ee6160 con 0x557b48fe5df0 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557b48ff4a00 con 0x557b48fe54d0 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3db493740 10 client.0 _ll_register_callbacks cb 0x557b48eff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36458/0 (socket says 172.21.15.96:36458) 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 -- 172.21.15.96:0/3764220699 learned_addr learned my addr 172.21.15.96:0/3764220699 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ff16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d259c640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48feeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 -- 172.21.15.96:0/3764220699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 msgr2=0x557b48ff16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ff16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 -- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 msgr2=0x557b48feeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48feeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d359e640 1 -- 172.21.15.96:0/3764220699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b48f13c90 con 0x557b48ff1d80 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d259c640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48feeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.506+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ff16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.507+0000 7ff3d359e640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc003f90 tx=0x7ff3cc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd79510059877b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.507+0000 7ff3b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.507+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00fd10 con 0x557b48ff1d80 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.507+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc009c20 con 0x557b48ff1d80 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.507+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff3cc01f7d0 con 0x557b48ff1d80 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3cc01f480 con 0x557b48ff1d80 2024-09-10T23:02:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 10 client.18660 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b49000370 con 0x557b48ff1d80 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc028470 con 0x557b48ff1d80 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3cc030070 con 0x557b48ff1d80 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 10 client.18660 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 10 client.18660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 20 client.18660 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b49001aa0 0x557b49003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 4 client.18660 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.508+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff3cc009c20 con 0x557b49001aa0 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.509+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b49001aa0 0x557b49003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.509+0000 7ff3b6ff5640 10 client.18660 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:02:16.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.509+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b49001aa0 0x557b49003ec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2785584928b98bf0 server_cookie=4c598e6865c3f89f in_seq=0 out_seq=0 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/3764220699 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff3cc009c20 con 0x557b49001aa0 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3b6ff5640 10 client.18660 handle_command_reply: tid=0 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 client.18660 shutdown 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 msgr2=0x557b48fef180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc003f90 tx=0x7ff3cc005710 comp rx=0 tx=0).stop 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 shutdown_connections 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b49001aa0 0x557b49003ec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48feeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fef180 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 --2- 172.21.15.96:0/3764220699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ff16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 >> 172.21.15.96:0/3764220699 conn(0x557b48fe3300 msgr2=0x557b48fe8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3b6ff5640 0 client.18660 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:02:16.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 shutdown_connections 2024-09-10T23:02:16.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 1 -- 172.21.15.96:0/3764220699 wait complete. 2024-09-10T23:02:16.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.510+0000 7ff3db493740 20 client.18660 trim_cache size 0 max 16384 2024-09-10T23:02:16.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.529+0000 7ff3db493740 1 Processor -- start 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 -- start start 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48fedff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fe9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b4840 con 0x557b48ff1d80 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b6380 con 0x557b48fe5df0 2024-09-10T23:02:16.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51230/0 (socket says 172.21.15.96:51230) 2024-09-10T23:02:16.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3d2d9d640 1 -- 172.21.15.96:0/29730469 learned_addr learned my addr 172.21.15.96:0/29730469 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:16.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.533+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b4f10 con 0x557b48fe54d0 2024-09-10T23:02:16.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.535+0000 7ff3d359e640 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fe9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.536+0000 7ff3d359e640 1 -- 172.21.15.96:0/29730469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 msgr2=0x557b48ffdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.536+0000 7ff3d359e640 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.536+0000 7ff3d359e640 1 -- 172.21.15.96:0/29730469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 msgr2=0x557b48fedff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.536+0000 7ff3d359e640 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48fedff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.536+0000 7ff3d359e640 1 -- 172.21.15.96:0/29730469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b48fe5c20 con 0x557b48ff1d80 2024-09-10T23:02:16.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.537+0000 7ff3d359e640 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fe9d30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00f730 tx=0x7ff3cc013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bede973693bbabe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.538+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.538+0000 7ff3b8ff9640 1 -- 172.21.15.96:0/29730469 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc005b90 con 0x557b48ff1d80 2024-09-10T23:02:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.538+0000 7ff3b8ff9640 1 -- 172.21.15.96:0/29730469 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc0096c0 con 0x557b48ff1d80 2024-09-10T23:02:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.538+0000 7ff3b8ff9640 1 -- 172.21.15.96:0/29730469 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc028470 con 0x557b48ff1d80 2024-09-10T23:02:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.539+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 msgr2=0x557b48fe9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.539+0000 7ff3db493740 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fe9d30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00f730 tx=0x7ff3cc013d10 comp rx=0 tx=0).stop 2024-09-10T23:02:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.543+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 shutdown_connections 2024-09-10T23:02:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.543+0000 7ff3db493740 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48fedff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.543+0000 7ff3db493740 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48fe9d30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.543+0000 7ff3db493740 1 --2- 172.21.15.96:0/29730469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b48ffdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.543+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 >> 172.21.15.96:0/29730469 conn(0x557b48fe3300 msgr2=0x557b48fe43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.552+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 shutdown_connections 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.553+0000 7ff3db493740 1 -- 172.21.15.96:0/29730469 wait complete. 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.553+0000 7ff3db493740 1 -- start start 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.553+0000 7ff3db493740 1 Processor -- start 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.554+0000 7ff3db493740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.554+0000 7ff3db493740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48ff9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.554+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.554+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51242/0 (socket says 172.21.15.96:51242) 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.555+0000 7ff3d2d9d640 1 -- 172.21.15.96:0/1685528273 learned_addr learned my addr 172.21.15.96:0/1685528273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.555+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48ffa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.555+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b5cb0 con 0x557b48ff1d80 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.555+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b55e0 con 0x557b48fe5df0 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.555+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41b2820 con 0x557b48fe54d0 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3db493740 10 client.0 _ll_register_callbacks cb 0x7ff3c8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3d2d9d640 1 -- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 msgr2=0x557b48ff9c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48ff9c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3d2d9d640 1 -- 172.21.15.96:0/1685528273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 msgr2=0x557b48ffa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48ffa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.556+0000 7ff3d2d9d640 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b48ffa7b0 con 0x557b48fe54d0 2024-09-10T23:02:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.557+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc0096c0 con 0x557b48fe54d0 2024-09-10T23:02:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.557+0000 7ff3d2d9d640 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4006010 tx=0x7ff3c40315c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fbf46ec0e95e815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.557+0000 7ff3b6ff5640 10 client.28354 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.557+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/1685528273 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4003970 con 0x557b48fe54d0 2024-09-10T23:02:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.558+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/1685528273 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff3c4003b10 con 0x557b48fe54d0 2024-09-10T23:02:16.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.558+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/1685528273 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3c4003d50 con 0x557b48fe54d0 2024-09-10T23:02:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.561+0000 7ff3db493740 10 client.28354 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.561+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x557b49009ff0 con 0x557b48fe54d0 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/1685528273 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff3c4056030 con 0x557b48fe54d0 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3db493740 10 client.28354 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3db493740 10 client.28354 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3db493740 20 client.28354 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b4907fd90 0x557b490821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.563+0000 7ff3db493740 4 client.28354 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.564+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff3cc0096c0 con 0x557b4907fd90 2024-09-10T23:02:16.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.564+0000 7ff3d259c640 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b4907fd90 0x557b490821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:16.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.565+0000 7ff3d259c640 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b4907fd90 0x557b490821b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a3d633fbf892f8a server_cookie=db6b80336fb0daf2 in_seq=0 out_seq=0 2024-09-10T23:02:16.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.565+0000 7ff3b6ff5640 10 client.28354 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:02:16.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.565+0000 7ff3b6ff5640 1 -- 172.21.15.96:0/1685528273 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff3cc0096c0 con 0x557b4907fd90 2024-09-10T23:02:16.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.565+0000 7ff3b6ff5640 10 client.28354 handle_command_reply: tid=0 2024-09-10T23:02:16.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.566+0000 7ff3db493740 1 client.28354 shutdown 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.570+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 msgr2=0x557b4906bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.570+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4006010 tx=0x7ff3c40315c0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.570+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 shutdown_connections 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.570+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x557b4907fd90 0x557b490821b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.571+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b48fe5df0 0x557b48ff9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.571+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b48ff1d80 0x557b48ffa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.571+0000 7ff3db493740 1 --2- 172.21.15.96:0/1685528273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b48fe54d0 0x557b4906bf90 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4006010 tx=0x7ff3c40315c0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.571+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 >> 172.21.15.96:0/1685528273 conn(0x557b48fe3300 msgr2=0x557b48fe9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.571+0000 7ff3b6ff5640 0 client.28354 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:02:16.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.572+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 shutdown_connections 2024-09-10T23:02:16.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.575+0000 7ff3db493740 1 -- 172.21.15.96:0/1685528273 wait complete. 2024-09-10T23:02:16.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.576+0000 7ff3db493740 20 client.28354 trim_cache size 0 max 16384 2024-09-10T23:02:16.577 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:02:16.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.577+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 msgr2=0x7ff3bc0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.577+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 0x7ff3bc0b7970 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc023000 tx=0x7ff3cc022000 comp rx=0 tx=0).stop 2024-09-10T23:02:16.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.577+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 msgr2=0x7ff3d419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:16.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.577+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d740 tx=0x7ff3c800dc00 comp rx=0 tx=0).stop 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 shutdown_connections 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff3bc0b54b0 0x7ff3bc0b7970 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3d4106f70 0x7ff3d41a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3d4102450 0x7ff3d419fe80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 --2- 172.21.15.96:0/4207535376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3d4101b30 0x7ff3d419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:16.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.579+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 >> 172.21.15.96:0/4207535376 conn(0x7ff3d40fd7e0 msgr2=0x7ff3d41048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:16.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.581+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 shutdown_connections 2024-09-10T23:02:16.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:16.582+0000 7ff3d90be640 1 -- 172.21.15.96:0/4207535376 wait complete. 2024-09-10T23:02:17.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T23:02:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:17 smithi096 ceph-mon[30869]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-09-10T23:02:17.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:17 smithi121 ceph-mon[35119]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-09-10T23:02:17.768 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:17.775 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:17 smithi117 ceph-mon[35081]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.362+0000 7faddaffd640 1 -- 172.21.15.96:0/4004649161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0318a0 con 0x7faddc101cc0 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 -- 172.21.15.96:0/4004649161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc101cc0 msgr2=0x7faddc1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 --2- 172.21.15.96:0/4004649161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc101cc0 0x7faddc1020a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009900 tx=0x7fadcc02a410 comp rx=0 tx=0).stop 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 -- 172.21.15.96:0/4004649161 shutdown_connections 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 --2- 172.21.15.96:0/4004649161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc108050 0x7faddc10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 --2- 172.21.15.96:0/4004649161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc1076d0 0x7faddc107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 --2- 172.21.15.96:0/4004649161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc101cc0 0x7faddc1020a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 -- 172.21.15.96:0/4004649161 >> 172.21.15.96:0/4004649161 conn(0x7faddc0fd770 msgr2=0x7faddc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 -- 172.21.15.96:0/4004649161 shutdown_connections 2024-09-10T23:02:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.363+0000 7fade28a9640 1 -- 172.21.15.96:0/4004649161 wait complete. 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.364+0000 7fade28a9640 1 Processor -- start 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.364+0000 7fade28a9640 1 -- start start 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.364+0000 7fade28a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.364+0000 7fade28a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1076d0 0x7faddc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.364+0000 7fade28a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 0x7faddc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7fade28a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faddc111d10 con 0x7faddc108050 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7fade28a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc111b90 con 0x7faddc1076d0 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7fade28a9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faddc111e90 con 0x7faddc101cc0 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1076d0 0x7faddc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51254/0 (socket says 172.21.15.96:51254) 2024-09-10T23:02:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 -- 172.21.15.96:0/526985641 learned_addr learned my addr 172.21.15.96:0/526985641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7fade0e46640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 0x7faddc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 -- 172.21.15.96:0/526985641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1076d0 msgr2=0x7faddc199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1076d0 0x7faddc199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 -- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 msgr2=0x7faddc19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 0x7faddc19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7faddbfff640 1 -- 172.21.15.96:0/526985641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc19f370 con 0x7faddc101cc0 2024-09-10T23:02:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.365+0000 7fade0e46640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 0x7faddc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7faddbfff640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005940 tx=0x7fadcc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7fadd97fa640 1 -- 172.21.15.96:0/526985641 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc03f030 con 0x7faddc101cc0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7fadd97fa640 1 -- 172.21.15.96:0/526985641 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadcc003aa0 con 0x7faddc101cc0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7fadd97fa640 1 -- 172.21.15.96:0/526985641 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc035070 con 0x7faddc101cc0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc19f600 con 0x7faddc101cc0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.366+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc19f9d0 con 0x7faddc101cc0 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 Processor -- start 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 -- start start 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 0x55ee387c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 0x55ee387c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fadd97fa640 1 -- 172.21.15.96:0/526985641 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fadcc03bcc0 con 0x7faddc101cc0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee3872cea0 con 0x55ee387b60f0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ee3871c2d0 con 0x55ee387c29a0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade4c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ee387200e0 con 0x55ee387b6a10 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36510/0 (socket says 172.21.15.96:36510) 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7fade0e46640 1 -- 172.21.15.96:0/3564839072 learned_addr learned my addr 172.21.15.96:0/3564839072 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.369+0000 7faddb7fe640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 0x55ee387c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7faddbfff640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 0x55ee387c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 -- 172.21.15.96:0/3564839072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 msgr2=0x55ee387c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 0x55ee387c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 -- 172.21.15.96:0/3564839072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 msgr2=0x55ee387c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 0x55ee387c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 -- 172.21.15.96:0/3564839072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3847a650 con 0x55ee387b60f0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7faddb7fe640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 0x55ee387c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7faddbfff640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 0x55ee387c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fade0e46640 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fadd40117e0 tx=0x7fadd4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502664e8d0bdbf2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fadd97fa640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 0x7fadc40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fadc27fc640 1 -- 172.21.15.96:0/3564839072 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4011310 con 0x55ee387b60f0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fadc27fc640 1 -- 172.21.15.96:0/3564839072 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4003e20 con 0x55ee387b60f0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fadc27fc640 1 -- 172.21.15.96:0/3564839072 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40044f0 con 0x55ee387b60f0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7faddbfff640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 0x7fadc40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.370+0000 7fadd97fa640 1 -- 172.21.15.96:0/526985641 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fadcc106390 con 0x7faddc101cc0 2024-09-10T23:02:18.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- 172.21.15.96:0/3564839072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 msgr2=0x55ee387b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7faddbfff640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 0x7fadc40b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fadcc103f30 tx=0x7fadcc02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fadd40117e0 tx=0x7fadd4011ca0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- 172.21.15.96:0/3564839072 shutdown_connections 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387c29a0 0x55ee387c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b6a10 0x55ee387c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 --2- 172.21.15.96:0/3564839072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387b64d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- 172.21.15.96:0/3564839072 >> 172.21.15.96:0/3564839072 conn(0x55ee387b3f20 msgr2=0x55ee387b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- 172.21.15.96:0/3564839072 shutdown_connections 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- 172.21.15.96:0/3564839072 wait complete. 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.371+0000 7fade4c7e740 1 -- start start 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 Processor -- start 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee3857d700 con 0x55ee387b60f0 2024-09-10T23:02:18.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ee386b7160 con 0x55ee387b6a10 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ee387c5620 con 0x55ee387c29a0 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36706/0 (socket says 172.21.15.96:36706) 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 -- 172.21.15.96:0/2802868356 learned_addr learned my addr 172.21.15.96:0/2802868356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade4c7e740 10 client.0 _ll_register_callbacks cb 0x55ee386d09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade0e46640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade0e46640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 -- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 msgr2=0x55ee387bfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 -- 172.21.15.96:0/2802868356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 msgr2=0x55ee387c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7faddb7fe640 1 -- 172.21.15.96:0/2802868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee386e4c90 con 0x55ee387b6a10 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.372+0000 7fade0e46640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:18.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fadd000b9c0 tx=0x7fadd000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a9e61a73dfe149 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fadbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003ea0 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd4003e20 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fadd0011440 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fadd001b640 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fade4c7e740 10 client.28286 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.373+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee387d0f30 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.375+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4003e20 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.375+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fadd0019020 con 0x55ee387b6a10 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.375+0000 7fade4c7e740 10 client.28286 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.375+0000 7fade4c7e740 10 client.28286 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade4c7e740 20 client.28286 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:18.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee387d2710 0x55ee387d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade4c7e740 4 client.28286 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:18.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee3872edf0 con 0x55ee387d2710 2024-09-10T23:02:18.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade0e46640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee387d2710 0x55ee387d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fadbb7fe640 10 client.28286 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.376+0000 7fade0e46640 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee387d2710 0x55ee387d4bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd51c387b0b43ec3 server_cookie=6c1233ea0dd576bd in_seq=0 out_seq=0 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2802868356 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee3872edf0 con 0x55ee387d2710 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fadbb7fe640 10 client.28286 handle_command_reply: tid=0 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 client.28286 shutdown 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 msgr2=0x55ee387bf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fadd000b9c0 tx=0x7fadd000be80 comp rx=0 tx=0).stop 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 shutdown_connections 2024-09-10T23:02:18.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee387d2710 0x55ee387d4bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387bf6b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.377+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2802868356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.378+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 >> 172.21.15.96:0/2802868356 conn(0x55ee387b3f20 msgr2=0x55ee387b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.378+0000 7fadbb7fe640 0 client.28286 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.378+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 shutdown_connections 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.378+0000 7fade4c7e740 1 -- 172.21.15.96:0/2802868356 wait complete. 2024-09-10T23:02:18.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.378+0000 7fade4c7e740 20 client.28286 trim_cache size 0 max 16384 2024-09-10T23:02:18.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 Processor -- start 2024-09-10T23:02:18.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 -- start start 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b60f0 0x55ee387b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ae290 con 0x55ee387c29a0 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1afdd0 con 0x55ee387b6a10 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.396+0000 7fade4c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ae960 con 0x55ee387b60f0 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36714/0 (socket says 172.21.15.96:36714) 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36526/0 (socket says 172.21.15.96:36526) 2024-09-10T23:02:18.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 -- 172.21.15.96:0/2016115916 learned_addr learned my addr 172.21.15.96:0/2016115916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fade0e46640 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b60f0 0x55ee387b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 -- 172.21.15.96:0/2016115916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b60f0 msgr2=0x55ee387b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b60f0 0x55ee387b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 -- 172.21.15.96:0/2016115916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 msgr2=0x55ee387c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 -- 172.21.15.96:0/2016115916 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee387b6840 con 0x55ee387b6a10 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fade0e46640 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000c00 tx=0x7fadd0003b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353d5b31c967ac5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fadc1ffb640 1 -- 172.21.15.96:0/2016115916 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd002a070 con 0x55ee387b6a10 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fadc1ffb640 1 -- 172.21.15.96:0/2016115916 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0005520 con 0x55ee387b6a10 2024-09-10T23:02:18.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.397+0000 7fadc1ffb640 1 -- 172.21.15.96:0/2016115916 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd001bda0 con 0x55ee387b6a10 2024-09-10T23:02:18.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- 172.21.15.96:0/2016115916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 msgr2=0x55ee387c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000c00 tx=0x7fadd0003b00 comp rx=0 tx=0).stop 2024-09-10T23:02:18.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- 172.21.15.96:0/2016115916 shutdown_connections 2024-09-10T23:02:18.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387c3d90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387c29a0 0x55ee387c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2016115916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387b60f0 0x55ee387b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- 172.21.15.96:0/2016115916 >> 172.21.15.96:0/2016115916 conn(0x55ee387b3f20 msgr2=0x55ee387bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- 172.21.15.96:0/2016115916 shutdown_connections 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- 172.21.15.96:0/2016115916 wait complete. 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.398+0000 7fade4c7e740 1 -- start start 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 Processor -- start 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade0e46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36530/0 (socket says 172.21.15.96:36530) 2024-09-10T23:02:18.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade0e46640 1 -- 172.21.15.96:0/2056504892 learned_addr learned my addr 172.21.15.96:0/2056504892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1af700 con 0x55ee387b60f0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1af030 con 0x55ee387b6a10 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ac270 con 0x55ee387c29a0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.399+0000 7fade4c7e740 10 client.0 _ll_register_callbacks cb 0x7fadd0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 -- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 msgr2=0x55ee387bb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 -- 172.21.15.96:0/2056504892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 msgr2=0x55ee387cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee387bbf10 con 0x55ee387b60f0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade0e46640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fadd4014040 tx=0x7fadd4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c237db0cb9446ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:18.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fadbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4023040 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fadd4011450 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd0005520 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fadd400f9a0 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade4c7e740 10 client.18681 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee387bd210 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.400+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40051c0 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fadd4026030 con 0x55ee387b60f0 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 10 client.18681 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 10 client.18681 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 20 client.18681 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee388406d0 0x55ee38842ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 4 client.18681 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:18.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fadd0005520 con 0x55ee388406d0 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee388406d0 0x55ee38842ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7fadbb7fe640 10 client.18681 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.401+0000 7faddb7fe640 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee388406d0 0x55ee38842ac0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c507d873f80ac4e server_cookie=b609245acc41db38 in_seq=0 out_seq=0 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fadbb7fe640 1 -- 172.21.15.96:0/2056504892 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fadd0005520 con 0x55ee388406d0 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fadbb7fe640 10 client.18681 handle_command_reply: tid=0 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 client.18681 shutdown 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 msgr2=0x55ee387caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fadd4014040 tx=0x7fadd4003ed0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 shutdown_connections 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x55ee388406d0 0x55ee38842ac0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ee387b6a10 0x55ee387cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee387c29a0 0x55ee387bb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 --2- 172.21.15.96:0/2056504892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee387b60f0 0x55ee387caac0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 >> 172.21.15.96:0/2056504892 conn(0x55ee387b3f20 msgr2=0x55ee387c0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.402+0000 7fadbb7fe640 0 client.18681 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.403+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 shutdown_connections 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.403+0000 7fade4c7e740 1 -- 172.21.15.96:0/2056504892 wait complete. 2024-09-10T23:02:18.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.404+0000 7fade4c7e740 20 client.18681 trim_cache size 0 max 16384 2024-09-10T23:02:18.411 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.410+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 msgr2=0x7fadc40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.410+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 0x7fadc40b7820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fadcc103f30 tx=0x7fadcc02a410 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.410+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 msgr2=0x7faddc1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.410+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005940 tx=0x7fadcc004270 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 shutdown_connections 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1076d0 0x7faddc199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fadc40b5360 0x7fadc40b7820 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faddc108050 0x7faddc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 --2- 172.21.15.96:0/526985641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faddc101cc0 0x7faddc1992c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:18.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 >> 172.21.15.96:0/526985641 conn(0x7faddc0fd770 msgr2=0x7faddc06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:18.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 shutdown_connections 2024-09-10T23:02:18.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:18.411+0000 7fade28a9640 1 -- 172.21.15.96:0/526985641 wait complete. 2024-09-10T23:02:19.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T23:02:19.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:19 smithi096 ceph-mon[30869]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-09-10T23:02:19.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:19 smithi121 ceph-mon[35119]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-09-10T23:02:19.763 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:19.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:19 smithi117 ceph-mon[35081]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-09-10T23:02:20.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.322+0000 7efef6717640 1 -- 172.21.15.96:0/2784638652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 msgr2=0x7efef01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.322+0000 7efef6717640 1 --2- 172.21.15.96:0/2784638652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef01111b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efee8009be0 tx=0x7efee802a710 comp rx=0 tx=0).stop 2024-09-10T23:02:20.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.326+0000 7efef6717640 1 -- 172.21.15.96:0/2784638652 shutdown_connections 2024-09-10T23:02:20.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.326+0000 7efef6717640 1 --2- 172.21.15.96:0/2784638652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 0x7efef011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.326+0000 7efef6717640 1 --2- 172.21.15.96:0/2784638652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.326+0000 7efef6717640 1 --2- 172.21.15.96:0/2784638652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef01111b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.326+0000 7efef6717640 1 -- 172.21.15.96:0/2784638652 >> 172.21.15.96:0/2784638652 conn(0x7efef006cb30 msgr2=0x7efef006cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.330+0000 7efef6717640 1 -- 172.21.15.96:0/2784638652 shutdown_connections 2024-09-10T23:02:20.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.330+0000 7efef6717640 1 -- 172.21.15.96:0/2784638652 wait complete. 2024-09-10T23:02:20.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 Processor -- start 2024-09-10T23:02:20.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 -- start start 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef0117830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 0x7efef01a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef0114110 con 0x7efef010a530 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efef0113f90 con 0x7efef011c0a0 2024-09-10T23:02:20.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.331+0000 7efef6717640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efef0114290 con 0x7efef010a150 2024-09-10T23:02:20.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.337+0000 7efef4f14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.337+0000 7efef4f14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36548/0 (socket says 172.21.15.96:36548) 2024-09-10T23:02:20.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.337+0000 7efef4f14640 1 -- 172.21.15.96:0/1710171001 learned_addr learned my addr 172.21.15.96:0/1710171001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.337+0000 7efef5715640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef0117830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.340+0000 7efef5f16640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 0x7efef01a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.343+0000 7efef4f14640 1 -- 172.21.15.96:0/1710171001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 msgr2=0x7efef0117830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.343+0000 7efef4f14640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef0117830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.343+0000 7efef4f14640 1 -- 172.21.15.96:0/1710171001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 msgr2=0x7efef01a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.343+0000 7efef4f14640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 0x7efef01a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.343+0000 7efef4f14640 1 -- 172.21.15.96:0/1710171001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef01a5720 con 0x7efef010a530 2024-09-10T23:02:20.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.344+0000 7efef4f14640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efeec01f980 tx=0x7efeec01fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.345+0000 7efee67fc640 1 -- 172.21.15.96:0/1710171001 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec01db90 con 0x7efef010a530 2024-09-10T23:02:20.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.345+0000 7efee67fc640 1 -- 172.21.15.96:0/1710171001 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeec01dd30 con 0x7efef010a530 2024-09-10T23:02:20.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.345+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01a5a10 con 0x7efef010a530 2024-09-10T23:02:20.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.345+0000 7efee67fc640 1 -- 172.21.15.96:0/1710171001 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec003e10 con 0x7efef010a530 2024-09-10T23:02:20.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.345+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef007b3b0 con 0x7efef010a530 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.351+0000 7efef9d4e740 1 Processor -- start 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.351+0000 7efef9d4e740 1 -- start start 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f08674dea0 con 0x55f0867d7bf0 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f08673d2d0 con 0x55f0867d72d0 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef9d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f0867410e0 con 0x55f0867e3b80 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef5f16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef5f16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36750/0 (socket says 172.21.15.96:36750) 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.352+0000 7efef5f16640 1 -- 172.21.15.96:0/2045163430 learned_addr learned my addr 172.21.15.96:0/2045163430 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5715640 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5f16640 1 -- 172.21.15.96:0/2045163430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 msgr2=0x55f0867e5f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5f16640 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e5f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5f16640 1 -- 172.21.15.96:0/2045163430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 msgr2=0x55f0867e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5f16640 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867e3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5f16640 1 -- 172.21.15.96:0/2045163430 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08649b650 con 0x55f0867d72d0 2024-09-10T23:02:20.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.353+0000 7efef5715640 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:20.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.355+0000 7efef5f16640 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efee0009c10 tx=0x7efee00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb04f16597116b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.356+0000 7efee67fc640 1 -- 172.21.15.96:0/1710171001 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7efeec004020 con 0x7efef010a530 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.356+0000 7efedb7fe640 1 -- 172.21.15.96:0/2045163430 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0012450 con 0x55f0867d72d0 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.356+0000 7efedb7fe640 1 -- 172.21.15.96:0/2045163430 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0012aa0 con 0x55f0867d72d0 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.356+0000 7efedb7fe640 1 -- 172.21.15.96:0/2045163430 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee000fb20 con 0x55f0867d72d0 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.357+0000 7efee67fc640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 0x7efedc0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 -- 172.21.15.96:0/2045163430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 msgr2=0x55f0867d76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efee67fc640 1 -- 172.21.15.96:0/1710171001 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efeec0f6c20 con 0x7efef010a530 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efee0009c10 tx=0x7efee00117b0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 -- 172.21.15.96:0/2045163430 shutdown_connections 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef5715640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 0x7efedc0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 --2- 172.21.15.96:0/2045163430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867d76b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 -- 172.21.15.96:0/2045163430 >> 172.21.15.96:0/2045163430 conn(0x55f0867d5100 msgr2=0x55f0867d62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 -- 172.21.15.96:0/2045163430 shutdown_connections 2024-09-10T23:02:20.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.358+0000 7efef9d4e740 1 -- 172.21.15.96:0/2045163430 wait complete. 2024-09-10T23:02:20.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.359+0000 7efef9d4e740 1 -- start start 2024-09-10T23:02:20.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.359+0000 7efef9d4e740 1 Processor -- start 2024-09-10T23:02:20.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.359+0000 7efef5715640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 0x7efedc0b7a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efee8031040 tx=0x7efee8000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 0x55f0867e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f08659e700 con 0x55f0867d72d0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0866d8160 con 0x55f0867d7bf0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f0867e6800 con 0x55f0867e3b80 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef5715640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef9d4e740 10 client.0 _ll_register_callbacks cb 0x55f0866f19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef5715640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36556/0 (socket says 172.21.15.96:36556) 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.360+0000 7efef5715640 1 -- 172.21.15.96:0/4054824456 learned_addr learned my addr 172.21.15.96:0/4054824456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef4f14640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 0x55f0867e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5f16640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 -- 172.21.15.96:0/4054824456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 msgr2=0x55f0867e0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 -- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 msgr2=0x55f0867e0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 0x55f0867e0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 -- 172.21.15.96:0/4054824456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f086705c90 con 0x55f0867d72d0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef4f14640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 0x55f0867e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efef5715640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efee8033000 tx=0x7efee802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203e2cbd23064b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.361+0000 7efed8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8044070 con 0x55f0867d72d0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efee8031850 con 0x55f0867d72d0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee0012aa0 con 0x55f0867d72d0 2024-09-10T23:02:20.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efee8036040 con 0x55f0867d72d0 2024-09-10T23:02:20.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efef9d4e740 10 client.18690 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:20.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0867f2140 con 0x55f0867d72d0 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0012aa0 con 0x55f0867d72d0 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.362+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efee803c040 con 0x55f0867d72d0 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 10 client.18690 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 10 client.18690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 20 client.18690 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f0867f36a0 0x55f0867f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 4 client.18690 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f08674fdf0 con 0x55f0867f36a0 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efef4f14640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f0867f36a0 0x55f0867f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.363+0000 7efed8ff9640 10 client.18690 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:02:20.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.364+0000 7efef4f14640 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f0867f36a0 0x55f0867f5b10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5c5785f77b06156 server_cookie=51113288bbd6105b in_seq=0 out_seq=0 2024-09-10T23:02:20.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.366+0000 7efed8ff9640 1 -- 172.21.15.96:0/4054824456 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f08674fdf0 con 0x55f0867f36a0 2024-09-10T23:02:20.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.366+0000 7efed8ff9640 10 client.18690 handle_command_reply: tid=0 2024-09-10T23:02:20.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.366+0000 7efef9d4e740 1 client.18690 shutdown 2024-09-10T23:02:20.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 msgr2=0x55f0867e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efee8033000 tx=0x7efee802a710 comp rx=0 tx=0).stop 2024-09-10T23:02:20.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 shutdown_connections 2024-09-10T23:02:20.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f0867f36a0 0x55f0867f5b10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efed8ff9640 0 client.18690 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d7bf0 0x55f0867e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867e0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 --2- 172.21.15.96:0/4054824456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d72d0 0x55f0867e34b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.367+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 >> 172.21.15.96:0/4054824456 conn(0x55f0867d5100 msgr2=0x55f0867dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.368+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 shutdown_connections 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.368+0000 7efef9d4e740 1 -- 172.21.15.96:0/4054824456 wait complete. 2024-09-10T23:02:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.368+0000 7efef9d4e740 20 client.18690 trim_cache size 0 max 16384 2024-09-10T23:02:20.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.386+0000 7efef9d4e740 1 Processor -- start 2024-09-10T23:02:20.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.386+0000 7efef9d4e740 1 -- start start 2024-09-10T23:02:20.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.386+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.386+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 0x55f0867d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.386+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867e3b80 0x55f0867df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef9d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efed400e320 con 0x55f0867e3b80 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef5f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867e3b80 0x55f0867df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef5715640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef5715640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867dfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36768/0 (socket says 172.21.15.96:36768) 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efed400fe60 con 0x55f0867d72d0 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 0x55f0867d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efed400e9f0 con 0x55f0867d7bf0 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef5715640 1 -- 172.21.15.96:0/1020780067 learned_addr learned my addr 172.21.15.96:0/1020780067 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 -- 172.21.15.96:0/1020780067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 msgr2=0x55f0867dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 -- 172.21.15.96:0/1020780067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867e3b80 msgr2=0x55f0867df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867e3b80 0x55f0867df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 -- 172.21.15.96:0/1020780067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0867e12f0 con 0x55f0867d7bf0 2024-09-10T23:02:20.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.387+0000 7efef4f14640 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 0x55f0867d4a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec0e0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b3bf5128decdbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efedaffd640 1 -- 172.21.15.96:0/1020780067 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0bdae0 con 0x55f0867d7bf0 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efedaffd640 1 -- 172.21.15.96:0/1020780067 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeec0bd3e0 con 0x55f0867d7bf0 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efedaffd640 1 -- 172.21.15.96:0/1020780067 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0bd640 con 0x55f0867d7bf0 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 msgr2=0x55f0867d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 0x55f0867d4a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec0e0040 comp rx=0 tx=0).stop 2024-09-10T23:02:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 shutdown_connections 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867e3b80 0x55f0867df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867d7bf0 0x55f0867d4a10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 --2- 172.21.15.96:0/1020780067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 >> 172.21.15.96:0/1020780067 conn(0x55f0867d5100 msgr2=0x55f0867d61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 shutdown_connections 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.388+0000 7efef9d4e740 1 -- 172.21.15.96:0/1020780067 wait complete. 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 -- start start 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 Processor -- start 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efed400f790 con 0x55f0867d7bf0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efed400f0c0 con 0x55f0867d72d0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efed400c300 con 0x55f0867e3b80 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef9d4e740 10 client.0 _ll_register_callbacks cb 0x7efeec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.389+0000 7efef5715640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36778/0 (socket says 172.21.15.96:36778) 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 -- 172.21.15.96:0/3945632052 learned_addr learned my addr 172.21.15.96:0/3945632052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef4f14640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5f16640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 -- 172.21.15.96:0/3945632052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 msgr2=0x55f0867dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867dcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 -- 172.21.15.96:0/3945632052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 msgr2=0x55f0867ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 -- 172.21.15.96:0/3945632052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0867ec880 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5f16640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef5715640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efee804b470 tx=0x7efee8031be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbad605f1871d9df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efed8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8063020 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7efee80043c0 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.390+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeec0bd3e0 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efee8044050 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 10 client.28310 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0867de190 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeec0bd3e0 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7efee803c040 con 0x55f0867d72d0 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 10 client.28310 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 10 client.28310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 20 client.28310 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f086861630 0x55f086863a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 4 client.28310 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.391+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f08674b8a0 con 0x55f086861630 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef4f14640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f086861630 0x55f086863a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef4f14640 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f086861630 0x55f086863a20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d05a6ab3fec05a3d server_cookie=c42775c464aeb059 in_seq=0 out_seq=0 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efed8ff9640 10 client.28310 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efed8ff9640 1 -- 172.21.15.96:0/3945632052 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f08674b8a0 con 0x55f086861630 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efed8ff9640 10 client.28310 handle_command_reply: tid=0 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 client.28310 shutdown 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 msgr2=0x55f0867ebaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efee804b470 tx=0x7efee8031be0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 shutdown_connections 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55f086861630 0x55f086863a20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0867e3b80 0x55f0867dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f0867d7bf0 0x55f0867ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 --2- 172.21.15.96:0/3945632052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0867d72d0 0x55f0867ebaf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 >> 172.21.15.96:0/3945632052 conn(0x55f0867d5100 msgr2=0x55f0867df160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.392+0000 7efed8ff9640 0 client.28310 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 shutdown_connections 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef9d4e740 1 -- 172.21.15.96:0/3945632052 wait complete. 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef9d4e740 20 client.28310 trim_cache size 0 max 16384 2024-09-10T23:02:20.400 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:02:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 msgr2=0x7efedc0b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 0x7efedc0b7a70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efee8031040 tx=0x7efee8000f40 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 msgr2=0x7efef0117d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efeec01f980 tx=0x7efeec01fe40 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 shutdown_connections 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efef011c0a0 0x7efef01a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7efedc0b55b0 0x7efedc0b7a70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef010a530 0x7efef0117d70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 --2- 172.21.15.96:0/1710171001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef010a150 0x7efef0117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.393+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 >> 172.21.15.96:0/1710171001 conn(0x7efef006cb30 msgr2=0x7efef00727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.394+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 shutdown_connections 2024-09-10T23:02:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:20.394+0000 7efef6717640 1 -- 172.21.15.96:0/1710171001 wait complete. 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 -- 172.21.15.96:0/3159003985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc104ba0 msgr2=0x7f23fc104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 --2- 172.21.15.96:0/3159003985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc104ba0 0x7f23fc104fa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009930 tx=0x7f23ec02a420 comp rx=0 tx=0).stop 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 -- 172.21.15.96:0/3159003985 shutdown_connections 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 --2- 172.21.15.96:0/3159003985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23fc105cf0 0x7f23fc106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 --2- 172.21.15.96:0/3159003985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23fc1066b0 0x7f23fc10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 --2- 172.21.15.96:0/3159003985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc104ba0 0x7f23fc104fa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 -- 172.21.15.96:0/3159003985 >> 172.21.15.96:0/3159003985 conn(0x7f23fc100440 msgr2=0x7f23fc102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 -- 172.21.15.96:0/3159003985 shutdown_connections 2024-09-10T23:02:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.924+0000 7f2400c48640 1 -- 172.21.15.96:0/3159003985 wait complete. 2024-09-10T23:02:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.925+0000 7f2400c48640 1 Processor -- start 2024-09-10T23:02:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 -- start start 2024-09-10T23:02:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23fc104ba0 0x7f23fc196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23fc1066b0 0x7f23fc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc110840 con 0x7f23fc105cf0 2024-09-10T23:02:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1106c0 con 0x7f23fc1066b0 2024-09-10T23:02:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.926+0000 7f2400c48640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1109c0 con 0x7f23fc104ba0 2024-09-10T23:02:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36602/0 (socket says 172.21.15.96:36602) 2024-09-10T23:02:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 -- 172.21.15.96:0/3365620303 learned_addr learned my addr 172.21.15.96:0/3365620303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 -- 172.21.15.96:0/3365620303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23fc104ba0 msgr2=0x7f23fc196b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:02:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23fc104ba0 0x7f23fc196b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 -- 172.21.15.96:0/3365620303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23fc1066b0 msgr2=0x7f23fc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23fc1066b0 0x7f23fc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.928+0000 7f23f9d9b640 1 -- 172.21.15.96:0/3365620303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc198680 con 0x7f23fc105cf0 2024-09-10T23:02:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.929+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f23f000e9f0 tx=0x7f23f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.929+0000 7f23eb7fe640 1 -- 172.21.15.96:0/3365620303 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000cd80 con 0x7f23fc105cf0 2024-09-10T23:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.929+0000 7f23eb7fe640 1 -- 172.21.15.96:0/3365620303 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f00108e0 con 0x7f23fc105cf0 2024-09-10T23:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.929+0000 7f23eb7fe640 1 -- 172.21.15.96:0/3365620303 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0018980 con 0x7f23fc105cf0 2024-09-10T23:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.929+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc198970 con 0x7f23fc105cf0 2024-09-10T23:02:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.930+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc19f040 con 0x7f23fc105cf0 2024-09-10T23:02:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.933+0000 7f2403a14740 1 Processor -- start 2024-09-10T23:02:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 -- start start 2024-09-10T23:02:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cb190 0x55c8e24cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8e243a6f0 con 0x55c8e24cb190 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8e2429b20 con 0x55c8e24cdbd0 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.937+0000 7f2403a14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c8e242d930 con 0x55c8e24c8860 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51336/0 (socket says 172.21.15.96:51336) 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 -- 172.21.15.96:0/2125022673 learned_addr learned my addr 172.21.15.96:0/2125022673 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 -- 172.21.15.96:0/2125022673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 msgr2=0x55c8e24d0090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 -- 172.21.15.96:0/2125022673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cb190 msgr2=0x55c8e24cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cb190 0x55c8e24cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.938+0000 7f23fad9d640 1 -- 172.21.15.96:0/2125022673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e23ef530 con 0x55c8e24c8860 2024-09-10T23:02:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.939+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007cc0 tx=0x7f23f400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d534727977c0aece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.939+0000 7f23eb7fe640 1 -- 172.21.15.96:0/3365620303 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f23f0010430 con 0x7f23fc105cf0 2024-09-10T23:02:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.939+0000 7f23e8ff9640 1 -- 172.21.15.96:0/2125022673 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4014070 con 0x55c8e24c8860 2024-09-10T23:02:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.940+0000 7f23eb7fe640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 0x7f23e40b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.941+0000 7f23eb7fe640 1 -- 172.21.15.96:0/3365620303 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f23f00e5f60 con 0x7f23fc105cf0 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.941+0000 7f23e8ff9640 1 -- 172.21.15.96:0/2125022673 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f400fb00 con 0x55c8e24c8860 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.941+0000 7f23e8ff9640 1 -- 172.21.15.96:0/2125022673 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400eaf0 con 0x55c8e24c8860 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.943+0000 7f2403a14740 1 -- 172.21.15.96:0/2125022673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 msgr2=0x55c8e24cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.943+0000 7f2403a14740 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007cc0 tx=0x7f23f400c6a0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f23fa59c640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 0x7f23e40b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f2403a14740 1 -- 172.21.15.96:0/2125022673 shutdown_connections 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f2403a14740 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f2403a14740 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cb190 0x55c8e24cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f2403a14740 1 --2- 172.21.15.96:0/2125022673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24cac50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f2403a14740 1 -- 172.21.15.96:0/2125022673 >> 172.21.15.96:0/2125022673 conn(0x55c8e24c39b0 msgr2=0x55c8e24c4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.944+0000 7f23fa59c640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 0x7f23e40b7970 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0096c0 tx=0x7f23ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.953 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 885.0 s, 1685.4 tps, lat 60.824 ms stddev 79.659 2024-09-10T23:02:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.955+0000 7f2403a14740 1 -- 172.21.15.96:0/2125022673 shutdown_connections 2024-09-10T23:02:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 -- 172.21.15.96:0/2125022673 wait complete. 2024-09-10T23:02:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 -- start start 2024-09-10T23:02:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 Processor -- start 2024-09-10T23:02:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8e2361ab0 con 0x55c8e24cdbd0 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8e23cf730 con 0x55c8e24cb190 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c8e24d0bb0 con 0x55c8e24c8860 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.956+0000 7f2403a14740 10 client.0 _ll_register_callbacks cb 0x55c8e24236b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36630/0 (socket says 172.21.15.96:36630) 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 -- 172.21.15.96:0/2512203479 learned_addr learned my addr 172.21.15.96:0/2512203479 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 -- 172.21.15.96:0/2512203479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 msgr2=0x55c8e24d1ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:02:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24d1ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 -- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 msgr2=0x55c8e24d2590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24d2590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.957+0000 7f23fad9d640 1 -- 172.21.15.96:0/2512203479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e218b790 con 0x55c8e24cdbd0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.958+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e5c0 tx=0x7f23f400f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0625ed532fa20de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.958+0000 7f23e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.958+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4014040 con 0x55c8e24cdbd0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.958+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f400fb00 con 0x55c8e24cdbd0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.958+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f23f400ea70 con 0x55c8e24cdbd0 2024-09-10T23:02:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.959+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f23f400f940 con 0x55c8e24cdbd0 2024-09-10T23:02:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.961+0000 7f2403a14740 10 client.18705 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.961+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c8e2204620 con 0x55c8e24cdbd0 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f400fb00 con 0x55c8e24cdbd0 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f23f4021420 con 0x55c8e24cdbd0 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 10 client.18705 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 10 client.18705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 20 client.18705 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24dda60 0x55c8e24dff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 4 client.18705 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e243c640 con 0x55c8e24dda60 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.963+0000 7f23fa59c640 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24dda60 0x55c8e24dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.965+0000 7f23fa59c640 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24dda60 0x55c8e24dff20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3bedf6e01aea35 server_cookie=f8700512a2fe0177 in_seq=0 out_seq=0 2024-09-10T23:02:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.965+0000 7f23e1ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.967+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2512203479 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c8e243c640 con 0x55c8e24dda60 2024-09-10T23:02:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.967+0000 7f23e1ffb640 10 client.18705 handle_command_reply: tid=0 2024-09-10T23:02:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.967+0000 7f2403a14740 1 client.18705 shutdown 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.976+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 msgr2=0x55c8e24d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.976+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e5c0 tx=0x7f23f400f450 comp rx=0 tx=0).stop 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 shutdown_connections 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24dda60 0x55c8e24dff20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d3160 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 --2- 172.21.15.96:0/2512203479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24c8860 0x55c8e24d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 >> 172.21.15.96:0/2512203479 conn(0x55c8e24c39b0 msgr2=0x55c8e24c2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f23e1ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 shutdown_connections 2024-09-10T23:02:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.977+0000 7f2403a14740 1 -- 172.21.15.96:0/2512203479 wait complete. 2024-09-10T23:02:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.978+0000 7f2403a14740 20 client.18705 trim_cache size 0 max 16384 2024-09-10T23:02:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.999+0000 7f2403a14740 1 Processor -- start 2024-09-10T23:02:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:20.999+0000 7f2403a14740 1 -- start start 2024-09-10T23:02:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cdbd0 0x55c8e24c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a7d50 con 0x55c8e24c8860 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a9890 con 0x55c8e24cb190 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f2403a14740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a8420 con 0x55c8e24cdbd0 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36642/0 (socket says 172.21.15.96:36642) 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 -- 172.21.15.96:0/2414707530 learned_addr learned my addr 172.21.15.96:0/2414707530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fad9d640 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cdbd0 0x55c8e24c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 -- 172.21.15.96:0/2414707530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cdbd0 msgr2=0x55c8e24c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cdbd0 0x55c8e24c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 -- 172.21.15.96:0/2414707530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 msgr2=0x55c8e24ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.000+0000 7f23fa59c640 1 -- 172.21.15.96:0/2414707530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e24e0290 con 0x55c8e24c8860 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.001+0000 7f23fa59c640 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f23ec031500 tx=0x7f23ec03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a1e3e90e6ee2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.001+0000 7f23e3fff640 1 -- 172.21.15.96:0/2414707530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec054070 con 0x55c8e24c8860 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.001+0000 7f23e3fff640 1 -- 172.21.15.96:0/2414707530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ec04c070 con 0x55c8e24c8860 2024-09-10T23:02:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.001+0000 7f23e3fff640 1 -- 172.21.15.96:0/2414707530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec035040 con 0x55c8e24c8860 2024-09-10T23:02:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- 172.21.15.96:0/2414707530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 msgr2=0x55c8e24ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f23ec031500 tx=0x7f23ec03a000 comp rx=0 tx=0).stop 2024-09-10T23:02:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- 172.21.15.96:0/2414707530 shutdown_connections 2024-09-10T23:02:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cb190 0x55c8e24ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cdbd0 0x55c8e24c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 --2- 172.21.15.96:0/2414707530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24ce040 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- 172.21.15.96:0/2414707530 >> 172.21.15.96:0/2414707530 conn(0x55c8e24c39b0 msgr2=0x55c8e24c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- 172.21.15.96:0/2414707530 shutdown_connections 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- 172.21.15.96:0/2414707530 wait complete. 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.002+0000 7f2403a14740 1 -- start start 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 Processor -- start 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 0x55c8e24d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36658/0 (socket says 172.21.15.96:36658) 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f23fa59c640 1 -- 172.21.15.96:0/120308745 learned_addr learned my addr 172.21.15.96:0/120308745 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a91c0 con 0x55c8e24c8860 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f23fad9d640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a8af0 con 0x55c8e24cdbd0 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 0x55c8e24d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.003+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a5d30 con 0x55c8e24cb190 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f2403a14740 10 client.0 _ll_register_callbacks cb 0x7f23f4007b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 -- 172.21.15.96:0/120308745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 msgr2=0x55c8e24d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 0x55c8e24d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 -- 172.21.15.96:0/120308745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 msgr2=0x55c8e24d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e24d8140 con 0x55c8e24cdbd0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23f9d9b640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 0x55c8e24d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fa59c640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ec04c070 con 0x55c8e24cdbd0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23fad9d640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d7910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007a80 tx=0x7f23f4021650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fcd5fe949a7ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23e1ffb640 10 client.28319 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23e1ffb640 1 -- 172.21.15.96:0/120308745 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4038020 con 0x55c8e24cdbd0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23e1ffb640 1 -- 172.21.15.96:0/120308745 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f23f4003b70 con 0x55c8e24cdbd0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f23e1ffb640 1 -- 172.21.15.96:0/120308745 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f23f4014040 con 0x55c8e24cdbd0 2024-09-10T23:02:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f2403a14740 10 client.28319 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.004+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55c8e24d93b0 con 0x55c8e24cdbd0 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f23e1ffb640 1 -- 172.21.15.96:0/120308745 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f23f4004040 con 0x55c8e24cdbd0 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f2403a14740 10 client.28319 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f2403a14740 10 client.28319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f2403a14740 20 client.28319 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24d9bd0 0x55c8e254ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.005+0000 7f2403a14740 4 client.28319 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.006+0000 7f23fa59c640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24d9bd0 0x55c8e254ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.006+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8e24380f0 con 0x55c8e24d9bd0 2024-09-10T23:02:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.008+0000 7f23fa59c640 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24d9bd0 0x55c8e254ca40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5acf951e79883c8 server_cookie=ea9ccb340d521bd2 in_seq=0 out_seq=0 2024-09-10T23:02:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.008+0000 7f23e1ffb640 10 client.28319 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f23e1ffb640 1 -- 172.21.15.96:0/120308745 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8e24380f0 con 0x55c8e24d9bd0 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f23e1ffb640 10 client.28319 handle_command_reply: tid=0 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 client.28319 shutdown 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 msgr2=0x55c8e24d7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d7910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007a80 tx=0x7f23f4021650 comp rx=0 tx=0).stop 2024-09-10T23:02:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 shutdown_connections 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55c8e24d9bd0 0x55c8e254ca40 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e24cdbd0 0x55c8e24d7910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e24cb190 0x55c8e24d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 --2- 172.21.15.96:0/120308745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8e24c8860 0x55c8e24d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 >> 172.21.15.96:0/120308745 conn(0x55c8e24c39b0 msgr2=0x55c8e24c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.023+0000 7f23e1ffb640 0 client.28319 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.024+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 shutdown_connections 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.024+0000 7f2403a14740 1 -- 172.21.15.96:0/120308745 wait complete. 2024-09-10T23:02:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.024+0000 7f2403a14740 20 client.28319 trim_cache size 0 max 16384 2024-09-10T23:02:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:02:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:02:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:02:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 msgr2=0x7f23e40b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 0x7f23e40b7970 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0096c0 tx=0x7f23ec000bc0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 msgr2=0x7f23fc197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f23f000e9f0 tx=0x7f23f000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 shutdown_connections 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23fc1066b0 0x7f23fc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f23e40b54b0 0x7f23e40b7970 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23fc105cf0 0x7f23fc197190 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 --2- 172.21.15.96:0/3365620303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23fc104ba0 0x7f23fc196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 >> 172.21.15.96:0/3365620303 conn(0x7f23fc100440 msgr2=0x7f23fc0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 shutdown_connections 2024-09-10T23:02:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:21.025+0000 7f2400c48640 1 -- 172.21.15.96:0/3365620303 wait complete. 2024-09-10T23:02:21.404 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T23:02:21.747 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:21 smithi096 ceph-mon[30869]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-09-10T23:02:21.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:21 smithi121 ceph-mon[35119]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-09-10T23:02:21.769 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:21.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:21 smithi117 ceph-mon[35081]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-09-10T23:02:22.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 -- 172.21.15.96:0/202431868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8102420 msgr2=0x7f64b8102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 --2- 172.21.15.96:0/202431868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8102420 0x7f64b8102800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a802a400 comp rx=0 tx=0).stop 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 -- 172.21.15.96:0/202431868 shutdown_connections 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 --2- 172.21.15.96:0/202431868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8104e80 0x7f64b810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 --2- 172.21.15.96:0/202431868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 --2- 172.21.15.96:0/202431868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8102420 0x7f64b8102800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 -- 172.21.15.96:0/202431868 >> 172.21.15.96:0/202431868 conn(0x7f64b8078140 msgr2=0x7f64b80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 -- 172.21.15.96:0/202431868 shutdown_connections 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.339+0000 7f64bdde2640 1 -- 172.21.15.96:0/202431868 wait complete. 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 Processor -- start 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 -- start start 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 0x7f64b80fd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8104e80 0x7f64b80fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111cd0 con 0x7f64b81044c0 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111b50 con 0x7f64b8102420 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64bdde2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111e50 con 0x7f64b8104e80 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36684/0 (socket says 172.21.15.96:36684) 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 -- 172.21.15.96:0/663680916 learned_addr learned my addr 172.21.15.96:0/663680916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b7fff640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8104e80 0x7f64b80fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b77fe640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 0x7f64b80fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 -- 172.21.15.96:0/663680916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8104e80 msgr2=0x7f64b80fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8104e80 0x7f64b80fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 -- 172.21.15.96:0/663680916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 msgr2=0x7f64b80fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.340+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 0x7f64b80fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b6ffd640 1 -- 172.21.15.96:0/663680916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b81a76c0 con 0x7f64b81044c0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b77fe640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 0x7f64b80fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d950 tx=0x7f64ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b4ff9640 1 -- 172.21.15.96:0/663680916 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac017070 con 0x7f64b81044c0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b81a79b0 con 0x7f64b81044c0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b4ff9640 1 -- 172.21.15.96:0/663680916 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ac009c30 con 0x7f64b81044c0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64b4ff9640 1 -- 172.21.15.96:0/663680916 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00f620 con 0x7f64b81044c0 2024-09-10T23:02:22.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.341+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a7f40 con 0x7f64b81044c0 2024-09-10T23:02:22.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.344+0000 7f64c01b7740 1 Processor -- start 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 -- start start 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef351b0 0x55d64ef375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d64ee96ea0 con 0x55d64ef29220 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d64ee862d0 con 0x55d64ef28900 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64c01b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d64ee8a0e0 con 0x55d64ef351b0 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36840/0 (socket says 172.21.15.96:36840) 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.345+0000 7f64b7fff640 1 -- 172.21.15.96:0/4282263847 learned_addr learned my addr 172.21.15.96:0/4282263847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:22.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 -- 172.21.15.96:0/4282263847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef351b0 msgr2=0x55d64ef375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef351b0 0x55d64ef375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 -- 172.21.15.96:0/4282263847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 msgr2=0x55d64ef34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 -- 172.21.15.96:0/4282263847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64ebe4650 con 0x55d64ef28900 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b77fe640 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f64b7fff640 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009fc0 tx=0x7f64b0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7a2ddd2cc9f757 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f649dffb640 1 -- 172.21.15.96:0/4282263847 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0016070 con 0x55d64ef28900 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f649dffb640 1 -- 172.21.15.96:0/4282263847 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0012bc0 con 0x55d64ef28900 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.346+0000 7f649dffb640 1 -- 172.21.15.96:0/4282263847 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b001bc90 con 0x55d64ef28900 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 -- 172.21.15.96:0/4282263847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 msgr2=0x55d64ef28ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009fc0 tx=0x7f64b0010800 comp rx=0 tx=0).stop 2024-09-10T23:02:22.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 -- 172.21.15.96:0/4282263847 shutdown_connections 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef351b0 0x55d64ef375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 --2- 172.21.15.96:0/4282263847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef28ce0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64c01b7740 1 -- 172.21.15.96:0/4282263847 >> 172.21.15.96:0/4282263847 conn(0x55d64ef26730 msgr2=0x55d64ef278d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.347+0000 7f64b4ff9640 1 -- 172.21.15.96:0/663680916 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f64ac010460 con 0x7f64b81044c0 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.348+0000 7f64c01b7740 1 -- 172.21.15.96:0/4282263847 shutdown_connections 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.348+0000 7f64c01b7740 1 -- 172.21.15.96:0/4282263847 wait complete. 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.348+0000 7f64c01b7740 1 -- start start 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 Processor -- start 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b4ff9640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 0x7f64a00b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef32060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef41190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d64ece7700 con 0x55d64ef32750 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d64ee21160 con 0x55d64ef29220 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d64ef37e30 con 0x55d64ef28900 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64c01b7740 10 client.0 _ll_register_callbacks cb 0x55d64ee3a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b7fff640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 0x7f64a00b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef41190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51398/0 (socket says 172.21.15.96:51398) 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b77fe640 1 -- 172.21.15.96:0/3274738463 learned_addr learned my addr 172.21.15.96:0/3274738463 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.349+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef32060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b4ff9640 1 -- 172.21.15.96:0/663680916 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f64ac0e7980 con 0x7f64b81044c0 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 -- 172.21.15.96:0/3274738463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 msgr2=0x55d64ef32060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef32060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 -- 172.21.15.96:0/3274738463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 msgr2=0x55d64ef41190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef41190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 -- 172.21.15.96:0/3274738463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64ee4ec90 con 0x55d64ef28900 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b7fff640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 0x7f64a00b7730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64b0014000 tx=0x7f64b0000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64b77fe640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a8005390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f338ff9c7e49f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f6496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8035040 con 0x55d64ef28900 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f64a80048c0 con 0x55d64ef28900 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0012bc0 con 0x55d64ef28900 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64a8004ac0 con 0x55d64ef28900 2024-09-10T23:02:22.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64c01b7740 10 client.28393 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d64ef43d00 con 0x55d64ef28900 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.350+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0012bc0 con 0x55d64ef28900 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f64a80314a0 con 0x55d64ef28900 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 10 client.28393 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 10 client.28393 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 20 client.28393 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64ef451c0 0x55d64ef475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 4 client.28393 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:22.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64ee98df0 con 0x55d64ef451c0 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.351+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64ef451c0 0x55d64ef475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.352+0000 7f6496ffd640 10 client.28393 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.353+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64ef451c0 0x55d64ef475b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4396649c7ef676aa server_cookie=38a274cd80bba94b in_seq=0 out_seq=0 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.353+0000 7f6496ffd640 1 -- 172.21.15.96:0/3274738463 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d64ee98df0 con 0x55d64ef451c0 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.353+0000 7f6496ffd640 10 client.28393 handle_command_reply: tid=0 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.353+0000 7f64c01b7740 1 client.28393 shutdown 2024-09-10T23:02:22.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 msgr2=0x55d64ef34c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a8005390 comp rx=0 tx=0).stop 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 shutdown_connections 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64ef451c0 0x55d64ef475b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef32060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3274738463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef34c10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f6496ffd640 0 client.28393 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 >> 172.21.15.96:0/3274738463 conn(0x55d64ef26730 msgr2=0x55d64ef2c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 shutdown_connections 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 1 -- 172.21.15.96:0/3274738463 wait complete. 2024-09-10T23:02:22.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.354+0000 7f64c01b7740 20 client.28393 trim_cache size 0 max 16384 2024-09-10T23:02:22.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.370+0000 7f64c01b7740 1 Processor -- start 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.370+0000 7f64c01b7740 1 -- start start 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef27f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ac100 con 0x55d64ef32750 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81adc40 con 0x55d64ef29220 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64c01b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ac7d0 con 0x55d64ef28900 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36716/0 (socket says 172.21.15.96:36716) 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b7fff640 1 -- 172.21.15.96:0/371014170 learned_addr learned my addr 172.21.15.96:0/371014170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef27f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 -- 172.21.15.96:0/371014170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 msgr2=0x55d64ef41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 -- 172.21.15.96:0/371014170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 msgr2=0x55d64ef415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.371+0000 7f64b77fe640 1 -- 172.21.15.96:0/371014170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64ef34840 con 0x55d64ef28900 2024-09-10T23:02:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.372+0000 7f64b77fe640 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef27f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64a80318f0 tx=0x7f64a8031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcde8f79fbc4bcc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.372+0000 7f649d7fa640 1 -- 172.21.15.96:0/371014170 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8035070 con 0x55d64ef28900 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.372+0000 7f649d7fa640 1 -- 172.21.15.96:0/371014170 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a8040070 con 0x55d64ef28900 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.372+0000 7f649d7fa640 1 -- 172.21.15.96:0/371014170 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004920 con 0x55d64ef28900 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- 172.21.15.96:0/371014170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 msgr2=0x55d64ef27f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef27f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64a80318f0 tx=0x7f64a8031400 comp rx=0 tx=0).stop 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- 172.21.15.96:0/371014170 shutdown_connections 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef29220 0x55d64ef41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef32750 0x55d64ef415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 --2- 172.21.15.96:0/371014170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef28900 0x55d64ef27f70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- 172.21.15.96:0/371014170 >> 172.21.15.96:0/371014170 conn(0x55d64ef26730 msgr2=0x55d64ef27860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- 172.21.15.96:0/371014170 shutdown_connections 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- 172.21.15.96:0/371014170 wait complete. 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.373+0000 7f64c01b7740 1 -- start start 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 Processor -- start 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef32750 0x55d64ef2e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ad570 con 0x55d64ef29220 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81acea0 con 0x55d64ef28900 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81aa140 con 0x55d64ef32750 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64c01b7740 10 client.0 _ll_register_callbacks cb 0x7f64a8005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36732/0 (socket says 172.21.15.96:36732) 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 -- 172.21.15.96:0/3519518782 learned_addr learned my addr 172.21.15.96:0/3519518782 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b7fff640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef32750 0x55d64ef2e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 -- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef32750 msgr2=0x55d64ef2e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef32750 0x55d64ef2e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 -- 172.21.15.96:0/3519518782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 msgr2=0x55d64ef3d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef3d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.374+0000 7f64b6ffd640 1 -- 172.21.15.96:0/3519518782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64ef3de80 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64b6ffd640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0aa710 tx=0x7f64ac010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c32630c2f40c829 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac020040 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f64ac0ad3a0 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a8040070 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64ac0d50d0 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 10 client.18729 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d64ef2f900 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a8040070 con 0x55d64ef29220 2024-09-10T23:02:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f64ac010780 con 0x55d64ef29220 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 10 client.18729 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 20 client.18729 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64efb3070 0x55d64efb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 4 client.18729 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.375+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d64ee948a0 con 0x55d64efb3070 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64b77fe640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64efb3070 0x55d64efb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f6496ffd640 10 client.18729 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64b77fe640 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64efb3070 0x55d64efb5460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a6de8a061d1c9d8 server_cookie=b7b1bbc80c7a696d in_seq=0 out_seq=0 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f6496ffd640 1 -- 172.21.15.96:0/3519518782 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d64ee948a0 con 0x55d64efb3070 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f6496ffd640 10 client.18729 handle_command_reply: tid=0 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 client.18729 shutdown 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 msgr2=0x55d64ef3d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0aa710 tx=0x7f64ac010cb0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 shutdown_connections 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d64ef32750 0x55d64ef2e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d64ef29220 0x55d64ef3d630 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x55d64efb3070 0x55d64efb5460 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 --2- 172.21.15.96:0/3519518782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ef28900 0x55d64ef3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 >> 172.21.15.96:0/3519518782 conn(0x55d64ef26730 msgr2=0x55d64ef33690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f6496ffd640 0 client.18729 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 shutdown_connections 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.376+0000 7f64c01b7740 1 -- 172.21.15.96:0/3519518782 wait complete. 2024-09-10T23:02:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.377+0000 7f64c01b7740 20 client.18729 trim_cache size 0 max 16384 2024-09-10T23:02:22.385 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.378+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 msgr2=0x7f64a00b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.378+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 0x7f64a00b7730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64b0014000 tx=0x7f64b0000f40 comp rx=0 tx=0).stop 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.378+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 msgr2=0x7f64b80fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.378+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d950 tx=0x7f64ac00de10 comp rx=0 tx=0).stop 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 shutdown_connections 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f64a00b5270 0x7f64a00b7730 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64b8104e80 0x7f64b80fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64b81044c0 0x7f64b80fdb80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 --2- 172.21.15.96:0/663680916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64b8102420 0x7f64b80fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:22.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 >> 172.21.15.96:0/663680916 conn(0x7f64b8078140 msgr2=0x7f64b80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:22.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 shutdown_connections 2024-09-10T23:02:22.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:22.379+0000 7f64bdde2640 1 -- 172.21.15.96:0/663680916 wait complete. 2024-09-10T23:02:23.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:02:23.478 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:23.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:23 smithi096 ceph-mon[30869]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-10T23:02:23.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:23 smithi121 ceph-mon[35119]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-10T23:02:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:23 smithi117 ceph-mon[35081]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 -- 172.21.15.96:0/4035934837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954102dc0 msgr2=0x7f69541031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 --2- 172.21.15.96:0/4035934837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954102dc0 0x7f69541031a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6948009900 tx=0x7f694802a410 comp rx=0 tx=0).stop 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 -- 172.21.15.96:0/4035934837 shutdown_connections 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 --2- 172.21.15.96:0/4035934837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954109150 0x7f695410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 --2- 172.21.15.96:0/4035934837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69541087d0 0x7f6954108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 --2- 172.21.15.96:0/4035934837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954102dc0 0x7f69541031a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 -- 172.21.15.96:0/4035934837 >> 172.21.15.96:0/4035934837 conn(0x7f69540fe830 msgr2=0x7f6954100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 -- 172.21.15.96:0/4035934837 shutdown_connections 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.068+0000 7f695a945640 1 -- 172.21.15.96:0/4035934837 wait complete. 2024-09-10T23:02:24.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.069+0000 7f695a945640 1 Processor -- start 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.069+0000 7f695a945640 1 -- start start 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69541087d0 0x7f695410c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954109150 0x7f695410cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6954115060 con 0x7f6954102dc0 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6954114ee0 con 0x7f69541087d0 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f695a945640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f69541151e0 con 0x7f6954109150 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f6953fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f6953fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36752/0 (socket says 172.21.15.96:36752) 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f6953fff640 1 -- 172.21.15.96:0/3393682702 learned_addr learned my addr 172.21.15.96:0/3393682702 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.070+0000 7f6958ee2640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954109150 0x7f695410cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 -- 172.21.15.96:0/3393682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954109150 msgr2=0x7f695410cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954109150 0x7f695410cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 -- 172.21.15.96:0/3393682702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69541087d0 msgr2=0x7f695410c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69541087d0 0x7f695410c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 -- 172.21.15.96:0/3393682702 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695410d5d0 con 0x7f6954102dc0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f6953fff640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6948005ab0 tx=0x7f6948003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f69517fa640 1 -- 172.21.15.96:0/3393682702 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694803f070 con 0x7f6954102dc0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f69517fa640 1 -- 172.21.15.96:0/3393682702 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6948031b60 con 0x7f6954102dc0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f69517fa640 1 -- 172.21.15.96:0/3393682702 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948035030 con 0x7f6954102dc0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541a1520 con 0x7f6954102dc0 2024-09-10T23:02:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.071+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a19b0 con 0x7f6954102dc0 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 Processor -- start 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 -- start start 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c41840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e642babea0 con 0x55e642c35df0 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e642b9b2d0 con 0x55e642c41d80 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.075+0000 7f695cd1a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e642b9f0e0 con 0x55e642c354d0 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51442/0 (socket says 172.21.15.96:51442) 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 -- 172.21.15.96:0/1471120289 learned_addr learned my addr 172.21.15.96:0/1471120289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f69537fe640 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 -- 172.21.15.96:0/1471120289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 msgr2=0x55e642c44170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c44170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 -- 172.21.15.96:0/1471120289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 msgr2=0x55e642c41840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c41840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.076+0000 7f6958ee2640 1 -- 172.21.15.96:0/1471120289 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6428f9650 con 0x55e642c354d0 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.077+0000 7f6958ee2640 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f694c009f20 tx=0x7f694c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651dd6ecff9d1779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.077+0000 7f69517fa640 1 -- 172.21.15.96:0/3393682702 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f694803b460 con 0x7f6954102dc0 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.077+0000 7f693a7fc640 1 -- 172.21.15.96:0/1471120289 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c004d20 con 0x55e642c354d0 2024-09-10T23:02:24.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.077+0000 7f693a7fc640 1 -- 172.21.15.96:0/1471120289 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694c004050 con 0x55e642c354d0 2024-09-10T23:02:24.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.077+0000 7f693a7fc640 1 -- 172.21.15.96:0/1471120289 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c005280 con 0x55e642c354d0 2024-09-10T23:02:24.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.078+0000 7f69517fa640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 0x7f693c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.078+0000 7f6953fff640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 0x7f693c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.078+0000 7f69517fa640 1 -- 172.21.15.96:0/3393682702 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f69481064f0 con 0x7f6954102dc0 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.078+0000 7f695cd1a740 1 -- 172.21.15.96:0/1471120289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 msgr2=0x55e642c358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.078+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f694c009f20 tx=0x7f694c0118d0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 -- 172.21.15.96:0/1471120289 shutdown_connections 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c44170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c41840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1471120289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c358b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 -- 172.21.15.96:0/1471120289 >> 172.21.15.96:0/1471120289 conn(0x55e642c33300 msgr2=0x55e642c344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 -- 172.21.15.96:0/1471120289 shutdown_connections 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 -- 172.21.15.96:0/1471120289 wait complete. 2024-09-10T23:02:24.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.079+0000 7f695cd1a740 1 -- start start 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.080+0000 7f6953fff640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 0x7f693c0b7820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f694802a9d0 tx=0x7f694803f970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.080+0000 7f695cd1a740 1 Processor -- start 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c41d80 0x55e642c3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6429fc700 con 0x55e642c354d0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e642b36160 con 0x55e642c35df0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e642c44aa0 con 0x55e642c41d80 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f695cd1a740 10 client.0 _ll_register_callbacks cb 0x55e642b4f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36762/0 (socket says 172.21.15.96:36762) 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 -- 172.21.15.96:0/2961424814 learned_addr learned my addr 172.21.15.96:0/2961424814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c41d80 0x55e642c3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f69537fe640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 -- 172.21.15.96:0/2961424814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c41d80 msgr2=0x55e642c3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c41d80 0x55e642c3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 -- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 msgr2=0x55e642c3ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c3ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.081+0000 7f6958ee2640 1 -- 172.21.15.96:0/2961424814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e642b63c90 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f6958ee2640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f694c005c40 tx=0x7f694c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04be18e4a5de0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f69337fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c004410 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f694c0045b0 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694c004050 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f694c005040 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f695cd1a740 10 client.18738 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.082+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e642c50420 con 0x55e642c354d0 2024-09-10T23:02:24.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694c004050 con 0x55e642c354d0 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f694c044650 con 0x55e642c354d0 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 10 client.18738 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 20 client.18738 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642c51770 0x55e642c53b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 4 client.18738 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e642baddf0 con 0x55e642c51770 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.083+0000 7f69537fe640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642c51770 0x55e642c53b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.084+0000 7f69537fe640 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642c51770 0x55e642c53b60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbcf03cbd8d6a3c server_cookie=492e66f9e61d03ee in_seq=0 out_seq=0 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.084+0000 7f69337fe640 10 client.18738 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.085+0000 7f69337fe640 1 -- 172.21.15.96:0/2961424814 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e642baddf0 con 0x55e642c51770 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.085+0000 7f69337fe640 10 client.18738 handle_command_reply: tid=0 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.085+0000 7f695cd1a740 1 client.18738 shutdown 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 msgr2=0x55e642c41680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f694c005c40 tx=0x7f694c003f50 comp rx=0 tx=0).stop 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 shutdown_connections 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642c51770 0x55e642c53b60 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c3ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c41d80 0x55e642c3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 --2- 172.21.15.96:0/2961424814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c354d0 0x55e642c41680 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 >> 172.21.15.96:0/2961424814 conn(0x55e642c33300 msgr2=0x55e642c38ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.086+0000 7f69337fe640 0 client.18738 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:24.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.087+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 shutdown_connections 2024-09-10T23:02:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.087+0000 7f695cd1a740 1 -- 172.21.15.96:0/2961424814 wait complete. 2024-09-10T23:02:24.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.087+0000 7f695cd1a740 20 client.18738 trim_cache size 0 max 16384 2024-09-10T23:02:24.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.108+0000 7f695cd1a740 1 Processor -- start 2024-09-10T23:02:24.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.110+0000 7f695cd1a740 1 -- start start 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.110+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.110+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c3bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f695cd1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6954073e30 con 0x55e642c35df0 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f695cd1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b6300 con 0x55e642c41d80 2024-09-10T23:02:24.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f69537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f695cd1a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6954074500 con 0x55e642c354d0 2024-09-10T23:02:24.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f69537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36778/0 (socket says 172.21.15.96:36778) 2024-09-10T23:02:24.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.111+0000 7f69537fe640 1 -- 172.21.15.96:0/3016691655 learned_addr learned my addr 172.21.15.96:0/3016691655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:24.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.112+0000 7f6958ee2640 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.112+0000 7f6958ee2640 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c3bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.114+0000 7f69537fe640 1 -- 172.21.15.96:0/3016691655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 msgr2=0x55e642c343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.114+0000 7f69537fe640 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.114+0000 7f69537fe640 1 -- 172.21.15.96:0/3016691655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 msgr2=0x55e642c3bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.114+0000 7f69537fe640 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c3bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.114+0000 7f69537fe640 1 -- 172.21.15.96:0/3016691655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e642c33d80 con 0x55e642c35df0 2024-09-10T23:02:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.118+0000 7f69537fe640 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f695410f370 tx=0x7f6944004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab12b619069a2ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.119+0000 7f6939ffb640 1 -- 172.21.15.96:0/3016691655 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944003d50 con 0x55e642c35df0 2024-09-10T23:02:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.119+0000 7f6939ffb640 1 -- 172.21.15.96:0/3016691655 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694401c870 con 0x55e642c35df0 2024-09-10T23:02:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.119+0000 7f6939ffb640 1 -- 172.21.15.96:0/3016691655 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944010790 con 0x55e642c35df0 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.124+0000 7f695cd1a740 1 -- 172.21.15.96:0/3016691655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 msgr2=0x55e642c32c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.124+0000 7f695cd1a740 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f695410f370 tx=0x7f6944004410 comp rx=0 tx=0).stop 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.125+0000 7f695cd1a740 1 -- 172.21.15.96:0/3016691655 shutdown_connections 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.125+0000 7f695cd1a740 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c41d80 0x55e642c3bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.125+0000 7f695cd1a740 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c35df0 0x55e642c32c10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.125+0000 7f695cd1a740 1 --2- 172.21.15.96:0/3016691655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.125+0000 7f695cd1a740 1 -- 172.21.15.96:0/3016691655 >> 172.21.15.96:0/3016691655 conn(0x55e642c33300 msgr2=0x55e642c3d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.130+0000 7f695cd1a740 1 -- 172.21.15.96:0/3016691655 shutdown_connections 2024-09-10T23:02:24.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.131+0000 7f695cd1a740 1 -- 172.21.15.96:0/3016691655 wait complete. 2024-09-10T23:02:24.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.131+0000 7f695cd1a740 1 -- start start 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.131+0000 7f695cd1a740 1 Processor -- start 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c49d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c41d80 0x55e642c44d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b5d00 con 0x55e642c41d80 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6954074bd0 con 0x55e642c35df0 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6954071e10 con 0x55e642c354d0 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f695cd1a740 10 client.0 _ll_register_callbacks cb 0x7f69440093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f69537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f69537fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:36938/0 (socket says 172.21.15.96:36938) 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f69537fe640 1 -- 172.21.15.96:0/1503138400 learned_addr learned my addr 172.21.15.96:0/1503138400 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:24.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.132+0000 7f6958ee2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c49d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.136+0000 7f69537fe640 1 -- 172.21.15.96:0/1503138400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 msgr2=0x55e642c49d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.136+0000 7f69537fe640 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c49d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.136+0000 7f69537fe640 1 -- 172.21.15.96:0/1503138400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c41d80 msgr2=0x55e642c44d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:24.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.136+0000 7f69537fe640 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c41d80 0x55e642c44d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.136+0000 7f69537fe640 1 -- 172.21.15.96:0/1503138400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e642c4a8a0 con 0x55e642c35df0 2024-09-10T23:02:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.141+0000 7f69537fe640 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6944009ed0 tx=0x7f6944005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd639bf4b7d3241 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:24.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.141+0000 7f69337fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.142+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944003bd0 con 0x55e642c35df0 2024-09-10T23:02:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.142+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f694401c3c0 con 0x55e642c35df0 2024-09-10T23:02:24.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.143+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694401c870 con 0x55e642c35df0 2024-09-10T23:02:24.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.148+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6944003d70 con 0x55e642c35df0 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.148+0000 7f695cd1a740 10 client.28349 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.148+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e642cbe890 con 0x55e642c35df0 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694401c870 con 0x55e642c35df0 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6944024b40 con 0x55e642c35df0 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 10 client.28349 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 10 client.28349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 20 client.28349 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642cbef30 0x55e642cc1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 4 client.28349 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.152+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e642ba98a0 con 0x55e642cbef30 2024-09-10T23:02:24.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.153+0000 7f6958ee2640 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642cbef30 0x55e642cc1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.169+0000 7f69337fe640 10 client.28349 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.169+0000 7f6958ee2640 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642cbef30 0x55e642cc1350 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67b95e7fd71d56f server_cookie=af66c02c480cd1e in_seq=0 out_seq=0 2024-09-10T23:02:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.170+0000 7f69337fe640 1 -- 172.21.15.96:0/1503138400 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e642ba98a0 con 0x55e642cbef30 2024-09-10T23:02:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.170+0000 7f69337fe640 10 client.28349 handle_command_reply: tid=0 2024-09-10T23:02:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.170+0000 7f695cd1a740 1 client.28349 shutdown 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 msgr2=0x55e642c4a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6944009ed0 tx=0x7f6944005fa0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 shutdown_connections 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x55e642cbef30 0x55e642cc1350 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e642c35df0 0x55e642c4a2b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e642c41d80 0x55e642c44d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f69337fe640 0 client.28349 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 --2- 172.21.15.96:0/1503138400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e642c354d0 0x55e642c49d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.171+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 >> 172.21.15.96:0/1503138400 conn(0x55e642c33300 msgr2=0x55e642c40920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.172+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 shutdown_connections 2024-09-10T23:02:24.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.174+0000 7f695cd1a740 1 -- 172.21.15.96:0/1503138400 wait complete. 2024-09-10T23:02:24.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.175+0000 7f695cd1a740 20 client.28349 trim_cache size 0 max 16384 2024-09-10T23:02:24.178 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:02:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.179+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 msgr2=0x7f693c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.179+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 0x7f693c0b7820 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f694802a9d0 tx=0x7f694803f970 comp rx=0 tx=0).stop 2024-09-10T23:02:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.179+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 msgr2=0x7f695410f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.179+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6948005ab0 tx=0x7f6948003c10 comp rx=0 tx=0).stop 2024-09-10T23:02:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 shutdown_connections 2024-09-10T23:02:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69541087d0 0x7f695410c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f693c0b5360 0x7f693c0b7820 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6954109150 0x7f695410cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 --2- 172.21.15.96:0/3393682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6954102dc0 0x7f695410f620 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 >> 172.21.15.96:0/3393682702 conn(0x7f69540fe830 msgr2=0x7f6954100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 shutdown_connections 2024-09-10T23:02:24.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:24.183+0000 7f695a945640 1 -- 172.21.15.96:0/3393682702 wait complete. 2024-09-10T23:02:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:25 smithi096 ceph-mon[30869]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-09-10T23:02:25.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:25 smithi121 ceph-mon[35119]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-09-10T23:02:25.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:25 smithi117 ceph-mon[35081]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.6 MiB/s wr, 579 op/s 2024-09-10T23:02:25.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 890.0 s, 1825.4 tps, lat 55.045 ms stddev 61.669 2024-09-10T23:02:25.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T23:02:25.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T23:02:26.368 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:26.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.922+0000 7f9016a8c640 1 -- 172.21.15.96:0/4229857543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 msgr2=0x7f9010106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.922+0000 7f9016a8c640 1 --2- 172.21.15.96:0/4229857543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010106880 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009860 tx=0x7f8ffc02a380 comp rx=0 tx=0).stop 2024-09-10T23:02:26.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 -- 172.21.15.96:0/4229857543 shutdown_connections 2024-09-10T23:02:26.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 --2- 172.21.15.96:0/4229857543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9010106ed0 0x7f9010107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 --2- 172.21.15.96:0/4229857543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010106880 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 --2- 172.21.15.96:0/4229857543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 -- 172.21.15.96:0/4229857543 >> 172.21.15.96:0/4229857543 conn(0x7f90100fd700 msgr2=0x7f90100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:26.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 -- 172.21.15.96:0/4229857543 shutdown_connections 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.923+0000 7f9016a8c640 1 -- 172.21.15.96:0/4229857543 wait complete. 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 Processor -- start 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 -- start start 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9010106ed0 0x7f901006dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9010119390 con 0x7f9010106ed0 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9010119210 con 0x7f9010101a90 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9016a8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9010119510 con 0x7f90101023b0 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9014828640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9014828640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44392/0 (socket says 172.21.15.96:44392) 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.924+0000 7f9014828640 1 -- 172.21.15.96:0/3281186353 learned_addr learned my addr 172.21.15.96:0/3281186353 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:26.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.925+0000 7f900ffff640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 -- 172.21.15.96:0/3281186353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 msgr2=0x7f9010074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 -- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9010106ed0 msgr2=0x7f901006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9010106ed0 0x7f901006dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 -- 172.21.15.96:0/3281186353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901006e560 con 0x7f9010101a90 2024-09-10T23:02:26.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.927+0000 7f9014828640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f900400c890 tx=0x7f900400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:26.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.928+0000 7f900dffb640 1 -- 172.21.15.96:0/3281186353 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004004220 con 0x7f9010101a90 2024-09-10T23:02:26.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.928+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901006e850 con 0x7f9010101a90 2024-09-10T23:02:26.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.928+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90100737b0 con 0x7f9010101a90 2024-09-10T23:02:26.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.930+0000 7f900dffb640 1 -- 172.21.15.96:0/3281186353 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90040043c0 con 0x7f9010101a90 2024-09-10T23:02:26.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.930+0000 7f900dffb640 1 -- 172.21.15.96:0/3281186353 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004004a00 con 0x7f9010101a90 2024-09-10T23:02:26.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.934+0000 7f9018e61740 1 Processor -- start 2024-09-10T23:02:26.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.935+0000 7f900dffb640 1 -- 172.21.15.96:0/3281186353 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9004004ba0 con 0x7f9010101a90 2024-09-10T23:02:26.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.936+0000 7f900dffb640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 0x7f90000b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.936+0000 7f900dffb640 1 -- 172.21.15.96:0/3281186353 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f90040e5c10 con 0x7f9010101a90 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 -- start start 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a7ee0 0x559f577a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f5771ee60 con 0x559f577a7ee0 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f5770e290 con 0x559f577a8800 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.937+0000 7f9018e61740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f577120a0 con 0x559f577b4790 2024-09-10T23:02:26.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.938+0000 7f900ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.938+0000 7f900ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44394/0 (socket says 172.21.15.96:44394) 2024-09-10T23:02:26.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.938+0000 7f900ffff640 1 -- 172.21.15.96:0/2987953775 learned_addr learned my addr 172.21.15.96:0/2987953775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:26.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.938+0000 7f9015029640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 0x7f90000b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f9015029640 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f9014828640 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a7ee0 0x559f577a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f900ffff640 1 -- 172.21.15.96:0/2987953775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 msgr2=0x559f577b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f900ffff640 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f900ffff640 1 -- 172.21.15.96:0/2987953775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a7ee0 msgr2=0x559f577a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f900ffff640 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a7ee0 0x559f577a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f900ffff640 1 -- 172.21.15.96:0/2987953775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5746c930 con 0x559f577a8800 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.939+0000 7f9015029640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 0x7f90000b79b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9008005c40 tx=0x7f9008009ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:26.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.940+0000 7f900ffff640 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009860 tx=0x7f8ffc030bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b1cd9bed9cfb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:26.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.940+0000 7f8ffaffd640 1 -- 172.21.15.96:0/2987953775 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc041020 con 0x559f577a8800 2024-09-10T23:02:26.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.940+0000 7f8ffaffd640 1 -- 172.21.15.96:0/2987953775 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc030970 con 0x559f577a8800 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 -- 172.21.15.96:0/2987953775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 msgr2=0x559f577b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009860 tx=0x7f8ffc030bc0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 -- 172.21.15.96:0/2987953775 shutdown_connections 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a8800 0x559f577b4250 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 --2- 172.21.15.96:0/2987953775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a7ee0 0x559f577a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.941+0000 7f9018e61740 1 -- 172.21.15.96:0/2987953775 >> 172.21.15.96:0/2987953775 conn(0x559f577a5d10 msgr2=0x559f577a6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:26.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.942+0000 7f9018e61740 1 -- 172.21.15.96:0/2987953775 shutdown_connections 2024-09-10T23:02:26.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.942+0000 7f9018e61740 1 -- 172.21.15.96:0/2987953775 wait complete. 2024-09-10T23:02:26.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.942+0000 7f9018e61740 1 -- start start 2024-09-10T23:02:26.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.942+0000 7f9018e61740 1 Processor -- start 2024-09-10T23:02:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f5756f890 con 0x559f577a8800 2024-09-10T23:02:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f576a8d40 con 0x559f577b4790 2024-09-10T23:02:26.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f577b7430 con 0x559f577a7ee0 2024-09-10T23:02:26.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9018e61740 10 client.0 _ll_register_callbacks cb 0x559f5770cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:26.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9014828640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9014828640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51480/0 (socket says 172.21.15.96:51480) 2024-09-10T23:02:26.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9014828640 1 -- 172.21.15.96:0/3807608046 learned_addr learned my addr 172.21.15.96:0/3807608046 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.943+0000 7f9015029640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.944+0000 7f900ffff640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 -- 172.21.15.96:0/3807608046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 msgr2=0x559f577b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 -- 172.21.15.96:0/3807608046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 msgr2=0x559f577b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 -- 172.21.15.96:0/3807608046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f577c07b0 con 0x559f577b4790 2024-09-10T23:02:26.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f9015029640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577b1e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9008009fd0 tx=0x7f90080095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8164dca004393ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.945+0000 7f8feffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.946+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008010070 con 0x559f577b4790 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.946+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9008010210 con 0x559f577b4790 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.946+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ffc030970 con 0x559f577b4790 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.946+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9008027470 con 0x559f577b4790 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.946+0000 7f9018e61740 10 client.28358 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:26.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.947+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f577c3120 con 0x559f577b4790 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.950+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc030970 con 0x559f577b4790 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.950+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f900801d050 con 0x559f577b4790 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 10 client.28358 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 10 client.28358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 20 client.28358 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f577c4530 0x559f577c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 4 client.28358 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.951+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f57720db0 con 0x559f577c4530 2024-09-10T23:02:26.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.952+0000 7f9014828640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f577c4530 0x559f577c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.953+0000 7f9014828640 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f577c4530 0x559f577c6940 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39757843b8804de5 server_cookie=b9e11ea3edc39343 in_seq=0 out_seq=0 2024-09-10T23:02:26.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.953+0000 7f8feffff640 10 client.28358 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:26.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.954+0000 7f8feffff640 1 -- 172.21.15.96:0/3807608046 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f57720db0 con 0x559f577c4530 2024-09-10T23:02:26.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.954+0000 7f8feffff640 10 client.28358 handle_command_reply: tid=0 2024-09-10T23:02:26.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.954+0000 7f9018e61740 1 client.28358 shutdown 2024-09-10T23:02:26.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.956+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 msgr2=0x559f577b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.956+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577b1e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9008009fd0 tx=0x7f90080095c0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 shutdown_connections 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f577c4530 0x559f577c6940 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577b1e70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 --2- 172.21.15.96:0/3807608046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 >> 172.21.15.96:0/3807608046 conn(0x559f577a5d10 msgr2=0x559f577a4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f8feffff640 0 client.28358 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 shutdown_connections 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.957+0000 7f9018e61740 1 -- 172.21.15.96:0/3807608046 wait complete. 2024-09-10T23:02:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.958+0000 7f9018e61740 20 client.28358 trim_cache size 0 max 16384 2024-09-10T23:02:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 Processor -- start 2024-09-10T23:02:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 -- start start 2024-09-10T23:02:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9010110950 con 0x559f577a8800 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90101c7560 con 0x559f577b4790 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9018e61740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9010111020 con 0x559f577a7ee0 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9014828640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9014828640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51488/0 (socket says 172.21.15.96:51488) 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.989+0000 7f9014828640 1 -- 172.21.15.96:0/1839532273 learned_addr learned my addr 172.21.15.96:0/1839532273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9015029640 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 -- 172.21.15.96:0/1839532273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 msgr2=0x559f577aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 -- 172.21.15.96:0/1839532273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 msgr2=0x559f577ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 -- 172.21.15.96:0/1839532273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f577b1a50 con 0x559f577a7ee0 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9015029640 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f9014828640 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9004007fd0 tx=0x7f90040e35e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96637de9e2338972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f8ffa7fc640 1 -- 172.21.15.96:0/1839532273 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900401d040 con 0x559f577a7ee0 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f8ffa7fc640 1 -- 172.21.15.96:0/1839532273 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90040abee0 con 0x559f577a7ee0 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.990+0000 7f8ffa7fc640 1 -- 172.21.15.96:0/1839532273 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040d3370 con 0x559f577a7ee0 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.991+0000 7f9018e61740 1 -- 172.21.15.96:0/1839532273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 msgr2=0x559f577ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.991+0000 7f9018e61740 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9004007fd0 tx=0x7f90040e35e0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 -- 172.21.15.96:0/1839532273 shutdown_connections 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577b4790 0x559f577aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 --2- 172.21.15.96:0/1839532273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577a7ee0 0x559f577ab7a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 -- 172.21.15.96:0/1839532273 >> 172.21.15.96:0/1839532273 conn(0x559f577a5d10 msgr2=0x559f577ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:26.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 -- 172.21.15.96:0/1839532273 shutdown_connections 2024-09-10T23:02:26.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 -- 172.21.15.96:0/1839532273 wait complete. 2024-09-10T23:02:26.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.992+0000 7f9018e61740 1 -- start start 2024-09-10T23:02:26.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 Processor -- start 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a7ee0 0x559f577bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90101c6e90 con 0x559f577a8800 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90101c68b0 con 0x559f577a7ee0 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f901010e930 con 0x559f577b4790 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f900ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9018e61740 10 client.0 _ll_register_callbacks cb 0x7f9008009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f900ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45404/0 (socket says 172.21.15.96:45404) 2024-09-10T23:02:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f900ffff640 1 -- 172.21.15.96:0/632852459 learned_addr learned my addr 172.21.15.96:0/632852459 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.993+0000 7f9015029640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9014828640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a7ee0 0x559f577bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9015029640 1 -- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a7ee0 msgr2=0x559f577bce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9015029640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a7ee0 0x559f577bce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9015029640 1 -- 172.21.15.96:0/632852459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 msgr2=0x559f577b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9015029640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.994+0000 7f9015029640 1 -- 172.21.15.96:0/632852459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f577ba070 con 0x559f577b4790 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f9015029640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b9890 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9008009c70 tx=0x7f900800e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50164ce8902a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f8feffff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900800ce50 con 0x559f577b4790 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90040abee0 con 0x559f577b4790 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9008004b20 con 0x559f577b4790 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9008010740 con 0x559f577b4790 2024-09-10T23:02:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f9018e61740 10 client.28435 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.996+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f578324d0 con 0x559f577b4790 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90040abee0 con 0x559f577b4790 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9008039030 con 0x559f577b4790 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 10 client.28435 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 10 client.28435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 20 client.28435 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f57832ca0 0x559f578350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 4 client.28435 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f5771c860 con 0x559f57832ca0 2024-09-10T23:02:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.997+0000 7f9014828640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f57832ca0 0x559f578350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.998+0000 7f9014828640 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f57832ca0 0x559f578350c0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aafbd7984c44b6c server_cookie=b26b743f0f8e9b6a in_seq=0 out_seq=0 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:26.999+0000 7f8feffff640 10 client.28435 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.000+0000 7f8feffff640 1 -- 172.21.15.96:0/632852459 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f5771c860 con 0x559f57832ca0 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.000+0000 7f8feffff640 10 client.28435 handle_command_reply: tid=0 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.000+0000 7f9018e61740 1 client.28435 shutdown 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.000+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 msgr2=0x559f577b9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.000+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b9890 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9008009c70 tx=0x7f900800e990 comp rx=0 tx=0).stop 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 shutdown_connections 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x559f57832ca0 0x559f578350c0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f577b4790 0x559f577b9890 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f8feffff640 0 client.28435 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f577a8800 0x559f577b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 --2- 172.21.15.96:0/632852459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f577a7ee0 0x559f577bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 >> 172.21.15.96:0/632852459 conn(0x559f577a5d10 msgr2=0x559f577a4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 shutdown_connections 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 1 -- 172.21.15.96:0/632852459 wait complete. 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.001+0000 7f9018e61740 20 client.28435 trim_cache size 0 max 16384 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "9cb25013-2658-4000-80d3-47238c6a5a9d", 2024-09-10T23:02:27.004 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-09-10T23:02:27.005 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 msgr2=0x7f90000b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 0x7f90000b79b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9008005c40 tx=0x7f9008009ca0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 msgr2=0x7f9010074860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f900400c890 tx=0x7f900400cd50 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 shutdown_connections 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f90000b54f0 0x7f90000b79b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9010106ed0 0x7f901006dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90101023b0 0x7f9010074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 --2- 172.21.15.96:0/3281186353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010101a90 0x7f9010074860 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 >> 172.21.15.96:0/3281186353 conn(0x7f90100fd700 msgr2=0x7f9010104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 shutdown_connections 2024-09-10T23:02:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:27.002+0000 7f9016a8c640 1 -- 172.21.15.96:0/3281186353 wait complete. 2024-09-10T23:02:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-10T23:02:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: scrub queued for path: / 2024-09-10T23:02:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:27.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:02:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: scrub summary: active paths [/] 2024-09-10T23:02:27.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:27 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: scrub queued for path: / 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: scrub summary: active paths [/] 2024-09-10T23:02:27.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:27 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: scrub queued for path: / 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: scrub summary: active paths [/] 2024-09-10T23:02:27.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:27 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:28.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:02:28.519 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:02:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:28 smithi096 ceph-mon[30869]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:28.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:28 smithi121 ceph-mon[35119]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:28.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:28 smithi117 ceph-mon[35081]: scrub summary: idle+waiting paths [/] 2024-09-10T23:02:29.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 -- 172.21.15.96:0/421031750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 msgr2=0x7f98b4117e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/421031750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4117e00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009ca0 tx=0x7f98ac02a940 comp rx=0 tx=0).stop 2024-09-10T23:02:29.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 -- 172.21.15.96:0/421031750 shutdown_connections 2024-09-10T23:02:29.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/421031750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4117e00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/421031750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410aa70 0x7f98b41154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/421031750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410a150 0x7f98b410a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.106+0000 7f98bc2c4640 1 -- 172.21.15.96:0/421031750 >> 172.21.15.96:0/421031750 conn(0x7f98b406dfc0 msgr2=0x7f98b406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 -- 172.21.15.96:0/421031750 shutdown_connections 2024-09-10T23:02:29.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 -- 172.21.15.96:0/421031750 wait complete. 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 Processor -- start 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 -- start start 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410a150 0x7f98b41153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.107+0000 7f98bc2c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410aa70 0x7f98b4112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98bc2c4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98bc2c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410ce50 con 0x7f98b410a150 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98bc2c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410ccd0 con 0x7f98b410aa70 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98bc2c4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410cfd0 con 0x7f98b4115a10 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51522/0 (socket says 172.21.15.96:51522) 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 -- 172.21.15.96:0/3868541767 learned_addr learned my addr 172.21.15.96:0/3868541767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba060640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410a150 0x7f98b41153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98b985f640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410aa70 0x7f98b4112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 -- 172.21.15.96:0/3868541767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410aa70 msgr2=0x7f98b4112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410aa70 0x7f98b4112620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 -- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410a150 msgr2=0x7f98b41153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410a150 0x7f98b41153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 -- 172.21.15.96:0/3868541767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b41b42e0 con 0x7f98b4115a10 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.108+0000 7f98ba861640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02aeb0 tx=0x7f98ac003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.109+0000 7f98ab7fe640 1 -- 172.21.15.96:0/3868541767 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac034040 con 0x7f98b4115a10 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.109+0000 7f98ab7fe640 1 -- 172.21.15.96:0/3868541767 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac031e80 con 0x7f98b4115a10 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.109+0000 7f98ab7fe640 1 -- 172.21.15.96:0/3868541767 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004690 con 0x7f98b4115a10 2024-09-10T23:02:29.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.109+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b41b4570 con 0x7f98b4115a10 2024-09-10T23:02:29.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.109+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41b49d0 con 0x7f98b4115a10 2024-09-10T23:02:29.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.114+0000 7f98ab7fe640 1 -- 172.21.15.96:0/3868541767 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f98ac02f070 con 0x7f98b4115a10 2024-09-10T23:02:29.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.115+0000 7f98ab7fe640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 0x7f98a00b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.115+0000 7f98be699740 1 Processor -- start 2024-09-10T23:02:29.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.115+0000 7f98ba060640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 0x7f98a00b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.115+0000 7f98ab7fe640 1 -- 172.21.15.96:0/3868541767 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98ac106b00 con 0x7f98b4115a10 2024-09-10T23:02:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.115+0000 7f98be699740 1 -- start start 2024-09-10T23:02:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba060640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 0x7f98a00b79b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9480 0x563a67bd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9da0 0x563a67be57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a67b4fe50 con 0x563a67bd9da0 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a67b3f280 con 0x563a67bd9480 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98be699740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563a67b43090 con 0x563a67be5d30 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98b985f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9da0 0x563a67be57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51530/0 (socket says 172.21.15.96:51530) 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 -- 172.21.15.96:0/2161369811 learned_addr learned my addr 172.21.15.96:0/2161369811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 -- 172.21.15.96:0/2161369811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9480 msgr2=0x563a67bd9860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9480 0x563a67bd9860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 -- 172.21.15.96:0/2161369811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9da0 msgr2=0x563a67be57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9da0 0x563a67be57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.116+0000 7f98ba861640 1 -- 172.21.15.96:0/2161369811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6789d650 con 0x563a67be5d30 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.117+0000 7f98ba861640 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0cd250 tx=0x7f98ac003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2030f15b59a7f99f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.117+0000 7f98a8de6640 1 -- 172.21.15.96:0/2161369811 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0ccdd0 con 0x563a67be5d30 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.117+0000 7f98a8de6640 1 -- 172.21.15.96:0/2161369811 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac0cd9c0 con 0x563a67be5d30 2024-09-10T23:02:29.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.117+0000 7f98a8de6640 1 -- 172.21.15.96:0/2161369811 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0f02e0 con 0x563a67be5d30 2024-09-10T23:02:29.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.119+0000 7f98be699740 1 -- 172.21.15.96:0/2161369811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 msgr2=0x563a67be8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0cd250 tx=0x7f98ac003f10 comp rx=0 tx=0).stop 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 -- 172.21.15.96:0/2161369811 shutdown_connections 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be8120 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9da0 0x563a67be57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 --2- 172.21.15.96:0/2161369811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9480 0x563a67bd9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 -- 172.21.15.96:0/2161369811 >> 172.21.15.96:0/2161369811 conn(0x563a67bd72b0 msgr2=0x563a67bd8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 -- 172.21.15.96:0/2161369811 shutdown_connections 2024-09-10T23:02:29.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.120+0000 7f98be699740 1 -- 172.21.15.96:0/2161369811 wait complete. 2024-09-10T23:02:29.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.121+0000 7f98be699740 1 -- start start 2024-09-10T23:02:29.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.121+0000 7f98be699740 1 Processor -- start 2024-09-10T23:02:29.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.121+0000 7f98be699740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67be5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 0x563a67be2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 0x563a67be3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a679a0700 con 0x563a67bd9480 2024-09-10T23:02:29.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a67ada110 con 0x563a67be5d30 2024-09-10T23:02:29.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563a67be8770 con 0x563a67bd9da0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98ba861640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 0x563a67be3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98ba861640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 0x563a67be3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44456/0 (socket says 172.21.15.96:44456) 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 0x563a67be2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98ba861640 1 -- 172.21.15.96:0/3961674757 learned_addr learned my addr 172.21.15.96:0/3961674757 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98be699740 10 client.0 _ll_register_callbacks cb 0x563a67af3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98ba060640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67be5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 -- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 msgr2=0x563a67be3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 0x563a67be3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 -- 172.21.15.96:0/3961674757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 msgr2=0x563a67be5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67be5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 -- 172.21.15.96:0/3961674757 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a67b07c40 con 0x563a67bd9da0 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98ba060640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67be5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:29.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f98b985f640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 0x563a67be2ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x563a67bd8190 tx=0x7f98a4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9b199a0ef19ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.122+0000 7f989dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4012860 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac0cd9c0 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98a4012a00 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f98a4012c00 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 10 client.28447 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a67bf4090 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac0cd9c0 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f98a400f040 con 0x563a67bd9da0 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 10 client.28447 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 10 client.28447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 20 client.28447 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67bf79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 4 client.28447 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:29.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.123+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a67b51da0 con 0x563a67bf5550 2024-09-10T23:02:29.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.124+0000 7f98ba060640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67bf79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.124+0000 7f989dffb640 10 client.28447 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:29.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.124+0000 7f98ba060640 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67bf79a0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28183e2f7ed4f4a8 server_cookie=31e82e8bbcefcef6 in_seq=0 out_seq=0 2024-09-10T23:02:29.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f989dffb640 1 -- 172.21.15.96:0/3961674757 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a67b51da0 con 0x563a67bf5550 2024-09-10T23:02:29.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f989dffb640 10 client.28447 handle_command_reply: tid=0 2024-09-10T23:02:29.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 client.28447 shutdown 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 msgr2=0x563a67be2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 0x563a67be2ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x563a67bd8190 tx=0x7f98a4010800 comp rx=0 tx=0).stop 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 shutdown_connections 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67bf79a0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67be5d30 0x563a67be3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f989dffb640 0 client.28447 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67bd9da0 0x563a67be2ac0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 --2- 172.21.15.96:0/3961674757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67be5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.125+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 >> 172.21.15.96:0/3961674757 conn(0x563a67bd72b0 msgr2=0x563a67bdce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.126+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 shutdown_connections 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.126+0000 7f98be699740 1 -- 172.21.15.96:0/3961674757 wait complete. 2024-09-10T23:02:29.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.126+0000 7f98be699740 20 client.28447 trim_cache size 0 max 16384 2024-09-10T23:02:29.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.146+0000 7f98be699740 1 Processor -- start 2024-09-10T23:02:29.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.146+0000 7f98be699740 1 -- start start 2024-09-10T23:02:29.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.146+0000 7f98be699740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98be699740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67be1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98be699740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98be699740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41bd080 con 0x563a67bd9480 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98be699740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41bebc0 con 0x563a67bd9da0 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98ba861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51546/0 (socket says 172.21.15.96:51546) 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98ba861640 1 -- 172.21.15.96:0/2373462950 learned_addr learned my addr 172.21.15.96:0/2373462950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.147+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41bd750 con 0x563a67be5d30 2024-09-10T23:02:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 -- 172.21.15.96:0/2373462950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 msgr2=0x563a67be1fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T23:02:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67be1fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 -- 172.21.15.96:0/2373462950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 msgr2=0x563a67bf18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 -- 172.21.15.96:0/2373462950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a67be39e0 con 0x563a67be5d30 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f98ba861640 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0c9890 tx=0x7f98ac0c9290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41511a08e1c5298d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.148+0000 7f989ffff640 1 -- 172.21.15.96:0/2373462950 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0ccdd0 con 0x563a67be5d30 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.149+0000 7f989ffff640 1 -- 172.21.15.96:0/2373462950 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac0f4020 con 0x563a67be5d30 2024-09-10T23:02:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.149+0000 7f989ffff640 1 -- 172.21.15.96:0/2373462950 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0f0110 con 0x563a67be5d30 2024-09-10T23:02:29.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 msgr2=0x563a67be1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0c9890 tx=0x7f98ac0c9290 comp rx=0 tx=0).stop 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 shutdown_connections 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67be1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be1340 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 --2- 172.21.15.96:0/2373462950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bf18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 >> 172.21.15.96:0/2373462950 conn(0x563a67bd72b0 msgr2=0x563a67bddd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 shutdown_connections 2024-09-10T23:02:29.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- 172.21.15.96:0/2373462950 wait complete. 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.150+0000 7f98be699740 1 -- start start 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 Processor -- start 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67bede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98ba060640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41be4f0 con 0x563a67bd9480 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41bde20 con 0x563a67bd9da0 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98b985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67bede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98b985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67bede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.96:44458/0 (socket says 172.21.15.96:44458) 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98b985f640 1 -- 172.21.15.96:0/2949216201 learned_addr learned my addr 172.21.15.96:0/2949216201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98ba861640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.151+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41bb060 con 0x563a67be5d30 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98be699740 10 client.0 _ll_register_callbacks cb 0x7f98a4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:29.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 -- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 msgr2=0x563a67bede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67bede80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 -- 172.21.15.96:0/2949216201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 msgr2=0x563a67bed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 -- 172.21.15.96:0/2949216201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a67be8e80 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba060640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98ba861640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be88f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02a980 tx=0x7f98ac0c9760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6815865499f7c4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f989dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0f0320 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f98ac0cc520 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.152+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac0f4020 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f98ac0cd9c0 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f98be699740 10 client.28453 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a67c62d20 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98ac0cd2c0 con 0x563a67be5d30 2024-09-10T23:02:29.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f98ac0d2f30 con 0x563a67be5d30 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f98be699740 10 client.28453 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.153+0000 7f98be699740 10 client.28453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98be699740 20 client.28453 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67c65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98be699740 4 client.28453 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f98ac0f4020 con 0x563a67bf5550 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98ba060640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67c65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f98ba060640 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67c65070 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c848e57f3463c server_cookie=5996f1c3235a4f82 in_seq=0 out_seq=0 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.154+0000 7f989dffb640 10 client.28453 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f989dffb640 1 -- 172.21.15.96:0/2949216201 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x7f98ac0f4020 con 0x563a67bf5550 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f989dffb640 10 client.28453 handle_command_reply: tid=0 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 client.28453 shutdown 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 msgr2=0x563a67be88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be88f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02a980 tx=0x7f98ac0c9760 comp rx=0 tx=0).stop 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 shutdown_connections 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x563a67bf5550 0x563a67c65070 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f989dffb640 0 client.28453 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a67bd9da0 0x563a67bede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563a67be5d30 0x563a67be88f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 --2- 172.21.15.96:0/2949216201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a67bd9480 0x563a67bed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 >> 172.21.15.96:0/2949216201 conn(0x563a67bd72b0 msgr2=0x563a67be12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 shutdown_connections 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 1 -- 172.21.15.96:0/2949216201 wait complete. 2024-09-10T23:02:29.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.155+0000 7f98be699740 20 client.28453 trim_cache size 0 max 16384 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "9cb25013-2658-4000-80d3-47238c6a5a9d": { 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "9cb25013-2658-4000-80d3-47238c6a5a9d", 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "2df1ad6c-0476-4e75-81a3-b22370ae9b18": { 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:01:36.980195+0000", 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:02:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "9cb25013-2658-4000-80d3-47238c6a5a9d": { 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:02:26.999221+0000", 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:02:29.168 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:02:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 msgr2=0x7f98a00b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 0x7f98a00b79b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b000b590 comp rx=0 tx=0).stop 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 msgr2=0x7f98b4112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f98ac02aeb0 tx=0x7f98ac003ba0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 shutdown_connections 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98b410aa70 0x7f98b4112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f98a00b54f0 0x7f98a00b79b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b4115a10 0x7f98b4112d20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 --2- 172.21.15.96:0/3868541767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98b410a150 0x7f98b41153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 >> 172.21.15.96:0/3868541767 conn(0x7f98b406dfc0 msgr2=0x7f98b4071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 shutdown_connections 2024-09-10T23:02:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:02:29.156+0000 7f98bc2c4640 1 -- 172.21.15.96:0/3868541767 wait complete. 2024-09-10T23:02:29.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:29 smithi096 ceph-mon[30869]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-09-10T23:02:29.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:29 smithi121 ceph-mon[35119]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-09-10T23:02:29.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:29 smithi117 ceph-mon[35081]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-09-10T23:02:30.130 INFO:tasks.cephfs.filesystem:scrub status for tag:9cb25013-2658-4000-80d3-47238c6a5a9d - {'path': '/', 'tag': '9cb25013-2658-4000-80d3-47238c6a5a9d', 'options': 'recursive,force'} 2024-09-10T23:02:30.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 895.0 s, 1705.8 tps, lat 57.740 ms stddev 71.090 2024-09-10T23:02:31.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:31 smithi096 ceph-mon[30869]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.0 MiB/s wr, 783 op/s 2024-09-10T23:02:31.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:31 smithi121 ceph-mon[35119]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.0 MiB/s wr, 783 op/s 2024-09-10T23:02:31.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:31 smithi117 ceph-mon[35081]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.0 MiB/s wr, 783 op/s 2024-09-10T23:02:33.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:33 smithi121 ceph-mon[35119]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 747 op/s 2024-09-10T23:02:33.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:33 smithi117 ceph-mon[35081]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 747 op/s 2024-09-10T23:02:34.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:33 smithi096 ceph-mon[30869]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.6 MiB/s wr, 747 op/s 2024-09-10T23:02:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.161+0000 7f9e773ad640 1 -- 172.21.15.96:0/2184476230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010ab10 msgr2=0x7f9e7011a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.161+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2184476230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010ab10 0x7f9e7011a090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009ca0 tx=0x7f9e6002a8f0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.167+0000 7f9e773ad640 1 -- 172.21.15.96:0/2184476230 shutdown_connections 2024-09-10T23:02:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.167+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2184476230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010ab10 0x7f9e7011a090 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.167+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2184476230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e7010a150 0x7f9e7010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.167+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2184476230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e70074290 0x7f9e70074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.167+0000 7f9e773ad640 1 -- 172.21.15.96:0/2184476230 >> 172.21.15.96:0/2184476230 conn(0x7f9e7006fa40 msgr2=0x7f9e70071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.171+0000 7f9e773ad640 1 -- 172.21.15.96:0/2184476230 shutdown_connections 2024-09-10T23:02:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.174+0000 7f9e773ad640 1 -- 172.21.15.96:0/2184476230 wait complete. 2024-09-10T23:02:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.175+0000 7f9e773ad640 1 Processor -- start 2024-09-10T23:02:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.178+0000 7f9e773ad640 1 -- start start 2024-09-10T23:02:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010a150 0x7f9e701a8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45478/0 (socket says 172.21.15.96:45478) 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e75149640 1 -- 172.21.15.96:0/2330730290 learned_addr learned my addr 172.21.15.96:0/2330730290 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 0x7f9e701a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7011d3a0 con 0x7f9e70074290 2024-09-10T23:02:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7011d220 con 0x7f9e7010a150 2024-09-10T23:02:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.179+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7011d520 con 0x7f9e7010ab10 2024-09-10T23:02:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.182+0000 7f9e7594a640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 0x7f9e701a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e74948640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010a150 0x7f9e701a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e7594a640 1 -- 172.21.15.96:0/2330730290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010a150 msgr2=0x7f9e701a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e7594a640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010a150 0x7f9e701a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e7594a640 1 -- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 msgr2=0x7f9e701a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e7594a640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.183+0000 7f9e7594a640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e701a9600 con 0x7f9e7010ab10 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.184+0000 7f9e75149640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:02:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.184+0000 7f9e7594a640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 0x7f9e701a8d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009bf0 tx=0x7f9e6002f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.185+0000 7f9e6e7fc640 1 -- 172.21.15.96:0/2330730290 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002fd70 con 0x7f9e7010ab10 2024-09-10T23:02:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.185+0000 7f9e6e7fc640 1 -- 172.21.15.96:0/2330730290 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6002d070 con 0x7f9e7010ab10 2024-09-10T23:02:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.185+0000 7f9e6e7fc640 1 -- 172.21.15.96:0/2330730290 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6003b660 con 0x7f9e7010ab10 2024-09-10T23:02:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.186+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701a9890 con 0x7f9e7010ab10 2024-09-10T23:02:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.186+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701afde0 con 0x7f9e7010ab10 2024-09-10T23:02:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.190+0000 7f9e6e7fc640 1 -- 172.21.15.96:0/2330730290 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f9e600305f0 con 0x7f9e7010ab10 2024-09-10T23:02:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.193+0000 7f9e6e7fc640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 0x7f9e580b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.193+0000 7f9e7a257740 1 Processor -- start 2024-09-10T23:02:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.193+0000 7f9e6e7fc640 1 -- 172.21.15.96:0/2330730290 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9e60106fa0 con 0x7f9e7010ab10 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.193+0000 7f9e7a257740 1 -- start start 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.193+0000 7f9e75149640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 0x7f9e580b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.194+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 0x564ef9848c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.194+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef984b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.194+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef984e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.194+0000 7f9e7a257740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ef97b86f0 con 0x564ef9849190 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e7a257740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ef97a7b20 con 0x564ef984bbd0 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e7a257740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564ef97ab930 con 0x564ef9846860 2024-09-10T23:02:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e74948640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef984b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e74948640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef984b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45494/0 (socket says 172.21.15.96:45494) 2024-09-10T23:02:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e74948640 1 -- 172.21.15.96:0/947918382 learned_addr learned my addr 172.21.15.96:0/947918382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 0x564ef9848c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 0x7f9e580b7930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9e68005fd0 tx=0x7f9e6800b340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 -- 172.21.15.96:0/947918382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 msgr2=0x564ef984e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef984e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 -- 172.21.15.96:0/947918382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 msgr2=0x564ef984b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef984b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 -- 172.21.15.96:0/947918382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef976d530 con 0x564ef9846860 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.195+0000 7f9e75149640 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 0x564ef9848c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800fbc0 tx=0x7f9e68012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822d539ad2e591e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.196+0000 7f9e577fe640 1 -- 172.21.15.96:0/947918382 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68022020 con 0x564ef9846860 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.196+0000 7f9e577fe640 1 -- 172.21.15.96:0/947918382 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e68004230 con 0x564ef9846860 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.196+0000 7f9e577fe640 1 -- 172.21.15.96:0/947918382 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6801e7c0 con 0x564ef9846860 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 -- 172.21.15.96:0/947918382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 msgr2=0x564ef9848c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 0x564ef9848c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800fbc0 tx=0x7f9e68012000 comp rx=0 tx=0).stop 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 -- 172.21.15.96:0/947918382 shutdown_connections 2024-09-10T23:02:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef984e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef984b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 --2- 172.21.15.96:0/947918382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9846860 0x564ef9848c50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.197+0000 7f9e7a257740 1 -- 172.21.15.96:0/947918382 >> 172.21.15.96:0/947918382 conn(0x564ef98419b0 msgr2=0x564ef9842740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.198+0000 7f9e7a257740 1 -- 172.21.15.96:0/947918382 shutdown_connections 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.199+0000 7f9e7a257740 1 -- 172.21.15.96:0/947918382 wait complete. 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.199+0000 7f9e7a257740 1 -- start start 2024-09-10T23:02:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.199+0000 7f9e7a257740 1 Processor -- start 2024-09-10T23:02:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.200+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9846860 0x564ef9850040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.200+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef98506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.200+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7a257740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ef96dfab0 con 0x564ef9849190 2024-09-10T23:02:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7a257740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ef974d730 con 0x564ef9846860 2024-09-10T23:02:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:51590/0 (socket says 172.21.15.96:51590) 2024-09-10T23:02:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 -- 172.21.15.96:0/1880010657 learned_addr learned my addr 172.21.15.96:0/1880010657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e75149640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9846860 0x564ef9850040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564ef984ebb0 con 0x564ef984bbd0 2024-09-10T23:02:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7a257740 10 client.0 _ll_register_callbacks cb 0x564ef97a16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 -- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9846860 msgr2=0x564ef9850040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9846860 0x564ef9850040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 -- 172.21.15.96:0/1880010657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 msgr2=0x564ef98506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef98506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.201+0000 7f9e7594a640 1 -- 172.21.15.96:0/1880010657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef9509790 con 0x564ef984bbd0 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e7594a640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd6f0 tx=0x7f9e600cd5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c037ce7a1ecd72ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e68004230 con 0x564ef984bbd0 2024-09-10T23:02:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e54ff9640 1 -- 172.21.15.96:0/1880010657 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e600c7e60 con 0x564ef984bbd0 2024-09-10T23:02:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e54ff9640 1 -- 172.21.15.96:0/1880010657 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9e600d6a60 con 0x564ef984bbd0 2024-09-10T23:02:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.202+0000 7f9e54ff9640 1 -- 172.21.15.96:0/1880010657 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e600cde60 con 0x564ef984bbd0 2024-09-10T23:02:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 10 client.28465 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=32}) -- 0x564ef9582620 con 0x564ef984bbd0 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e54ff9640 1 -- 172.21.15.96:0/1880010657 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9e60004b40 con 0x564ef984bbd0 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 10 client.28465 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 10 client.28465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 20 client.28465 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.203+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef985dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.204+0000 7f9e7a257740 4 client.28465 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.204+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e68004230 con 0x564ef985b630 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.204+0000 7f9e75149640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef985dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.204+0000 7f9e54ff9640 10 client.28465 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.204+0000 7f9e75149640 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef985dab0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6582d7454ff0e7 server_cookie=343e686309779bc in_seq=0 out_seq=0 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e54ff9640 1 -- 172.21.15.96:0/1880010657 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f9e68004230 con 0x564ef985b630 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e54ff9640 10 client.28465 handle_command_reply: tid=0 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 client.28465 shutdown 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 msgr2=0x564ef98512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd6f0 tx=0x7f9e600cd5f0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 shutdown_connections 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef985dab0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e54ff9640 0 client.28465 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef98512b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9849190 0x564ef98506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 --2- 172.21.15.96:0/1880010657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9846860 0x564ef9850040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.206+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 >> 172.21.15.96:0/1880010657 conn(0x564ef98419b0 msgr2=0x564ef9840bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.207+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 shutdown_connections 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.207+0000 7f9e7a257740 1 -- 172.21.15.96:0/1880010657 wait complete. 2024-09-10T23:02:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.207+0000 7f9e7a257740 20 client.28465 trim_cache size 0 max 16384 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.231+0000 7f9e7a257740 1 Processor -- start 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.231+0000 7f9e7a257740 1 -- start start 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9849190 0x564ef984f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef984fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00e370 con 0x564ef9846860 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00feb0 con 0x564ef9849190 2024-09-10T23:02:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7a257740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00ea40 con 0x564ef984bbd0 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45512/0 (socket says 172.21.15.96:45512) 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 -- 172.21.15.96:0/4082200113 learned_addr learned my addr 172.21.15.96:0/4082200113 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e74948640 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9849190 0x564ef984f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e7594a640 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef984fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 -- 172.21.15.96:0/4082200113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 msgr2=0x564ef984fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef984fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 -- 172.21.15.96:0/4082200113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9849190 msgr2=0x564ef984f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9849190 0x564ef984f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 -- 172.21.15.96:0/4082200113 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef9850a50 con 0x564ef9846860 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e75149640 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800bff0 tx=0x7f9e6800ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd8509171c270e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e56ffd640 1 -- 172.21.15.96:0/4082200113 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68015040 con 0x564ef9846860 2024-09-10T23:02:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e56ffd640 1 -- 172.21.15.96:0/4082200113 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e68005560 con 0x564ef9846860 2024-09-10T23:02:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.232+0000 7f9e56ffd640 1 -- 172.21.15.96:0/4082200113 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68022040 con 0x564ef9846860 2024-09-10T23:02:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.233+0000 7f9e7a257740 1 -- 172.21.15.96:0/4082200113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 msgr2=0x564ef984efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.233+0000 7f9e7a257740 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800bff0 tx=0x7f9e6800ba20 comp rx=0 tx=0).stop 2024-09-10T23:02:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.234+0000 7f9e7a257740 1 -- 172.21.15.96:0/4082200113 shutdown_connections 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.234+0000 7f9e7a257740 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef9849190 0x564ef984f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.234+0000 7f9e7a257740 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef984bbd0 0x564ef984fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.234+0000 7f9e7a257740 1 --2- 172.21.15.96:0/4082200113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef984efe0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.234+0000 7f9e7a257740 1 -- 172.21.15.96:0/4082200113 >> 172.21.15.96:0/4082200113 conn(0x564ef98419b0 msgr2=0x564ef984c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.235+0000 7f9e7a257740 1 -- 172.21.15.96:0/4082200113 shutdown_connections 2024-09-10T23:02:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.235+0000 7f9e7a257740 1 -- 172.21.15.96:0/4082200113 wait complete. 2024-09-10T23:02:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 -- start start 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 Processor -- start 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9849190 0x564ef98566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef9856c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00f7e0 con 0x564ef9846860 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00f110 con 0x564ef984bbd0 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c00c350 con 0x564ef9849190 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7a257740 10 client.0 _ll_register_callbacks cb 0x7f9e60005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e75149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45526/0 (socket says 172.21.15.96:45526) 2024-09-10T23:02:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e75149640 1 -- 172.21.15.96:0/722214871 learned_addr learned my addr 172.21.15.96:0/722214871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e7594a640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef9856c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.236+0000 7f9e74948640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9849190 0x564ef98566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 -- 172.21.15.96:0/722214871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9849190 msgr2=0x564ef98566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9849190 0x564ef98566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 -- 172.21.15.96:0/722214871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 msgr2=0x564ef98579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 -- 172.21.15.96:0/722214871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef9848fc0 con 0x564ef984bbd0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e75149640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7594a640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef9856c10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd460 tx=0x7f9e60030d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca40be2f98260b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e600c7e60 con 0x564ef984bbd0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f9e600cc9c0 con 0x564ef984bbd0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e68005560 con 0x564ef984bbd0 2024-09-10T23:02:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e600cd760 con 0x564ef984bbd0 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7a257740 10 client.28385 fetch_fsmap learned FSMap version 32 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564ef9855ff0 con 0x564ef984bbd0 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e68005560 con 0x564ef984bbd0 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.237+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9e600cc430 con 0x564ef984bbd0 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 10 client.28385 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 10 client.28385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 20 client.28385 populate_metadata read hostname 'smithi096' 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef98cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 4 client.28385 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ef97ba640 con 0x564ef985b630 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e75149640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef98cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e54ff9640 10 client.28385 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.238+0000 7f9e75149640 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef98cab40 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a0f68b8cb3fce7 server_cookie=600de260dedc7c2a in_seq=0 out_seq=0 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e54ff9640 1 -- 172.21.15.96:0/722214871 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ef97ba640 con 0x564ef985b630 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e54ff9640 10 client.28385 handle_command_reply: tid=0 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 client.28385 shutdown 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 msgr2=0x564ef9856c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef9856c10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e600cd460 tx=0x7f9e60030d60 comp rx=0 tx=0).stop 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 shutdown_connections 2024-09-10T23:02:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564ef985b630 0x564ef98cab40 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ef984bbd0 0x564ef9856c10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ef9849190 0x564ef98566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 --2- 172.21.15.96:0/722214871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ef9846860 0x564ef98579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e54ff9640 0 client.28385 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 >> 172.21.15.96:0/722214871 conn(0x564ef98419b0 msgr2=0x564ef984efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 shutdown_connections 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 1 -- 172.21.15.96:0/722214871 wait complete. 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.265+0000 7f9e7a257740 20 client.28385 trim_cache size 0 max 16384 2024-09-10T23:02:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-09-10T23:02:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-09-10T23:02:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-09-10T23:02:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 msgr2=0x7f9e580b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 0x7f9e580b7930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9e68005fd0 tx=0x7f9e6800b340 comp rx=0 tx=0).stop 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 msgr2=0x7f9e701a8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 0x7f9e701a8d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009bf0 tx=0x7f9e6002f870 comp rx=0 tx=0).stop 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 shutdown_connections 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e7010a150 0x7f9e701a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f9e580b5470 0x7f9e580b7930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e7010ab10 0x7f9e701a8d40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 --2- 172.21.15.96:0/2330730290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e70074290 0x7f9e701a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:02:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 >> 172.21.15.96:0/2330730290 conn(0x7f9e7006fa40 msgr2=0x7f9e70071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:02:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 shutdown_connections 2024-09-10T23:02:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-09-10T23:02:34.266+0000 7f9e773ad640 1 -- 172.21.15.96:0/2330730290 wait complete. 2024-09-10T23:02:34.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:34 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:02:34.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:34 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:02:35.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:34 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:02:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:35 smithi121 ceph-mon[35119]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 668 op/s 2024-09-10T23:02:35.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:35 smithi121 ceph-mon[35119]: scrub summary: idle 2024-09-10T23:02:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:35 smithi117 ceph-mon[35081]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 668 op/s 2024-09-10T23:02:35.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:35 smithi117 ceph-mon[35081]: scrub summary: idle 2024-09-10T23:02:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:progress: 900.0 s, 1626.4 tps, lat 61.471 ms stddev 72.689 2024-09-10T23:02:36.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:35 smithi096 ceph-mon[30869]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.6 MiB/s wr, 668 op/s 2024-09-10T23:02:36.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:35 smithi096 ceph-mon[30869]: scrub summary: idle 2024-09-10T23:02:36.150 INFO:teuthology.orchestra.run.smithi096.stdout:transaction type: 2024-09-10T23:02:36.150 INFO:teuthology.orchestra.run.smithi096.stdout:scaling factor: 32 2024-09-10T23:02:36.150 INFO:teuthology.orchestra.run.smithi096.stdout:query mode: simple 2024-09-10T23:02:36.150 INFO:teuthology.orchestra.run.smithi096.stdout:number of clients: 100 2024-09-10T23:02:36.150 INFO:teuthology.orchestra.run.smithi096.stdout:number of threads: 4 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:duration: 900 s 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:number of transactions actually processed: 2323089 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:latency average = 38.738 ms 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:latency stddev = 47.737 ms 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:tps = 2580.641135 (including connections establishing) 2024-09-10T23:02:36.151 INFO:teuthology.orchestra.run.smithi096.stdout:tps = 2580.647293 (excluding connections establishing) 2024-09-10T23:02:36.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-10T23:02:37.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:37 smithi117 ceph-mon[35081]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-09-10T23:02:38.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:37 smithi096 ceph-mon[30869]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-09-10T23:02:38.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:37 smithi121 ceph-mon[35119]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 7.0 MiB/s wr, 806 op/s 2024-09-10T23:02:40.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:39 smithi096 ceph-mon[30869]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-09-10T23:02:40.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:39 smithi121 ceph-mon[35119]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-09-10T23:02:40.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:39 smithi117 ceph-mon[35081]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.9 MiB/s wr, 689 op/s 2024-09-10T23:02:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:41 smithi096 ceph-mon[30869]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 8.7 MiB/s wr, 981 op/s 2024-09-10T23:02:42.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:41 smithi121 ceph-mon[35119]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 8.7 MiB/s wr, 981 op/s 2024-09-10T23:02:42.025 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:41 smithi117 ceph-mon[35081]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 8.7 MiB/s wr, 981 op/s 2024-09-10T23:02:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:43 smithi096 ceph-mon[30869]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 867 op/s 2024-09-10T23:02:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:43 smithi121 ceph-mon[35119]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 867 op/s 2024-09-10T23:02:44.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:43 smithi117 ceph-mon[35081]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.0 MiB/s wr, 867 op/s 2024-09-10T23:02:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:02:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:02:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:02:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:02:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:02:45.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:02:45.543 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-10T23:02:45.694 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/: 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 data 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data: 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:total 36 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 . 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 .. 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 base 2024-09-10T23:02:45.695 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 10 22:47 current_logfiles 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 10 22:47 global 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 log 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_commit_ts 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_dynshmem 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 10 22:47 pg_hba.conf 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 10 22:47 pg_ident.conf 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 pg_logical 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 pg_multixact 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_notify 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_replslot 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_serial 2024-09-10T23:02:45.696 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_snapshots 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 pg_stat 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 23:02 pg_stat_tmp 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 23:02 pg_subtrans 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_tblspc 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pg_twophase 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 10 23:02 pg_wal 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 pg_xact 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 10 22:47 postgresql.auto.conf 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 10 22:47 postgresql.conf 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 10 22:47 postmaster.opts 2024-09-10T23:02:45.697 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base: 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 . 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 1 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 13493 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 10 22:50 13494 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 pgsql_tmp 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:total 7777 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 . 2024-09-10T23:02:45.698 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 .. 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T23:02:45.699 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 22:47 1259 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T23:02:45.700 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T23:02:45.701 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T23:02:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T23:02:45.703 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T23:02:45.704 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T23:02:45.705 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 22:47 2619 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2619_fsm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T23:02:45.706 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T23:02:45.707 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T23:02:45.708 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2696 2024-09-10T23:02:45.709 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T23:02:45.710 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T23:02:45.711 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T23:02:45.712 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T23:02:45.713 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T23:02:45.714 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T23:02:45.715 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T23:02:45.716 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T23:02:45.717 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T23:02:45.718 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T23:02:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T23:02:45.785 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T23:02:45.786 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:total 7777 2024-09-10T23:02:45.787 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 10 22:47 . 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 .. 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T23:02:45.788 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 22:47 1259 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T23:02:45.789 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T23:02:45.790 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T23:02:45.791 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T23:02:45.792 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T23:02:45.793 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T23:02:45.794 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 10 22:47 2619 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2619_fsm 2024-09-10T23:02:45.795 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T23:02:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T23:02:45.797 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T23:02:45.798 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2696 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T23:02:45.799 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T23:02:45.800 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T23:02:45.801 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T23:02:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T23:02:45.803 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T23:02:45.804 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T23:02:45.805 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T23:02:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T23:02:45.807 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T23:02:45.808 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T23:02:45.810 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T23:02:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:total 635678 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 10 22:50 . 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 .. 2024-09-10T23:02:45.812 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 112 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 113 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 1247 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1247_fsm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1247_vm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 10 22:47 1249 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1249_fsm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1249_vm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 10 22:47 1255 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1255_fsm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1255_vm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 10 23:02 1259 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1259_fsm 2024-09-10T23:02:45.813 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1259_vm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 10 22:47 13344 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13344_fsm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13344_vm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13346 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13348 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13349_fsm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13349_vm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13351 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13353 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13354_fsm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13354_vm 2024-09-10T23:02:45.814 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13356 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13358 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 13359_fsm 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13359_vm 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 13361 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 13363 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1417 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 1418 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 446668800 Sep 10 23:02 16396 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 10 23:02 16396_fsm 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:50 16396_vm 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 598016 Sep 10 23:02 16397 2024-09-10T23:02:45.815 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 23:02 16397_fsm 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 23:02 16397_vm 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1753088 Sep 10 23:02 16399 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 23:01 16399_fsm 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 23:02 16399_vm 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 23:02 16400 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 23:02 16402 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 10 23:02 16404 2024-09-10T23:02:45.816 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 121520128 Sep 10 23:02 16406 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 23:02 16406_fsm 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:58 16406_vm 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 174 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 175 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2187 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2224 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2328 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2336 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2337 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2579 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2600 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2600_fsm 2024-09-10T23:02:45.817 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2600_vm 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2601_fsm 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2601_vm 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2602 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2602_fsm 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2602_vm 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2603 2024-09-10T23:02:45.818 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2603_fsm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2603_vm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2604 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2605 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2605_fsm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2605_vm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2606_fsm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2606_vm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2607 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2607_fsm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2607_vm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 10 22:47 2608 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2608_fsm 2024-09-10T23:02:45.819 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2608_vm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 10 22:47 2609 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2609_fsm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2609_vm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2610 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2610_fsm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2610_vm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2611 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2612_fsm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2612_vm 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2613 2024-09-10T23:02:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2615_fsm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2615_vm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2616 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2616_fsm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2616_vm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 10 22:47 2617 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2617_fsm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2617_vm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 10 22:47 2618 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2618_fsm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2618_vm 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 10 23:02 2619 2024-09-10T23:02:45.821 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:49 2619_fsm 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2619_vm 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2620 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2650 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2651 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2652 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2653 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2654 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2655 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2656 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2657 2024-09-10T23:02:45.822 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 22:47 2658 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 10 22:47 2659 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2660 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2661 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2662 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2663 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2664 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2665 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2666 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2667 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2668 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2669 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2670 2024-09-10T23:02:45.823 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 10 22:47 2673 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 10 22:47 2674 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 10 22:47 2675 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2678 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2679 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2680 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2681 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2682 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2683 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2684 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2685 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2686 2024-09-10T23:02:45.824 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2687 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2688 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 10 22:47 2689 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 10 22:47 2690 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 10 22:47 2691 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2692 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2693 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:53 2696 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2699 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2701 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2702 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2703 2024-09-10T23:02:45.825 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 2704 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2753 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2753_fsm 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2753_vm 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2754 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2755 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2756 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 2757 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2830 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2831 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2832 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2833 2024-09-10T23:02:45.826 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2834 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2835 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2836 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2837 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 10 22:47 2838 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2838_fsm 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2838_vm 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2839 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2840_fsm 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2840_vm 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2841 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2995 2024-09-10T23:02:45.827 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2996 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3079_fsm 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3079_vm 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3080 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3081 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3085 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3118 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3119 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 22:47 3164 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3256 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3257 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3258 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3350 2024-09-10T23:02:45.828 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3351 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3379 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3380 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3381 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3394 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3394_fsm 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3394_vm 2024-09-10T23:02:45.829 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3395 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3429 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3430 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3431 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3433 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3439 2024-09-10T23:02:45.830 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3440 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 10 22:47 3455 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 10 22:47 3456 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3456_fsm 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3456_vm 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3466 2024-09-10T23:02:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3467 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3468 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3501 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3502 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3503 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3534 2024-09-10T23:02:45.832 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3541_fsm 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3541_vm 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3542 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3574 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3575 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3576 2024-09-10T23:02:45.833 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3596 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3597 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3598 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3599 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3600_fsm 2024-09-10T23:02:45.834 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3600_vm 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3601_fsm 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3601_vm 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3602_fsm 2024-09-10T23:02:45.835 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3602_vm 2024-09-10T23:02:45.836 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603 2024-09-10T23:02:45.836 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3603_fsm 2024-09-10T23:02:45.836 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3603_vm 2024-09-10T23:02:45.836 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3604 2024-09-10T23:02:45.836 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3605 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3606 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3607 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3608 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 10 22:47 3609 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3712 2024-09-10T23:02:45.837 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 3764_fsm 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3764_vm 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3766 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 3767 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3997 2024-09-10T23:02:45.838 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4143 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4144 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4145 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4146 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4147 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4148 2024-09-10T23:02:45.839 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4149 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4150 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4151 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4152 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4153 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4154 2024-09-10T23:02:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4155 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4156 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4157 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4158 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4159 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4160 2024-09-10T23:02:45.841 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4161 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4162 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4163 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4164 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4165 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4166 2024-09-10T23:02:45.842 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4167 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4168 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4169 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4170 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4171 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4172 2024-09-10T23:02:45.843 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4173 2024-09-10T23:02:45.844 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4174 2024-09-10T23:02:45.844 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 5002 2024-09-10T23:02:45.844 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 548 2024-09-10T23:02:45.844 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 549 2024-09-10T23:02:45.844 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6102 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6104 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6106 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6110 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6111 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6112 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6113 2024-09-10T23:02:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6117 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 826 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 827 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 828 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 10 22:47 pg_internal.init 2024-09-10T23:02:45.846 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 22:47 PG_VERSION 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 10 22:47 .. 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.847 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/global: 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:total 560 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 10 22:47 . 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1213 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1213_fsm 2024-09-10T23:02:45.848 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1213_vm 2024-09-10T23:02:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1214 2024-09-10T23:02:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1214_fsm 2024-09-10T23:02:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1214_vm 2024-09-10T23:02:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 1232 2024-09-10T23:02:45.849 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 1233 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1260 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1260_fsm 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1260_vm 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1261 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1261_fsm 2024-09-10T23:02:45.850 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1261_vm 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1262 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 1262_fsm 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 1262_vm 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2396 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 10 22:47 2396_fsm 2024-09-10T23:02:45.851 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2396_vm 2024-09-10T23:02:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2397 2024-09-10T23:02:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2671 2024-09-10T23:02:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2672 2024-09-10T23:02:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2676 2024-09-10T23:02:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2677 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2694 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2695 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2697 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 10 22:47 2698 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2846 2024-09-10T23:02:45.853 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2847 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2964 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2965 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 2966 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 2967 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 3592 2024-09-10T23:02:45.854 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 3593 2024-09-10T23:02:45.855 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4060 2024-09-10T23:02:45.855 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4061 2024-09-10T23:02:45.855 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4175 2024-09-10T23:02:45.855 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4176 2024-09-10T23:02:45.855 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4177 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4178 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4181 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4182 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4183 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4184 2024-09-10T23:02:45.856 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 4185 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 4186 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6000 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6001 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6002 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 6100 2024-09-10T23:02:45.857 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6114 2024-09-10T23:02:45.858 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 6115 2024-09-10T23:02:45.858 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 23:02 pg_control 2024-09-10T23:02:45.858 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 10 22:47 pg_filenode.map 2024-09-10T23:02:45.858 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 10 22:47 pg_internal.init 2024-09-10T23:02:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/log: 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:total 2 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Sep 10 23:02 postgresql-Tue.log 2024-09-10T23:02:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.860 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-10T23:02:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.861 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:total 1 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 . 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 mappings 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 10 23:02 replorigin_checkpoint 2024-09-10T23:02:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 snapshots 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 .. 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.863 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 .. 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-10T23:02:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.865 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 . 2024-09-10T23:02:45.865 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.865 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 members 2024-09-10T23:02:45.865 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 offsets 2024-09-10T23:02:45.865 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 .. 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:47 0000 2024-09-10T23:02:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:total 8 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 22:47 . 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 10 22:47 .. 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 10 22:50 0000 2024-09-10T23:02:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.868 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-10T23:02:45.869 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-10T23:02:45.870 INFO:teuthology.orchestra.run.smithi096.stdout:total 15 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 . 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 10 23:02 db_0.stat 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 10 23:02 db_13494.stat 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 10 23:02 global.stat 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 23:02 . 2024-09-10T23:02:45.871 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:total 120 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 10 23:02 . 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 10 23:02 0023 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.872 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-10T23:02:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:total 1048576 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 10 23:02 . 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:02 0000000100000000000000CC 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000CD 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000CE 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:57 0000000100000000000000CF 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:58 0000000100000000000000D0 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:59 0000000100000000000000D1 2024-09-10T23:02:45.874 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000D2 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000D3 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000D4 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:57 0000000100000000000000D5 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000D6 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:57 0000000100000000000000D7 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000D8 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:58 0000000100000000000000D9 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000DA 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000DB 2024-09-10T23:02:45.875 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000DC 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000DD 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000DE 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000DF 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:57 0000000100000000000000E0 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000E1 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:57 0000000100000000000000E2 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:58 0000000100000000000000E3 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000E4 2024-09-10T23:02:45.876 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000E5 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000E6 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:56 0000000100000000000000E7 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 22:59 0000000100000000000000E8 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 0000000100000000000000E9 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000EA 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000EB 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 0000000100000000000000EC 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000ED 2024-09-10T23:02:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000EE 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000EF 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F0 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 0000000100000000000000F1 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F2 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F3 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F4 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F5 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F6 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 0000000100000000000000F7 2024-09-10T23:02:45.878 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F8 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000F9 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000FA 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000FB 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 0000000100000000000000FC 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:02 0000000100000000000000FD 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:02 0000000100000000000000FE 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 0000000100000000000000FF 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000000 2024-09-10T23:02:45.879 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000001 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 000000010000000100000002 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000003 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000004 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000005 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 000000010000000100000006 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000007 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 000000010000000100000008 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:01 000000010000000100000009 2024-09-10T23:02:45.880 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 00000001000000010000000A 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 10 23:00 00000001000000010000000B 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 archive_status 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout:total 0 2024-09-10T23:02:45.881 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 10 22:47 . 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 10 23:02 .. 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout:total 568 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 10 23:02 . 2024-09-10T23:02:45.882 INFO:teuthology.orchestra.run.smithi096.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 10 23:02 .. 2024-09-10T23:02:45.883 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 10 22:52 0000 2024-09-10T23:02:45.883 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 10 23:02 0001 2024-09-10T23:02:45.883 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 10 23:02 0002 2024-09-10T23:02:45.884 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:45 smithi096 ceph-mon[30869]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T23:02:45.885 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T23:02:45.898 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T23:02:45.906 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T23:02:45.944 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T23:02:46.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:45 smithi121 ceph-mon[35119]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T23:02:46.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:45 smithi117 ceph-mon[35081]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 787 op/s 2024-09-10T23:02:47.828 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:47 smithi117 ceph-mon[35081]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-10T23:02:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:47 smithi096 ceph-mon[30869]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-10T23:02:48.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:47 smithi121 ceph-mon[35119]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-10T23:02:49.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:49 smithi096 ceph-mon[30869]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.77k op/s 2024-09-10T23:02:50.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:49 smithi121 ceph-mon[35119]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.77k op/s 2024-09-10T23:02:50.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:49 smithi117 ceph-mon[35081]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.77k op/s 2024-09-10T23:02:52.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:51 smithi096 ceph-mon[30869]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 2.08k op/s 2024-09-10T23:02:52.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:51 smithi121 ceph-mon[35119]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 2.08k op/s 2024-09-10T23:02:52.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:51 smithi117 ceph-mon[35081]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 MiB/s wr, 2.08k op/s 2024-09-10T23:02:53.816 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:53 smithi121 ceph-mon[35119]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-09-10T23:02:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:53 smithi096 ceph-mon[30869]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-09-10T23:02:54.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:53 smithi117 ceph-mon[35081]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-09-10T23:02:56.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:55 smithi096 ceph-mon[30869]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-10T23:02:56.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:55 smithi121 ceph-mon[35119]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-10T23:02:56.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:55 smithi117 ceph-mon[35081]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-10T23:02:58.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:57 smithi096 ceph-mon[30869]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-09-10T23:02:58.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:57 smithi121 ceph-mon[35119]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-09-10T23:02:58.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:57 smithi117 ceph-mon[35081]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 1.61k op/s 2024-09-10T23:03:00.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:02:59 smithi096 ceph-mon[30869]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-10T23:03:00.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:02:59 smithi121 ceph-mon[35119]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-10T23:03:00.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:02:59 smithi117 ceph-mon[35081]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-10T23:03:00.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T23:03:00.395 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 -- 172.21.15.96:0/2755982240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d54103b30 msgr2=0x7f3d54103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 --2- 172.21.15.96:0/2755982240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d54103b30 0x7f3d54103f10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009900 tx=0x7f3d5002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 -- 172.21.15.96:0/2755982240 shutdown_connections 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 --2- 172.21.15.96:0/2755982240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d5410d0e0 0x7f3d5410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 --2- 172.21.15.96:0/2755982240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54104530 0x7f3d5410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 --2- 172.21.15.96:0/2755982240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d54103b30 0x7f3d54103f10 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 -- 172.21.15.96:0/2755982240 >> 172.21.15.96:0/2755982240 conn(0x7f3d540fd700 msgr2=0x7f3d540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 -- 172.21.15.96:0/2755982240 shutdown_connections 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.905+0000 7f3d5b759640 1 -- 172.21.15.96:0/2755982240 wait complete. 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 Processor -- start 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 -- start start 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 0x7f3d541992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d54104530 0x7f3d541997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54112d60 con 0x7f3d5410d0e0 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54112be0 con 0x7f3d54104530 2024-09-10T23:03:00.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d5b759640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54112ee0 con 0x7f3d54103b30 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38228/0 (socket says 172.21.15.96:38228) 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 -- 172.21.15.96:0/230491244 learned_addr learned my addr 172.21.15.96:0/230491244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d54104530 0x7f3d541997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d594f5640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 0x7f3d541992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 -- 172.21.15.96:0/230491244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 msgr2=0x7f3d541992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 0x7f3d541992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 -- 172.21.15.96:0/230491244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d54104530 msgr2=0x7f3d541997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d54104530 0x7f3d541997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d59cf6640 1 -- 172.21.15.96:0/230491244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5419f360 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.906+0000 7f3d594f5640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 0x7f3d541992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d59cf6640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00eb10 tx=0x7f3d4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d4a7fc640 1 -- 172.21.15.96:0/230491244 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c004b20 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5419f650 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d4a7fc640 1 -- 172.21.15.96:0/230491244 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c004cc0 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d4a7fc640 1 -- 172.21.15.96:0/230491244 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c010670 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.907+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d541a5d10 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.909+0000 7f3d5db2e740 1 Processor -- start 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d4a7fc640 1 -- 172.21.15.96:0/230491244 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f3d4c005020 con 0x7f3d5410d0e0 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 -- start start 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca2523860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca2536530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ca2499e50 con 0x558ca2523da0 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ca2489280 con 0x558ca2534140 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558ca248d090 con 0x558ca2523480 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38236/0 (socket says 172.21.15.96:38236) 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d594f5640 1 -- 172.21.15.96:0/1313963946 learned_addr learned my addr 172.21.15.96:0/1313963946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca2523860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.910+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca2536530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d4a7fc640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 0x7f3d400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 -- 172.21.15.96:0/1313963946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 msgr2=0x558ca2523860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca2523860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 -- 172.21.15.96:0/1313963946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 msgr2=0x558ca2536530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca2536530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 -- 172.21.15.96:0/1313963946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca21e7650 con 0x558ca2523da0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d59cf6640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 0x7f3d400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3d50032c30 tx=0x7f3d50032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7589b6997275660 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d4a7fc640 1 -- 172.21.15.96:0/230491244 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d4c0e6270 con 0x7f3d5410d0e0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1313963946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50035030 con 0x558ca2523da0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1313963946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d50004030 con 0x558ca2523da0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.911+0000 7f3d3f7fe640 1 -- 172.21.15.96:0/1313963946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50005550 con 0x558ca2523da0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1313963946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 msgr2=0x558ca2533c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d59cf6640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 0x7f3d400b77a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0aca60 tx=0x7f3d4c094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3d50032c30 tx=0x7f3d50032c60 comp rx=0 tx=0).stop 2024-09-10T23:03:00.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1313963946 shutdown_connections 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca2536530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca2533c00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1313963946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca2523860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1313963946 >> 172.21.15.96:0/1313963946 conn(0x558ca25212b0 msgr2=0x558ca2522450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1313963946 shutdown_connections 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1313963946 wait complete. 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 -- start start 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.912+0000 7f3d5db2e740 1 Processor -- start 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523da0 0x558ca2530ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca25314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ca22ea700 con 0x558ca2523480 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ca2424110 con 0x558ca2523da0 2024-09-10T23:03:00.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558ca2536d90 con 0x558ca2534140 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d5db2e740 10 client.0 _ll_register_callbacks cb 0x558ca243d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {}, 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_stats": { 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "2df1ad6c-0476-4e75-81a3-b22370ae9b18": { 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:01:36.980195+0000", 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.951 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "8e815330-f099-43c4-b9eb-e85915810f52": { 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:06.118134+0000", 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.952 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "6a1f25f9-789b-475d-b1e0-5f3a64165683": { 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:58:15.704117+0000", 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "9e936084-729f-4a85-93c6-78633e5cb370": { 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:59:55.952163+0000", 2024-09-10T23:03:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "51945418-7691-40a2-845b-cad5ed759ad6": { 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:47:22.978340+0000", 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.954 INFO:teuthology.orchestra.run.smithi096.stdout: "5c4cd589-68b1-4458-8ca3-09456aeee767": { 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:48:13.113312+0000", 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "9cb25013-2658-4000-80d3-47238c6a5a9d": { 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:02:26.999221+0000", 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.955 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "125bbf28-a536-4687-81aa-19c7bbe31e42": { 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:01.301322+0000", 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "a9b1e2a5-38f5-4899-a8a8-92c463641aa7": { 2024-09-10T23:03:00.956 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:49:51.313293+0000", 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "203971b1-c5e2-4181-ab92-43d1598bdd3f": { 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:50:43.764214+0000", 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.957 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cc4a82d5-9a16-48fd-980f-de2071c2da43": { 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:56:35.131095+0000", 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "1bf4032f-07a5-46ac-8fb1-5f28fcdd5108": { 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:51:33.561194+0000", 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.958 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "6f5238a0-8988-48e6-bc63-8f6795c3cce5": { 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:04.096114+0000", 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.959 INFO:teuthology.orchestra.run.smithi096.stdout: "c68fedf7-5b4d-4a68-bbe2-ee76cf1e8f5a": { 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:52:21.523211+0000", 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "12e98c91-25b6-43c1-b18c-73411aa73ff4": { 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:53:11.698183+0000", 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "bd68d588-6aa2-48ab-be58-244974ea644c": { 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:55:43.805110+0000", 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "b3181c62-cfec-4c60-99c6-52f389cd7930": { 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T23:00:46.882171+0000", 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.961 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "496acac3-8d58-4071-a703-8099cbf09bdf": { 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:54:53.963104+0000", 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "d789f646-bd3a-43aa-8c3e-87c97bf5fa06": { 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "start_time": "2024-09-10T22:57:24.724119+0000", 2024-09-10T23:03:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_passed": 0, 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_failed": 0, 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_skipped": 0 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:00.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38244/0 (socket says 172.21.15.96:38244) 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 -- 172.21.15.96:0/2085874463 learned_addr learned my addr 172.21.15.96:0/2085874463 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523da0 0x558ca2530ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 -- 172.21.15.96:0/2085874463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 msgr2=0x558ca25314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca25314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 -- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523da0 msgr2=0x558ca2530ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523da0 0x558ca2530ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.913+0000 7f3d58cf4640 1 -- 172.21.15.96:0/2085874463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca2451c40 con 0x558ca2523480 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009b30 tx=0x7f3d44009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e67103ec758379 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:00.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44013110 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d440132b0 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d50004030 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d44013940 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 10 client.18792 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ca2542750 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d50004030 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3d4401fce0 con 0x558ca2523480 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 10 client.18792 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 10 client.18792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 20 client.18792 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.914+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca2543c10 0x558ca2546000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.915+0000 7f3d5db2e740 4 client.18792 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.915+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca249bda0 con 0x558ca2543c10 2024-09-10T23:03:00.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.915+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca2543c10 0x558ca2546000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.915+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca2543c10 0x558ca2546000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a77e7ae22638c server_cookie=7d38a94130ff3a20 in_seq=0 out_seq=0 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.915+0000 7f3d3cff9640 10 client.18792 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d3cff9640 1 -- 172.21.15.96:0/2085874463 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ca249bda0 con 0x558ca2543c10 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d3cff9640 10 client.18792 handle_command_reply: tid=0 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 client.18792 shutdown 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 msgr2=0x558ca2533a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009b30 tx=0x7f3d44009b60 comp rx=0 tx=0).stop 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 shutdown_connections 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca2543c10 0x558ca2546000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523da0 0x558ca2530ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca25314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2085874463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523480 0x558ca2533a50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 >> 172.21.15.96:0/2085874463 conn(0x558ca25212b0 msgr2=0x558ca25203d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d3cff9640 0 client.18792 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 shutdown_connections 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.916+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2085874463 wait complete. 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.917+0000 7f3d5db2e740 20 client.18792 trim_cache size 0 max 16384 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 Processor -- start 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 -- start start 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 0x558ca2520bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca2522300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541ac090 con 0x558ca2523da0 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541adbd0 con 0x558ca2523480 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541ac760 con 0x558ca2534140 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38246/0 (socket says 172.21.15.96:38246) 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 -- 172.21.15.96:0/2447862891 learned_addr learned my addr 172.21.15.96:0/2447862891 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 -- 172.21.15.96:0/2447862891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 msgr2=0x558ca2522300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca2522300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 -- 172.21.15.96:0/2447862891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 msgr2=0x558ca2520bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 0x558ca2520bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 0x558ca2520bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.934+0000 7f3d594f5640 1 -- 172.21.15.96:0/2447862891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca2530c50 con 0x558ca2523da0 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.935+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 0x558ca2520bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:00.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.935+0000 7f3d594f5640 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d50005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd40a891caf7461d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.935+0000 7f3d3effd640 1 -- 172.21.15.96:0/2447862891 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50035040 con 0x558ca2523da0 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.935+0000 7f3d3effd640 1 -- 172.21.15.96:0/2447862891 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d50005610 con 0x558ca2523da0 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.935+0000 7f3d3effd640 1 -- 172.21.15.96:0/2447862891 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50032050 con 0x558ca2523da0 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2447862891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 msgr2=0x558ca253ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d50005050 comp rx=0 tx=0).stop 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2447862891 shutdown_connections 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2534140 0x558ca2522300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253ffc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/2447862891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2523480 0x558ca2520bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2447862891 >> 172.21.15.96:0/2447862891 conn(0x558ca25212b0 msgr2=0x558ca252f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2447862891 shutdown_connections 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- 172.21.15.96:0/2447862891 wait complete. 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 -- start start 2024-09-10T23:03:00.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.936+0000 7f3d5db2e740 1 Processor -- start 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca253c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca25390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541ad500 con 0x558ca2523da0 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541ace30 con 0x558ca2534140 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d541aa070 con 0x558ca2523480 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d5db2e740 10 client.0 _ll_register_callbacks cb 0x7f3d44009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38262/0 (socket says 172.21.15.96:38262) 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 -- 172.21.15.96:0/1905391979 learned_addr learned my addr 172.21.15.96:0/1905391979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca25390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca253c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 -- 172.21.15.96:0/1905391979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 msgr2=0x558ca253c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca253c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 -- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 msgr2=0x558ca25390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca25390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.937+0000 7f3d594f5640 1 -- 172.21.15.96:0/1905391979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca25398f0 con 0x558ca2523da0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d594f5640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3d500057c0 tx=0x7f3d5002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7156e1e438de73f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5005d020 con 0x558ca2523da0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3d50004b80 con 0x558ca2523da0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d50005610 con 0x558ca2523da0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d50035070 con 0x558ca2523da0 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 10 client.18798 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:00.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ca25b1680 con 0x558ca2523da0 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d50005610 con 0x558ca2523da0 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3d50054030 con 0x558ca2523da0 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 10 client.18798 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 20 client.18798 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca25b1940 0x558ca25b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 4 client.18798 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.938+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ca2497850 con 0x558ca25b1940 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca25b1940 0x558ca25b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d58cf4640 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca25b1940 0x558ca25b3d30 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285d84a585e9b50b server_cookie=14fb59b12f8e18fc in_seq=0 out_seq=0 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d3cff9640 10 client.18798 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:00.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d3cff9640 1 -- 172.21.15.96:0/1905391979 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x558ca2497850 con 0x558ca25b1940 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d3cff9640 10 client.18798 handle_command_reply: tid=0 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d5db2e740 1 client.18798 shutdown 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 msgr2=0x558ca253c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.939+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3d500057c0 tx=0x7f3d5002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 shutdown_connections 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x558ca25b1940 0x558ca25b3d30 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ca2534140 0x558ca25390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ca2523da0 0x558ca253c8e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 --2- 172.21.15.96:0/1905391979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558ca2523480 0x558ca253c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 >> 172.21.15.96:0/1905391979 conn(0x558ca25212b0 msgr2=0x558ca252f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d3cff9640 0 client.18798 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 shutdown_connections 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 1 -- 172.21.15.96:0/1905391979 wait complete. 2024-09-10T23:03:00.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5db2e740 20 client.18798 trim_cache size 0 max 16384 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 msgr2=0x7f3d400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 0x7f3d400b77a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0aca60 tx=0x7f3d4c094040 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 msgr2=0x7f3d5419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00eb10 tx=0x7f3d4c00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 shutdown_connections 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d54104530 0x7f3d541997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f3d400b52e0 0x7f3d400b77a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d5410d0e0 0x7f3d5419eb90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 --2- 172.21.15.96:0/230491244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d54103b30 0x7f3d541992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 >> 172.21.15.96:0/230491244 conn(0x7f3d540fd700 msgr2=0x7f3d540fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 shutdown_connections 2024-09-10T23:03:00.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:00.940+0000 7f3d5b759640 1 -- 172.21.15.96:0/230491244 wait complete. 2024-09-10T23:03:01.951 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:01 smithi096 ceph-mon[30869]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 410 op/s 2024-09-10T23:03:01.980 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T23:03:01.980 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:02.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:01 smithi121 ceph-mon[35119]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 410 op/s 2024-09-10T23:03:02.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:01 smithi117 ceph-mon[35081]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 410 op/s 2024-09-10T23:03:02.228 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:02.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 -- 172.21.15.96:0/3348144508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c108060 msgr2=0x7fa87c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 --2- 172.21.15.96:0/3348144508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c108060 0x7fa87c10e860 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa874009ca0 tx=0x7fa87402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 -- 172.21.15.96:0/3348144508 shutdown_connections 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 --2- 172.21.15.96:0/3348144508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c1076e0 0x7fa87c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 --2- 172.21.15.96:0/3348144508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c108060 0x7fa87c10e860 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 --2- 172.21.15.96:0/3348144508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 0x7fa87c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 -- 172.21.15.96:0/3348144508 >> 172.21.15.96:0/3348144508 conn(0x7fa87c0fd700 msgr2=0x7fa87c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 -- 172.21.15.96:0/3348144508 shutdown_connections 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.727+0000 7fa881cae640 1 -- 172.21.15.96:0/3348144508 wait complete. 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 Processor -- start 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 -- start start 2024-09-10T23:03:02.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 0x7fa87c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 0x7fa87c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c111d20 con 0x7fa87c1076e0 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c111ba0 con 0x7fa87c108060 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa881cae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c111ea0 con 0x7fa87c101cd0 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa873fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa873fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38286/0 (socket says 172.21.15.96:38286) 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa873fff640 1 -- 172.21.15.96:0/3507239911 learned_addr learned my addr 172.21.15.96:0/3507239911 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.728+0000 7fa87bfff640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 0x7fa87c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa87b7fe640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 0x7fa87c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 -- 172.21.15.96:0/3507239911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 msgr2=0x7fa87c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 0x7fa87c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 -- 172.21.15.96:0/3507239911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 msgr2=0x7fa87c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 0x7fa87c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 -- 172.21.15.96:0/3507239911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c19f2b0 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa87bfff640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 0x7fa87c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa873fff640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c890 tx=0x7fa86400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864004220 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c19f5a0 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8640043c0 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864004a00 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.729+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1a5c60 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.730+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c1020b0 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.731+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa8640079b0 con 0x7fa87c1076e0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.732+0000 7fa8797fa640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 0x7fa8680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.732+0000 7fa87b7fe640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 0x7fa8680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:02.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.732+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa8640e5d30 con 0x7fa87c1076e0 2024-09-10T23:03:02.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.732+0000 7fa87b7fe640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 0x7fa8680b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005fd0 tx=0x7fa86c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:02.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.735+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa864014030 con 0x7fa87c1076e0 2024-09-10T23:03:02.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.948+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa87c19a330 con 0x7fa87c1076e0 2024-09-10T23:03:02.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.948+0000 7fa8797fa640 1 -- 172.21.15.96:0/3507239911 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fa8640acc90 con 0x7fa87c1076e0 2024-09-10T23:03:02.950 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:02.950 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:03:02.950 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 msgr2=0x7fa8680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 0x7fa8680b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005fd0 tx=0x7fa86c005850 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 msgr2=0x7fa87c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c890 tx=0x7fa86400cd50 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 shutdown_connections 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa87c108060 0x7fa87c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa8680b52e0 0x7fa8680b77a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa87c1076e0 0x7fa87c199740 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 --2- 172.21.15.96:0/3507239911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa87c101cd0 0x7fa87c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.950+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 >> 172.21.15.96:0/3507239911 conn(0x7fa87c0fd700 msgr2=0x7fa87c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:02.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.951+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 shutdown_connections 2024-09-10T23:03:02.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:02.951+0000 7fa881cae640 1 -- 172.21.15.96:0/3507239911 wait complete. 2024-09-10T23:03:03.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:03 smithi096 ceph-mon[30869]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 775 KiB/s wr, 56 op/s 2024-09-10T23:03:03.735 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:03 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3507239911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:03.763 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T23:03:04.012 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:04.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:03 smithi121 ceph-mon[35119]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 775 KiB/s wr, 56 op/s 2024-09-10T23:03:04.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:03 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3507239911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:04.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:03 smithi117 ceph-mon[35081]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 775 KiB/s wr, 56 op/s 2024-09-10T23:03:04.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:03 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3507239911' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 -- 172.21.15.96:0/3605410243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 msgr2=0x7f264010f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 --2- 172.21.15.96:0/3605410243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264010f580 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2638009ca0 tx=0x7f263802a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 -- 172.21.15.96:0/3605410243 shutdown_connections 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 --2- 172.21.15.96:0/3605410243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f264010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 --2- 172.21.15.96:0/3605410243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264010f580 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 --2- 172.21.15.96:0/3605410243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 0x7f26401040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.510+0000 7f26484fa640 1 -- 172.21.15.96:0/3605410243 >> 172.21.15.96:0/3605410243 conn(0x7f26400fd700 msgr2=0x7f26400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- 172.21.15.96:0/3605410243 shutdown_connections 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- 172.21.15.96:0/3605410243 wait complete. 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 Processor -- start 2024-09-10T23:03:04.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- start start 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 0x7f26401992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f26401997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112d90 con 0x7f264010d120 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112c10 con 0x7f26401045e0 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f26484fa640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112f10 con 0x7f2640103cc0 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f2646a97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f2646a97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38312/0 (socket says 172.21.15.96:38312) 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.511+0000 7f2646a97640 1 -- 172.21.15.96:0/2711081002 learned_addr learned my addr 172.21.15.96:0/2711081002 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646296640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 0x7f26401992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2645a95640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f26401997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 -- 172.21.15.96:0/2711081002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 msgr2=0x7f26401992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 0x7f26401992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 -- 172.21.15.96:0/2711081002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 msgr2=0x7f26401997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f26401997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 -- 172.21.15.96:0/2711081002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264019f350 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2645a95640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f26401997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f2646a97640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2638005cc0 tx=0x7f263802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f26377fe640 1 -- 172.21.15.96:0/2711081002 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638003bf0 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264019f5e0 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f26377fe640 1 -- 172.21.15.96:0/2711081002 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638003d90 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f26377fe640 1 -- 172.21.15.96:0/2711081002 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263802fbc0 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.512+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401a5d00 con 0x7f264010d120 2024-09-10T23:03:04.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 Processor -- start 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f26377fe640 1 -- 172.21.15.96:0/2711081002 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f2638004610 con 0x7f264010d120 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 -- start start 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53af88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53af8df0 0x558b53b08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b53a6eea0 con 0x558b53b09190 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558b53a5e2d0 con 0x558b53af84d0 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f264a8cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558b53a620e0 con 0x558b53af8df0 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38320/0 (socket says 172.21.15.96:38320) 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.515+0000 7f2645a95640 1 -- 172.21.15.96:0/2692955364 learned_addr learned my addr 172.21.15.96:0/2692955364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2646296640 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53af8df0 0x558b53b08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53af88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f26377fe640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 0x7f262c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 -- 172.21.15.96:0/2692955364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53af8df0 msgr2=0x558b53b08c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53af8df0 0x558b53b08c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 -- 172.21.15.96:0/2692955364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 msgr2=0x558b53af88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53af88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 -- 172.21.15.96:0/2692955364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b537bc650 con 0x558b53b09190 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2646a97640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 0x7f262c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2645a95640 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f263c012520 tx=0x7f263c00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a31d6fb742864 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f26377fe640 1 -- 172.21.15.96:0/2711081002 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f2638107130 con 0x7f264010d120 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2634ff9640 1 -- 172.21.15.96:0/2692955364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c018070 con 0x558b53b09190 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2634ff9640 1 -- 172.21.15.96:0/2692955364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c0047a0 con 0x558b53b09190 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.516+0000 7f2634ff9640 1 -- 172.21.15.96:0/2692955364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c01c670 con 0x558b53b09190 2024-09-10T23:03:04.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- 172.21.15.96:0/2692955364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 msgr2=0x558b53b0b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f2646a97640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 0x7f262c0b77a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f26380cd690 tx=0x7f2638099cd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f263c012520 tx=0x7f263c00b600 comp rx=0 tx=0).stop 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- 172.21.15.96:0/2692955364 shutdown_connections 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53b09190 0x558b53b0b580 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53af8df0 0x558b53b08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 --2- 172.21.15.96:0/2692955364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53af88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- 172.21.15.96:0/2692955364 >> 172.21.15.96:0/2692955364 conn(0x558b53af6300 msgr2=0x558b53af74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- 172.21.15.96:0/2692955364 shutdown_connections 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- 172.21.15.96:0/2692955364 wait complete. 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 -- start start 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.517+0000 7f264a8cf740 1 Processor -- start 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53b08aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b06520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b538bf700 con 0x558b53af8df0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558b539f9160 con 0x558b53af84d0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558b53b0be40 con 0x558b53b09190 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 10 client.0 _ll_register_callbacks cb 0x558b53a129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38322/0 (socket says 172.21.15.96:38322) 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 -- 172.21.15.96:0/392615267 learned_addr learned my addr 172.21.15.96:0/392615267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2645a95640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53b08aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2645a95640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b06520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 -- 172.21.15.96:0/392615267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 msgr2=0x558b53b06520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b06520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 -- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 msgr2=0x558b53b08aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53b08aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 -- 172.21.15.96:0/392615267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b53a26c90 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2646296640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2630010840 tx=0x7f2630010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e996f06142a6e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263000cb10 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f263000ccb0 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.518+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263c0047a0 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2630011440 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 10 client.18816 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b53b177d0 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c0047a0 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f263001a550 con 0x558b53af8df0 2024-09-10T23:03:04.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 10 client.18816 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 10 client.18816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 20 client.18816 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b18be0 0x558b53b1b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 4 client.18816 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b53a70df0 con 0x558b53b18be0 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.519+0000 7f2645a95640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b18be0 0x558b53b1b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.520+0000 7f2629ffb640 10 client.18816 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.520+0000 7f2645a95640 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b18be0 0x558b53b1b040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=216a07596a69504c server_cookie=86f65605113bc26d in_seq=0 out_seq=0 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.520+0000 7f2629ffb640 1 -- 172.21.15.96:0/392615267 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b53a70df0 con 0x558b53b18be0 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.520+0000 7f2629ffb640 10 client.18816 handle_command_reply: tid=0 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 client.18816 shutdown 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 msgr2=0x558b53b05ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2630010840 tx=0x7f2630010d00 comp rx=0 tx=0).stop 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 shutdown_connections 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b18be0 0x558b53b1b040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b06520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f2629ffb640 0 client.18816 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af8df0 0x558b53b05ef0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 --2- 172.21.15.96:0/392615267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af84d0 0x558b53b08aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 >> 172.21.15.96:0/392615267 conn(0x558b53af6300 msgr2=0x558b53afbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 shutdown_connections 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 1 -- 172.21.15.96:0/392615267 wait complete. 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.521+0000 7f264a8cf740 20 client.18816 trim_cache size 0 max 16384 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 Processor -- start 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 -- start start 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53af1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ac080 con 0x558b53af84d0 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26401adbc0 con 0x558b53af8df0 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f264a8cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ac750 con 0x558b53b09190 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38338/0 (socket says 172.21.15.96:38338) 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 -- 172.21.15.96:0/3836677248 learned_addr learned my addr 172.21.15.96:0/3836677248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2646296640 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 -- 172.21.15.96:0/3836677248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 msgr2=0x558b53af1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53af1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 -- 172.21.15.96:0/3836677248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 msgr2=0x558b53b150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.534+0000 7f2645a95640 1 -- 172.21.15.96:0/3836677248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b53b04410 con 0x558b53af84d0 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f2645a95640 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f263c001eb0 tx=0x7f263c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0855dfb45a79f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f262bfff640 1 -- 172.21.15.96:0/3836677248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c018040 con 0x558b53af84d0 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f262bfff640 1 -- 172.21.15.96:0/3836677248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c004f40 con 0x558b53af84d0 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f262bfff640 1 -- 172.21.15.96:0/3836677248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c02c6e0 con 0x558b53af84d0 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 -- 172.21.15.96:0/3836677248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 msgr2=0x558b53af5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f263c001eb0 tx=0x7f263c0046c0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 -- 172.21.15.96:0/3836677248 shutdown_connections 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53af1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 --2- 172.21.15.96:0/3836677248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53af5c10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 -- 172.21.15.96:0/3836677248 >> 172.21.15.96:0/3836677248 conn(0x558b53af6300 msgr2=0x558b53af6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 -- 172.21.15.96:0/3836677248 shutdown_connections 2024-09-10T23:03:04.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.535+0000 7f264a8cf740 1 -- 172.21.15.96:0/3836677248 wait complete. 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 -- start start 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 Processor -- start 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ad4f0 con 0x558b53af84d0 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ace20 con 0x558b53af8df0 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26401aa0e0 con 0x558b53b09190 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f264a8cf740 10 client.0 _ll_register_callbacks cb 0x7f263000e3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38346/0 (socket says 172.21.15.96:38346) 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 -- 172.21.15.96:0/190161455 learned_addr learned my addr 172.21.15.96:0/190161455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:04.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2646296640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 -- 172.21.15.96:0/190161455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 msgr2=0x558b53b0e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b0e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 -- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 msgr2=0x558b53b11820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b11820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2645a95640 1 -- 172.21.15.96:0/190161455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b53b0e810 con 0x558b53af84d0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.536+0000 7f2646296640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2645a95640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b6e0 tx=0x7f263c036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3704368a0f211d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c018040 con 0x558b53af84d0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f263c005500 con 0x558b53af84d0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263c004f40 con 0x558b53af84d0 2024-09-10T23:03:04.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f263c02c3f0 con 0x558b53af84d0 2024-09-10T23:03:04.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 10 client.18822 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:04.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b53b86560 con 0x558b53af84d0 2024-09-10T23:03:04.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c004f40 con 0x558b53af84d0 2024-09-10T23:03:04.562 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:03:04.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f263c030020 con 0x558b53af84d0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 10 client.18822 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 10 client.18822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 20 client.18822 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b869b0 0x558b53b88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 4 client.18822 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 --> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b53a6c8a0 con 0x558b53b869b0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.537+0000 7f2646296640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b869b0 0x558b53b88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f2629ffb640 10 client.18822 ms_handle_connect on v2:172.21.15.117:6834/2943423694 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f2646296640 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b869b0 0x558b53b88dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=643c8a67f454e920 server_cookie=beaa8e24e40cd676 in_seq=0 out_seq=0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f2629ffb640 1 -- 172.21.15.96:0/190161455 <== mds.4 v2:172.21.15.117:6834/2943423694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b53a6c8a0 con 0x558b53b869b0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f2629ffb640 10 client.18822 handle_command_reply: tid=0 2024-09-10T23:03:04.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 client.18822 shutdown 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 msgr2=0x558b53b112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b6e0 tx=0x7f263c036040 comp rx=0 tx=0).stop 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 shutdown_connections 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:6834/2943423694,v1:172.21.15.117:6835/2943423694] conn(0x558b53b869b0 0x558b53b88dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558b53af8df0 0x558b53b11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f2629ffb640 0 client.18822 ms_handle_reset on v2:172.21.15.117:6834/2943423694 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b53b09190 0x558b53b0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 --2- 172.21.15.96:0/190161455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b53af84d0 0x558b53b112e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 >> 172.21.15.96:0/190161455 conn(0x558b53af6300 msgr2=0x558b53af6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 shutdown_connections 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 1 -- 172.21.15.96:0/190161455 wait complete. 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.538+0000 7f264a8cf740 20 client.18822 trim_cache size 0 max 16384 2024-09-10T23:03:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 msgr2=0x7f262c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 0x7f262c0b77a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f26380cd690 tx=0x7f2638099cd0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 msgr2=0x7f264019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2638005cc0 tx=0x7f263802f380 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 shutdown_connections 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26401045e0 0x7f26401997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f262c0b52e0 0x7f262c0b77a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f264010d120 0x7f264019eb80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 --2- 172.21.15.96:0/2711081002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2640103cc0 0x7f26401992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 >> 172.21.15.96:0/2711081002 conn(0x7f26400fd700 msgr2=0x7f26400fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 shutdown_connections 2024-09-10T23:03:04.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:04.539+0000 7f26484fa640 1 -- 172.21.15.96:0/2711081002 wait complete. 2024-09-10T23:03:05.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T23:03:05.457 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:05.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:05 smithi096 ceph-mon[30869]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T23:03:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.977+0000 7f393aeba640 1 -- 172.21.15.96:0/3254478414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 msgr2=0x7f393410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.977+0000 7f393aeba640 1 --2- 172.21.15.96:0/3254478414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f393410e820 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- 172.21.15.96:0/3254478414 shutdown_connections 2024-09-10T23:03:05.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- 172.21.15.96:0/3254478414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f393410e820 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- 172.21.15.96:0/3254478414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 0x7f3934107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- 172.21.15.96:0/3254478414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 0x7f3934102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- 172.21.15.96:0/3254478414 >> 172.21.15.96:0/3254478414 conn(0x7f39340fd700 msgr2=0x7f39340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- 172.21.15.96:0/3254478414 shutdown_connections 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- 172.21.15.96:0/3254478414 wait complete. 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 Processor -- start 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- start start 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 0x7f39340fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 0x7f39340fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3934111ca0 con 0x7f3934108020 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3934111b20 con 0x7f3934101c90 2024-09-10T23:03:05.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.978+0000 7f393aeba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3934111e20 con 0x7f39341076a0 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38366/0 (socket says 172.21.15.96:38366) 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 -- 172.21.15.96:0/1212962011 learned_addr learned my addr 172.21.15.96:0/1212962011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3938c56640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 0x7f39340fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3933fff640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 0x7f39340fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 -- 172.21.15.96:0/1212962011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 msgr2=0x7f39340fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 0x7f39340fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 -- 172.21.15.96:0/1212962011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 msgr2=0x7f39340fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 0x7f39340fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 -- 172.21.15.96:0/1212962011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a7670 con 0x7f3934108020 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3939457640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098d0 tx=0x7f392c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:05.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3931ffb640 1 -- 172.21.15.96:0/1212962011 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c003b70 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3931ffb640 1 -- 172.21.15.96:0/1212962011 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c003d10 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a7900 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f3931ffb640 1 -- 172.21.15.96:0/1212962011 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c02fa20 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.979+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a7e90 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3931ffb640 1 -- 172.21.15.96:0/1212962011 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f392c02fc40 con 0x7f3934108020 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 Processor -- start 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 -- start start 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c33df0 0x556d15c3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c42170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d15ba9ea0 con 0x556d15c334d0 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d15b992d0 con 0x556d15c33df0 2024-09-10T23:03:06.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f393d28f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d15b9d0e0 con 0x556d15c3fd80 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38376/0 (socket says 172.21.15.96:38376) 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 -- 172.21.15.96:0/1541124372 learned_addr learned my addr 172.21.15.96:0/1541124372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3938c56640 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c33df0 0x556d15c3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c42170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 -- 172.21.15.96:0/1541124372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 msgr2=0x556d15c42170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.982+0000 7f3933fff640 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c42170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3933fff640 1 -- 172.21.15.96:0/1541124372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c33df0 msgr2=0x556d15c3f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3933fff640 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c33df0 0x556d15c3f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3933fff640 1 -- 172.21.15.96:0/1541124372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d158f7650 con 0x556d15c334d0 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3931ffb640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 0x7f391c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3938c56640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 0x7f391c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3933fff640 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f39280135a0 tx=0x7f3928009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85ad399848c2549 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f3931ffb640 1 -- 172.21.15.96:0/1212962011 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f392c106d30 con 0x7f3934108020 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f391affd640 1 -- 172.21.15.96:0/1541124372 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928004050 con 0x556d15c334d0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f391affd640 1 -- 172.21.15.96:0/1541124372 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280041f0 con 0x556d15c334d0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.983+0000 7f391affd640 1 -- 172.21.15.96:0/1541124372 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800f7e0 con 0x556d15c334d0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- 172.21.15.96:0/1541124372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 msgr2=0x556d15c338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f39280135a0 tx=0x7f3928009830 comp rx=0 tx=0).stop 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f3938c56640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 0x7f391c0b7820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x556d15c321e0 tx=0x7f392400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- 172.21.15.96:0/1541124372 shutdown_connections 2024-09-10T23:03:06.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c33df0 0x556d15c3f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c42170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- 172.21.15.96:0/1541124372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c338b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- 172.21.15.96:0/1541124372 >> 172.21.15.96:0/1541124372 conn(0x556d15c31300 msgr2=0x556d15c324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- 172.21.15.96:0/1541124372 shutdown_connections 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- 172.21.15.96:0/1541124372 wait complete. 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 -- start start 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 Processor -- start 2024-09-10T23:03:06.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c3f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c3cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.984+0000 7f393d28f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556d159fa700 con 0x556d15c3fd80 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556d15b34160 con 0x556d15c334d0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d15c42a00 con 0x556d15c33df0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 10 client.0 _ll_register_callbacks cb 0x556d15b4d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38382/0 (socket says 172.21.15.96:38382) 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 -- 172.21.15.96:0/1546752747 learned_addr learned my addr 172.21.15.96:0/1546752747 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3938c56640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c3f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3933fff640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c3cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 -- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 msgr2=0x556d15c3cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c3cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 -- 172.21.15.96:0/1546752747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 msgr2=0x556d15c3f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c3f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 -- 172.21.15.96:0/1546752747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d15b61c90 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3938c56640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c3f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3939457640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f392c02f550 tx=0x7f392c0cd420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a48b78d2ef48f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3913fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0cd9f0 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f392c0d6690 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39280041f0 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f392c0e5070 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 10 client.18834 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.985+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d15c4e350 con 0x556d15c3fd80 2024-09-10T23:03:06.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280041f0 con 0x556d15c3fd80 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f392c0f4200 con 0x556d15c3fd80 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 10 client.18834 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 10 client.18834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 20 client.18834 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15c4f800 0x556d15c51c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 4 client.18834 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d15babdf0 con 0x556d15c4f800 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.986+0000 7f3938c56640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15c4f800 0x556d15c51c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f3913fff640 10 client.18834 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f3938c56640 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15c4f800 0x556d15c51c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=441b942ecccded66 server_cookie=9ba4baad492729d9 in_seq=0 out_seq=0 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f3913fff640 1 -- 172.21.15.96:0/1546752747 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d15babdf0 con 0x556d15c4f800 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f3913fff640 10 client.18834 handle_command_reply: tid=0 2024-09-10T23:03:06.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f393d28f740 1 client.18834 shutdown 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 msgr2=0x556d15c3d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.987+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f392c02f550 tx=0x7f392c0cd420 comp rx=0 tx=0).stop 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 shutdown_connections 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15c4f800 0x556d15c51c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c3fd80 0x556d15c3d0e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c3cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 --2- 172.21.15.96:0/1546752747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c3f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f3913fff640 0 client.18834 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 >> 172.21.15.96:0/1546752747 conn(0x556d15c31300 msgr2=0x556d15c36e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 shutdown_connections 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 1 -- 172.21.15.96:0/1546752747 wait complete. 2024-09-10T23:03:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:05.988+0000 7f393d28f740 20 client.18834 trim_cache size 0 max 16384 2024-09-10T23:03:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 Processor -- start 2024-09-10T23:03:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 -- start start 2024-09-10T23:03:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c4bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c41170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39341b03b0 con 0x556d15c33df0 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39341b1ef0 con 0x556d15c334d0 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f393d28f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f39341b0a80 con 0x556d15c3fd80 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38392/0 (socket says 172.21.15.96:38392) 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 -- 172.21.15.96:0/2045563680 learned_addr learned my addr 172.21.15.96:0/2045563680 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3939457640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c41170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 -- 172.21.15.96:0/2045563680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 msgr2=0x556d15c41170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c41170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3938c56640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c4bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 -- 172.21.15.96:0/2045563680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 msgr2=0x556d15c4bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c4bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3933fff640 1 -- 172.21.15.96:0/2045563680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d15c33c20 con 0x556d15c33df0 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3939457640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c41170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.001+0000 7f3938c56640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f3933fff640 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3928016010 tx=0x7f39280096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a40c128b9923a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f391a7fc640 1 -- 172.21.15.96:0/2045563680 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280035f0 con 0x556d15c33df0 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f391a7fc640 1 -- 172.21.15.96:0/2045563680 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280050b0 con 0x556d15c33df0 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f391a7fc640 1 -- 172.21.15.96:0/2045563680 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800f8e0 con 0x556d15c33df0 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 -- 172.21.15.96:0/2045563680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 msgr2=0x556d15c30310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3928016010 tx=0x7f39280096a0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 -- 172.21.15.96:0/2045563680 shutdown_connections 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c3fd80 0x556d15c41170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c33df0 0x556d15c30310 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 --2- 172.21.15.96:0/2045563680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c334d0 0x556d15c4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 -- 172.21.15.96:0/2045563680 >> 172.21.15.96:0/2045563680 conn(0x556d15c31300 msgr2=0x556d15c2fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 -- 172.21.15.96:0/2045563680 shutdown_connections 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.002+0000 7f393d28f740 1 -- 172.21.15.96:0/2045563680 wait complete. 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 -- start start 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 Processor -- start 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c48210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c3fd80 0x556d15c42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39341b1820 con 0x556d15c334d0 2024-09-10T23:03:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39341b1150 con 0x556d15c3fd80 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f39341ae390 con 0x556d15c33df0 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f393d28f740 10 client.0 _ll_register_callbacks cb 0x7f392c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f3938c56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f3938c56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38396/0 (socket says 172.21.15.96:38396) 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.003+0000 7f3938c56640 1 -- 172.21.15.96:0/4215995559 learned_addr learned my addr 172.21.15.96:0/4215995559 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3933fff640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c48210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 -- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 msgr2=0x556d15c48210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c48210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 -- 172.21.15.96:0/4215995559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c3fd80 msgr2=0x556d15c42c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c3fd80 0x556d15c42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 -- 172.21.15.96:0/4215995559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d15c43210 con 0x556d15c334d0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3933fff640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c48210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3938c56640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3924005d00 tx=0x7f3924016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c9c0fc7e165be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392401ede0 con 0x556d15c334d0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f3924005050 con 0x556d15c334d0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39280050b0 con 0x556d15c334d0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3924005250 con 0x556d15c334d0 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f393d28f740 10 client.18840 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556d15cbcf20 con 0x556d15c334d0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39280050b0 con 0x556d15c334d0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f392402a4d0 con 0x556d15c334d0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.004+0000 7f393d28f740 10 client.18840 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 10 client.18840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 20 client.18840 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15cbd6d0 0x556d15cbfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 4 client.18840 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 --> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d15ba78a0 con 0x556d15cbd6d0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f3933fff640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15cbd6d0 0x556d15cbfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f3913fff640 10 client.18840 ms_handle_connect on v2:172.21.15.121:6836/1420997271 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f3933fff640 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15cbd6d0 0x556d15cbfb10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3cabb8a6e92196ce server_cookie=66330fbbabd12fc6 in_seq=0 out_seq=0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f3913fff640 1 -- 172.21.15.96:0/4215995559 <== mds.3 v2:172.21.15.121:6836/1420997271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d15ba78a0 con 0x556d15cbd6d0 2024-09-10T23:03:06.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f3913fff640 10 client.18840 handle_command_reply: tid=0 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.005+0000 7f393d28f740 1 client.18840 shutdown 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 msgr2=0x556d15c47cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3924005d00 tx=0x7f3924016000 comp rx=0 tx=0).stop 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 shutdown_connections 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:6836/1420997271,v1:172.21.15.121:6837/1420997271] conn(0x556d15cbd6d0 0x556d15cbfb10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556d15c3fd80 0x556d15c42c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d15c33df0 0x556d15c48210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f3913fff640 0 client.18840 ms_handle_reset on v2:172.21.15.121:6836/1420997271 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 --2- 172.21.15.96:0/4215995559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556d15c334d0 0x556d15c47cd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 >> 172.21.15.96:0/4215995559 conn(0x556d15c31300 msgr2=0x556d15c3e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:06.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 shutdown_connections 2024-09-10T23:03:06.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 1 -- 172.21.15.96:0/4215995559 wait complete. 2024-09-10T23:03:06.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393d28f740 20 client.18840 trim_cache size 0 max 16384 2024-09-10T23:03:06.021 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 msgr2=0x7f391c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 0x7f391c0b7820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x556d15c321e0 tx=0x7f392400b390 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 msgr2=0x7f39340fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f392c0098d0 tx=0x7f392c0317b0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 shutdown_connections 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f391c0b5360 0x7f391c0b7820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3934108020 0x7f39340fe680 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39341076a0 0x7f39340fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 --2- 172.21.15.96:0/1212962011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3934101c90 0x7f39340fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:06.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.006+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 >> 172.21.15.96:0/1212962011 conn(0x7f39340fd700 msgr2=0x7f393406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:06.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.007+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 shutdown_connections 2024-09-10T23:03:06.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:06.007+0000 7f393aeba640 1 -- 172.21.15.96:0/1212962011 wait complete. 2024-09-10T23:03:06.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:05 smithi121 ceph-mon[35119]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T23:03:06.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:05 smithi117 ceph-mon[35081]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T23:03:06.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T23:03:07.218 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:07.664 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:07 smithi096 ceph-mon[30869]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T23:03:07.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 -- 172.21.15.96:0/1640973948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe8104570 msgr2=0x7fabe810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 --2- 172.21.15.96:0/1640973948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe8104570 0x7fabe810cb50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 -- 172.21.15.96:0/1640973948 shutdown_connections 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 --2- 172.21.15.96:0/1640973948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe810d120 0x7fabe810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 --2- 172.21.15.96:0/1640973948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe8104570 0x7fabe810cb50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 --2- 172.21.15.96:0/1640973948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 0x7fabe8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 -- 172.21.15.96:0/1640973948 >> 172.21.15.96:0/1640973948 conn(0x7fabe80fd700 msgr2=0x7fabe80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 -- 172.21.15.96:0/1640973948 shutdown_connections 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.744+0000 7fabecebc640 1 -- 172.21.15.96:0/1640973948 wait complete. 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 Processor -- start 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 -- start start 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 0x7fabe8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe8104570 0x7fabe8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8112d60 con 0x7fabe810d120 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8112be0 con 0x7fabe8103b70 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.745+0000 7fabecebc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8112ee0 con 0x7fabe8104570 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33250/0 (socket says 172.21.15.96:33250) 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2321131367 learned_addr learned my addr 172.21.15.96:0/2321131367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe659c640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 0x7fabe8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabdffff640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe8104570 0x7fabe8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2321131367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe8104570 msgr2=0x7fabe8199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe8104570 0x7fabe8199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2321131367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 msgr2=0x7fabe8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 0x7fabe8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2321131367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819f300 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fabe000eaa0 tx=0x7fabe000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabdf7fe640 1 -- 172.21.15.96:0/2321131367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe000cdb0 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe819f5f0 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabdf7fe640 1 -- 172.21.15.96:0/2321131367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe0005020 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabdf7fe640 1 -- 172.21.15.96:0/2321131367 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00116d0 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.746+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81a5cb0 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabdf7fe640 1 -- 172.21.15.96:0/2321131367 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fabe000f040 con 0x7fabe810d120 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 Processor -- start 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 -- start start 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e49d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e49ddb0 0x55b59e4a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b59e403ea0 con 0x55b59e48d0f0 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b59e3f32d0 con 0x55b59e49ddb0 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabef291740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b59e3f70e0 con 0x55b59e48da10 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33262/0 (socket says 172.21.15.96:33262) 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.749+0000 7fabdffff640 1 -- 172.21.15.96:0/910165884 learned_addr learned my addr 172.21.15.96:0/910165884 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabe659c640 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e49d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 -- 172.21.15.96:0/910165884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 msgr2=0x55b59e49d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e49d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 -- 172.21.15.96:0/910165884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e49ddb0 msgr2=0x55b59e4a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdf7fe640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 0x7fabd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e49ddb0 0x55b59e4a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 -- 172.21.15.96:0/910165884 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59e151650 con 0x55b59e48d0f0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabe659c640 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e49d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdffff640 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccafaf5515040d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabe659c640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 0x7fabd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdf7fe640 1 -- 172.21.15.96:0/2321131367 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabe00e5920 con 0x7fabe810d120 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdcff9640 1 -- 172.21.15.96:0/910165884 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0035040 con 0x55b59e48d0f0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdcff9640 1 -- 172.21.15.96:0/910165884 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd00316a0 con 0x55b59e48d0f0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.750+0000 7fabdcff9640 1 -- 172.21.15.96:0/910165884 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0005620 con 0x55b59e48d0f0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- 172.21.15.96:0/910165884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 msgr2=0x55b59e48d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabe659c640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 0x7fabd40b77a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55b59e491330 tx=0x7fabd800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd00095a0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- 172.21.15.96:0/910165884 shutdown_connections 2024-09-10T23:03:07.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e49ddb0 0x55b59e4a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e49d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 --2- 172.21.15.96:0/910165884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48d0f0 0x55b59e48d4d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- 172.21.15.96:0/910165884 >> 172.21.15.96:0/910165884 conn(0x55b59e48af20 msgr2=0x55b59e48c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- 172.21.15.96:0/910165884 shutdown_connections 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- 172.21.15.96:0/910165884 wait complete. 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.751+0000 7fabef291740 1 -- start start 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 Processor -- start 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48d0f0 0x55b59e49d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 0x55b59e49ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b59e254700 con 0x55b59e49ddb0 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b59e38e160 con 0x55b59e48da10 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b59e4a0a00 con 0x55b59e48d0f0 2024-09-10T23:03:07.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 10 client.0 _ll_register_callbacks cb 0x55b59e3a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33272/0 (socket says 172.21.15.96:33272) 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 -- 172.21.15.96:0/3628989450 learned_addr learned my addr 172.21.15.96:0/3628989450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabdffff640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 0x55b59e49ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe659c640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48d0f0 0x55b59e49d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 -- 172.21.15.96:0/3628989450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48d0f0 msgr2=0x55b59e49d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48d0f0 0x55b59e49d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 -- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 msgr2=0x55b59e49ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 0x55b59e49ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 -- 172.21.15.96:0/3628989450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59e3bbc90 con 0x55b59e49ddb0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabdffff640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 0x55b59e49ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fabe00a8800 tx=0x7fabe00ac120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7715b89f89c2bf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe001f030 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabe00054f0 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.752+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd00316a0 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabe00b0c60 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 10 client.18852 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b59e4ac380 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd00316a0 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fabe00a7b80 con 0x55b59e49ddb0 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 10 client.18852 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 10 client.18852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 20 client.18852 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e4ad890 0x55b59e4afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 4 client.18852 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59e405df0 con 0x55b59e4ad890 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.753+0000 7fabe659c640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e4ad890 0x55b59e4afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.754+0000 7fabcdffb640 10 client.18852 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.754+0000 7fabe659c640 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e4ad890 0x55b59e4afcc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ad7c1c32a877763 server_cookie=e936a5f0e0ba0c05 in_seq=0 out_seq=0 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.754+0000 7fabcdffb640 1 -- 172.21.15.96:0/3628989450 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b59e405df0 con 0x55b59e4ad890 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.754+0000 7fabcdffb640 10 client.18852 handle_command_reply: tid=0 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 client.18852 shutdown 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 msgr2=0x55b59e49b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fabe00a8800 tx=0x7fabe00ac120 comp rx=0 tx=0).stop 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 shutdown_connections 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e4ad890 0x55b59e4afcc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabcdffb640 0 client.18852 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48da10 0x55b59e49ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e49b110 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 --2- 172.21.15.96:0/3628989450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48d0f0 0x55b59e49d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 >> 172.21.15.96:0/3628989450 conn(0x55b59e48af20 msgr2=0x55b59e490a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 shutdown_connections 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 1 -- 172.21.15.96:0/3628989450 wait complete. 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.755+0000 7fabef291740 20 client.18852 trim_cache size 0 max 16384 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 Processor -- start 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 -- start start 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e48c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e4a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81abf40 con 0x55b59e49ddb0 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81ada80 con 0x55b59e48d0f0 2024-09-10T23:03:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabef291740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81ac610 con 0x55b59e48da10 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33286/0 (socket says 172.21.15.96:33286) 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2787364806 learned_addr learned my addr 172.21.15.96:0/2787364806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabdffff640 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e4a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2787364806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 msgr2=0x55b59e4a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e4a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2787364806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 msgr2=0x55b59e48c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e48c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 -- 172.21.15.96:0/2787364806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59e4867e0 con 0x55b59e49ddb0 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabdffff640 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e4a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.768+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009270 tx=0x7fabe0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae11537e2ebe5c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabcffff640 1 -- 172.21.15.96:0/2787364806 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe001f030 con 0x55b59e49ddb0 2024-09-10T23:03:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabcffff640 1 -- 172.21.15.96:0/2787364806 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe00054f0 con 0x55b59e49ddb0 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabcffff640 1 -- 172.21.15.96:0/2787364806 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00ab070 con 0x55b59e49ddb0 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 -- 172.21.15.96:0/2787364806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 msgr2=0x55b59e4b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009270 tx=0x7fabe0009710 comp rx=0 tx=0).stop 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 -- 172.21.15.96:0/2787364806 shutdown_connections 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e49ddb0 0x55b59e4b5590 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e48da10 0x55b59e4a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 --2- 172.21.15.96:0/2787364806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e48c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 -- 172.21.15.96:0/2787364806 >> 172.21.15.96:0/2787364806 conn(0x55b59e48af20 msgr2=0x55b59e48bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 -- 172.21.15.96:0/2787364806 shutdown_connections 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.769+0000 7fabef291740 1 -- 172.21.15.96:0/2787364806 wait complete. 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 -- start start 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 Processor -- start 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e4a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e49ddb0 0x55b59e4a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81ad3b0 con 0x55b59e48da10 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81acce0 con 0x55b59e48d0f0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81a9f40 con 0x55b59e49ddb0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabef291740 10 client.0 _ll_register_callbacks cb 0x7fabe000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33290/0 (socket says 172.21.15.96:33290) 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 -- 172.21.15.96:0/53374920 learned_addr learned my addr 172.21.15.96:0/53374920 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabe659c640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e4a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 -- 172.21.15.96:0/53374920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e49ddb0 msgr2=0x55b59e4a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabe6d9d640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e49ddb0 0x55b59e4a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e49ddb0 0x55b59e4a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 -- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 msgr2=0x55b59e4a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e4a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 -- 172.21.15.96:0/53374920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59e4a3630 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabdffff640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cc4129506691c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.770+0000 7fabcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd00318b0 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fabd0031a50 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe00054f0 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabd0057050 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 10 client.18858 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b59e51b3a0 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe00054f0 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fabd0004750 con 0x55b59e48da10 2024-09-10T23:03:07.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 10 client.18858 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 10 client.18858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 20 client.18858 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e51b6b0 0x55b59e51daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 4 client.18858 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.771+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 --> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b59e4018a0 con 0x55b59e51b6b0 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabe659c640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e51b6b0 0x55b59e51daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabcdffb640 10 client.18858 ms_handle_connect on v2:172.21.15.117:6836/214150535 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabe659c640 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e51b6b0 0x55b59e51daa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a424569d0ad615f1 server_cookie=ea416f73b2f6144f in_seq=0 out_seq=0 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabcdffb640 1 -- 172.21.15.96:0/53374920 <== mds.2 v2:172.21.15.117:6836/214150535 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b59e4018a0 con 0x55b59e51b6b0 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabcdffb640 10 client.18858 handle_command_reply: tid=0 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabef291740 1 client.18858 shutdown 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 msgr2=0x55b59e4a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.772+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd0031400 comp rx=0 tx=0).stop 2024-09-10T23:03:07.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 shutdown_connections 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:6836/214150535,v1:172.21.15.117:6837/214150535] conn(0x55b59e51b6b0 0x55b59e51daa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b59e49ddb0 0x55b59e4a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59e48da10 0x55b59e4a6620 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 --2- 172.21.15.96:0/53374920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b59e48d0f0 0x55b59e4a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabcdffb640 0 client.18858 ms_handle_reset on v2:172.21.15.117:6836/214150535 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 >> 172.21.15.96:0/53374920 conn(0x55b59e48af20 msgr2=0x55b59e490230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 shutdown_connections 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 1 -- 172.21.15.96:0/53374920 wait complete. 2024-09-10T23:03:07.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabef291740 20 client.18858 trim_cache size 0 max 16384 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 msgr2=0x7fabd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 0x7fabd40b77a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55b59e491330 tx=0x7fabd800b390 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 msgr2=0x7fabe819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fabe000eaa0 tx=0x7fabe000ef60 comp rx=0 tx=0).stop 2024-09-10T23:03:07.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 shutdown_connections 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fabd40b52e0 0x7fabd40b77a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabe810d120 0x7fabe819eb30 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe8104570 0x7fabe8199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 --2- 172.21.15.96:0/2321131367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8103b70 0x7fabe8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 >> 172.21.15.96:0/2321131367 conn(0x7fabe80fd700 msgr2=0x7fabe80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 shutdown_connections 2024-09-10T23:03:07.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:07.773+0000 7fabecebc640 1 -- 172.21.15.96:0/2321131367 wait complete. 2024-09-10T23:03:08.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:07 smithi121 ceph-mon[35119]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T23:03:08.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:07 smithi117 ceph-mon[35081]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T23:03:08.607 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T23:03:08.852 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- 172.21.15.96:0/3381485575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55981087a0 msgr2=0x7f5598108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 --2- 172.21.15.96:0/3381485575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55981087a0 0x7f5598108be0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5580009900 tx=0x7f558002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- 172.21.15.96:0/3381485575 shutdown_connections 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 --2- 172.21.15.96:0/3381485575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5598109120 0x7f559810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 --2- 172.21.15.96:0/3381485575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55981087a0 0x7f5598108be0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 --2- 172.21.15.96:0/3381485575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 0x7f5598103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- 172.21.15.96:0/3381485575 >> 172.21.15.96:0/3381485575 conn(0x7f55980fe820 msgr2=0x7f5598100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- 172.21.15.96:0/3381485575 shutdown_connections 2024-09-10T23:03:09.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- 172.21.15.96:0/3381485575 wait complete. 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 Processor -- start 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.361+0000 7f559e194640 1 -- start start 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 0x7f5598199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 0x7f55981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112de0 con 0x7f5598109120 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112c60 con 0x7f55981087a0 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5598112f60 con 0x7f5598102d90 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33312/0 (socket says 172.21.15.96:33312) 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 -- 172.21.15.96:0/2279881095 learned_addr learned my addr 172.21.15.96:0/2279881095 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f55977fe640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 0x7f5598199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f558ffff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 0x7f55981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 -- 172.21.15.96:0/2279881095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 msgr2=0x7f5598199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 0x7f5598199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 -- 172.21.15.96:0/2279881095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 msgr2=0x7f55981998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 0x7f55981998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 -- 172.21.15.96:0/2279881095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559819f420 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f558ffff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 0x7f55981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f5597fff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f559000b970 tx=0x7f559000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f55957fa640 1 -- 172.21.15.96:0/2279881095 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590019070 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f55957fa640 1 -- 172.21.15.96:0/2279881095 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590014440 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559819f710 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.362+0000 7f55957fa640 1 -- 172.21.15.96:0/2279881095 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900075d0 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.363+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a5dd0 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55957fa640 1 -- 172.21.15.96:0/2279881095 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f559000cb40 con 0x7f5598109120 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 Processor -- start 2024-09-10T23:03:09.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 -- start start 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd24d0 0x560fc9cd28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cded80 0x560fc9ce1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fc9c48ea0 con 0x560fc9cd2df0 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fc9c382d0 con 0x560fc9cd24d0 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55a0569740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fc9c3c0e0 con 0x560fc9cded80 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33322/0 (socket says 172.21.15.96:33322) 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55977fe640 1 -- 172.21.15.96:0/1522123536 learned_addr learned my addr 172.21.15.96:0/1522123536 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.365+0000 7f55977fe640 1 -- 172.21.15.96:0/1522123536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cded80 msgr2=0x560fc9ce1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55977fe640 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cded80 0x560fc9ce1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55977fe640 1 -- 172.21.15.96:0/1522123536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd24d0 msgr2=0x560fc9cd28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55977fe640 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd24d0 0x560fc9cd28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55977fe640 1 -- 172.21.15.96:0/1522123536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc9996650 con 0x560fc9cd2df0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55957fa640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 0x7f55840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55977fe640 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f558800b760 tx=0x7f558800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd0dad2d45af1db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f558ffff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 0x7f55840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f55957fa640 1 -- 172.21.15.96:0/2279881095 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55900e5270 con 0x7f5598109120 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f558dffb640 1 -- 172.21.15.96:0/1522123536 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588003dd0 con 0x560fc9cd2df0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f558dffb640 1 -- 172.21.15.96:0/1522123536 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5588015070 con 0x560fc9cd2df0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.366+0000 7f558dffb640 1 -- 172.21.15.96:0/1522123536 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588004e00 con 0x560fc9cd2df0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- 172.21.15.96:0/1522123536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 msgr2=0x560fc9cde840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f558800b760 tx=0x7f558800bc20 comp rx=0 tx=0).stop 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f558ffff640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 0x7f55840b7820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5580005ab0 tx=0x7f5580000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- 172.21.15.96:0/1522123536 shutdown_connections 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cded80 0x560fc9ce1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd2df0 0x560fc9cde840 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 --2- 172.21.15.96:0/1522123536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd24d0 0x560fc9cd28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- 172.21.15.96:0/1522123536 >> 172.21.15.96:0/1522123536 conn(0x560fc9cd0300 msgr2=0x560fc9cd14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- 172.21.15.96:0/1522123536 shutdown_connections 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- 172.21.15.96:0/1522123536 wait complete. 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 -- start start 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.367+0000 7f55a0569740 1 Processor -- start 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cdbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cdc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560fc9a99700 con 0x560fc9cd24d0 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fc9bd3160 con 0x560fc9cded80 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fc9ce1a00 con 0x560fc9cd2df0 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55a0569740 10 client.0 _ll_register_callbacks cb 0x560fc9bec9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33324/0 (socket says 172.21.15.96:33324) 2024-09-10T23:03:09.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 -- 172.21.15.96:0/2853388615 learned_addr learned my addr 172.21.15.96:0/2853388615 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f558ffff640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cdbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f5597fff640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cdc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 -- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 msgr2=0x560fc9cdbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cdbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 -- 172.21.15.96:0/2853388615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 msgr2=0x560fc9cdc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cdc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 -- 172.21.15.96:0/2853388615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc9c00c90 con 0x560fc9cd24d0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f55977fe640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f5588004ae0 tx=0x7f5588004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b059496daa65043b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f5576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588011440 con 0x560fc9cd24d0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.368+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f55880115e0 con 0x560fc9cd24d0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5588015070 con 0x560fc9cd24d0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55880117e0 con 0x560fc9cd24d0 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 10 client.18870 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:09.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fc9ced3d0 con 0x560fc9cd24d0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5588015070 con 0x560fc9cd24d0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5588025a30 con 0x560fc9cd24d0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 10 client.18870 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 10 client.18870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 20 client.18870 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9cee7d0 0x560fc9cf0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 4 client.18870 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.369+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fc9c4adf0 con 0x560fc9cee7d0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.370+0000 7f558ffff640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9cee7d0 0x560fc9cf0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.370+0000 7f558ffff640 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9cee7d0 0x560fc9cf0bc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffba804fdb95df93 server_cookie=297624ed710ce2fb in_seq=0 out_seq=0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.370+0000 7f5576ffd640 10 client.18870 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f5576ffd640 1 -- 172.21.15.96:0/2853388615 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560fc9c4adf0 con 0x560fc9cee7d0 2024-09-10T23:03:09.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f5576ffd640 10 client.18870 handle_command_reply: tid=0 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 client.18870 shutdown 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 msgr2=0x560fc9cde710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f5588004ae0 tx=0x7f5588004410 comp rx=0 tx=0).stop 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 shutdown_connections 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9cee7d0 0x560fc9cf0bc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f5576ffd640 0 client.18870 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 --2- 172.21.15.96:0/2853388615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9cde710 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 >> 172.21.15.96:0/2853388615 conn(0x560fc9cd0300 msgr2=0x560fc9cd5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 shutdown_connections 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 1 -- 172.21.15.96:0/2853388615 wait complete. 2024-09-10T23:03:09.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.371+0000 7f55a0569740 20 client.18870 trim_cache size 0 max 16384 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 Processor -- start 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 -- start start 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd24d0 0x560fc9ceac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd2df0 0x560fc9cdb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ae310 con 0x560fc9cded80 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55981afe50 con 0x560fc9cd2df0 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55a0569740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ae9e0 con 0x560fc9cd24d0 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33332/0 (socket says 172.21.15.96:33332) 2024-09-10T23:03:09.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 -- 172.21.15.96:0/1064029540 learned_addr learned my addr 172.21.15.96:0/1064029540 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f55977fe640 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd24d0 0x560fc9ceac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f558ffff640 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd2df0 0x560fc9cdb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 -- 172.21.15.96:0/1064029540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd24d0 msgr2=0x560fc9ceac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd24d0 0x560fc9ceac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.385+0000 7f5597fff640 1 -- 172.21.15.96:0/1064029540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd2df0 msgr2=0x560fc9cdb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f5597fff640 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd2df0 0x560fc9cdb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f5597fff640 1 -- 172.21.15.96:0/1064029540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc9cdbf90 con 0x560fc9cded80 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f5597fff640 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f55900ab880 tx=0x7f5590014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1d3b2affbcda4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f558d7fa640 1 -- 172.21.15.96:0/1064029540 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900abb80 con 0x560fc9cded80 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f558d7fa640 1 -- 172.21.15.96:0/1064029540 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900b05b0 con 0x560fc9cded80 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.386+0000 7f558d7fa640 1 -- 172.21.15.96:0/1064029540 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900b4bd0 con 0x560fc9cded80 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- 172.21.15.96:0/1064029540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 msgr2=0x560fc9cd6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f55900ab880 tx=0x7f5590014b60 comp rx=0 tx=0).stop 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- 172.21.15.96:0/1064029540 shutdown_connections 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cd2df0 0x560fc9cdb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cded80 0x560fc9cd6d90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 --2- 172.21.15.96:0/1064029540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd24d0 0x560fc9ceac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- 172.21.15.96:0/1064029540 >> 172.21.15.96:0/1064029540 conn(0x560fc9cd0300 msgr2=0x560fc9cf0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- 172.21.15.96:0/1064029540 shutdown_connections 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- 172.21.15.96:0/1064029540 wait complete. 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 -- start start 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.387+0000 7f55a0569740 1 Processor -- start 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cf5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cf62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55981af780 con 0x560fc9cd24d0 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55981af0b0 con 0x560fc9cded80 2024-09-10T23:03:09.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55981ac2f0 con 0x560fc9cd2df0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 10 client.0 _ll_register_callbacks cb 0x7f5588004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33334/0 (socket says 172.21.15.96:33334) 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 -- 172.21.15.96:0/4049128426 learned_addr learned my addr 172.21.15.96:0/4049128426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f558ffff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cf5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f5597fff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cf62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 -- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 msgr2=0x560fc9cf5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cf5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 -- 172.21.15.96:0/4049128426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 msgr2=0x560fc9cf62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cf62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 -- 172.21.15.96:0/4049128426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc9cf6920 con 0x560fc9cd24d0 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f5597fff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cf62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f558ffff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cf5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55977fe640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f558800b3f0 tx=0x7f55880042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85814f11c2d9240b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f5576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588034070 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55900b05b0 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.388+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f5588019420 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5588019620 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 10 client.18876 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fc9d5b550 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900b05b0 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f5588042050 con 0x560fc9cd24d0 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 10 client.18876 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 10 client.18876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 20 client.18876 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9d5ba90 0x560fc9d5deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:09.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 4 client.18876 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 --> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fc9c468a0 con 0x560fc9d5ba90 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.389+0000 7f558ffff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9d5ba90 0x560fc9d5deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f558ffff640 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9d5ba90 0x560fc9d5deb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21a990b1800f4da3 server_cookie=7bc726bfb0673ab6 in_seq=0 out_seq=0 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f5576ffd640 10 client.18876 ms_handle_connect on v2:172.21.15.121:6838/693851001 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f5576ffd640 1 -- 172.21.15.96:0/4049128426 <== mds.1 v2:172.21.15.121:6838/693851001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fc9c468a0 con 0x560fc9d5ba90 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f5576ffd640 10 client.18876 handle_command_reply: tid=0 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 client.18876 shutdown 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 msgr2=0x560fc9ce7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f558800b3f0 tx=0x7f55880042d0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 shutdown_connections 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:6838/693851001,v1:172.21.15.121:6839/693851001] conn(0x560fc9d5ba90 0x560fc9d5deb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fc9cded80 0x560fc9cf62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fc9cd2df0 0x560fc9cf5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 --2- 172.21.15.96:0/4049128426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560fc9cd24d0 0x560fc9ce7260 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f5576ffd640 0 client.18876 ms_handle_reset on v2:172.21.15.121:6838/693851001 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 >> 172.21.15.96:0/4049128426 conn(0x560fc9cd0300 msgr2=0x560fc9cd6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 shutdown_connections 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 1 -- 172.21.15.96:0/4049128426 wait complete. 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.390+0000 7f55a0569740 20 client.18876 trim_cache size 0 max 16384 2024-09-10T23:03:09.402 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 msgr2=0x7f55840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 0x7f55840b7820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5580005ab0 tx=0x7f5580000bc0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 msgr2=0x7f559819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f559000b970 tx=0x7f559000be30 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 shutdown_connections 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55981087a0 0x7f55981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f55840b5360 0x7f55840b7820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5598109120 0x7f559819ec50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 --2- 172.21.15.96:0/2279881095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5598102d90 0x7f5598199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 >> 172.21.15.96:0/2279881095 conn(0x7f55980fe820 msgr2=0x7f5598100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:09.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 shutdown_connections 2024-09-10T23:03:09.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:09.391+0000 7f559e194640 1 -- 172.21.15.96:0/2279881095 wait complete. 2024-09-10T23:03:10.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:09 smithi096 ceph-mon[30869]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T23:03:10.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:09 smithi121 ceph-mon[35119]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T23:03:10.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:09 smithi117 ceph-mon[35081]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T23:03:10.056 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T23:03:10.306 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:10.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 -- 172.21.15.96:0/51573014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 msgr2=0x7faf6410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 --2- 172.21.15.96:0/51573014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf6410cd10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7faf54009ca0 tx=0x7faf5402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 -- 172.21.15.96:0/51573014 shutdown_connections 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 --2- 172.21.15.96:0/51573014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6410d250 0x7faf6410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 --2- 172.21.15.96:0/51573014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf6410cd10 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 --2- 172.21.15.96:0/51573014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf64101810 0x7faf64101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 -- 172.21.15.96:0/51573014 >> 172.21.15.96:0/51573014 conn(0x7faf64076f50 msgr2=0x7faf64077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 -- 172.21.15.96:0/51573014 shutdown_connections 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 -- 172.21.15.96:0/51573014 wait complete. 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.820+0000 7faf6ab89640 1 Processor -- start 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 -- start start 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 0x7faf64199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6410d250 0x7faf6419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112d90 con 0x7faf6410a920 2024-09-10T23:03:10.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112c10 con 0x7faf64101810 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf6ab89640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112f10 con 0x7faf6410d250 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33356/0 (socket says 172.21.15.96:33356) 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 -- 172.21.15.96:0/176120179 learned_addr learned my addr 172.21.15.96:0/176120179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf68925640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 0x7faf64199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf69126640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6410d250 0x7faf6419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 -- 172.21.15.96:0/176120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6410d250 msgr2=0x7faf6419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6410d250 0x7faf6419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 -- 172.21.15.96:0/176120179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 msgr2=0x7faf64199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 0x7faf64199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 -- 172.21.15.96:0/176120179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419f2d0 con 0x7faf6410a920 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf68925640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 0x7faf64199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.821+0000 7faf63fff640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf540319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.822+0000 7faf61ffb640 1 -- 172.21.15.96:0/176120179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54003bf0 con 0x7faf6410a920 2024-09-10T23:03:10.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.822+0000 7faf61ffb640 1 -- 172.21.15.96:0/176120179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf54003d90 con 0x7faf6410a920 2024-09-10T23:03:10.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.822+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419f560 con 0x7faf6410a920 2024-09-10T23:03:10.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.822+0000 7faf61ffb640 1 -- 172.21.15.96:0/176120179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5402fc20 con 0x7faf6410a920 2024-09-10T23:03:10.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.822+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a5c80 con 0x7faf6410a920 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.824+0000 7faf61ffb640 1 -- 172.21.15.96:0/176120179 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7faf540046a0 con 0x7faf6410a920 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.824+0000 7faf6cf5e740 1 Processor -- start 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.824+0000 7faf6cf5e740 1 -- start start 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 0x564c044bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044ce190 0x564c044d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c04433ea0 con 0x564c044ce190 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564c044232d0 con 0x564c044bd4d0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c044270e0 con 0x564c044bddf0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044ce190 0x564c044d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf68925640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 0x564c044bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf68925640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.96:50002/0 (socket says 172.21.15.96:50002) 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf68925640 1 -- 172.21.15.96:0/2622473582 learned_addr learned my addr 172.21.15.96:0/2622473582 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf61ffb640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 0x7faf500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 0x7faf500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 -- 172.21.15.96:0/2622473582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 msgr2=0x564c044cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 -- 172.21.15.96:0/2622473582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044ce190 msgr2=0x564c044d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044ce190 0x564c044d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf69126640 1 -- 172.21.15.96:0/2622473582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c04181650 con 0x564c044bd4d0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.825+0000 7faf61ffb640 1 -- 172.21.15.96:0/176120179 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf54106f80 con 0x7faf6410a920 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.826+0000 7faf69126640 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 0x564c044bd8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0155a0 tx=0x7faf4c015a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6367ec2ce9e6b892 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.826+0000 7faf69126640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 0x7faf500b77a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c000 tx=0x7faf4c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.826+0000 7faf5affd640 1 -- 172.21.15.96:0/2622473582 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c012e60 con 0x564c044bd4d0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.826+0000 7faf5affd640 1 -- 172.21.15.96:0/2622473582 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c0054e0 con 0x564c044bd4d0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.826+0000 7faf5affd640 1 -- 172.21.15.96:0/2622473582 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c010750 con 0x564c044bd4d0 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2622473582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 msgr2=0x564c044bd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 0x564c044bd8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0155a0 tx=0x7faf4c015a60 comp rx=0 tx=0).stop 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2622473582 shutdown_connections 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044ce190 0x564c044d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2622473582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bd4d0 0x564c044bd8b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2622473582 >> 172.21.15.96:0/2622473582 conn(0x564c044bb300 msgr2=0x564c044bc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2622473582 shutdown_connections 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2622473582 wait complete. 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- start start 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 Processor -- start 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c04284700 con 0x564c044bd4d0 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564c043be160 con 0x564c044bddf0 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.827+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c044d0e40 con 0x564c044ce190 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf6cf5e740 10 client.0 _ll_register_callbacks cb 0x564c043d79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33368/0 (socket says 172.21.15.96:33368) 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 -- 172.21.15.96:0/3453705453 learned_addr learned my addr 172.21.15.96:0/3453705453 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf63fff640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf69126640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 -- 172.21.15.96:0/3453705453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 msgr2=0x564c044cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 -- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 msgr2=0x564c044caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 -- 172.21.15.96:0/3453705453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c043ebc90 con 0x564c044bd4d0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf68925640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7faf5c011a30 tx=0x7faf5c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db24e9b43c465cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf43fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c004460 con 0x564c044bd4d0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf5c00f9a0 con 0x564c044bd4d0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf4c0054e0 con 0x564c044bd4d0 2024-09-10T23:03:10.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf5c00fbe0 con 0x564c044bd4d0 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf6cf5e740 10 client.18888 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.828+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c044dc870 con 0x564c044bd4d0 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c0054e0 con 0x564c044bd4d0 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faf5c018070 con 0x564c044bd4d0 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 10 client.18888 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 10 client.18888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 20 client.18888 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c044ddcd0 0x564c044e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 4 client.18888 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c04435df0 con 0x564c044ddcd0 2024-09-10T23:03:10.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.829+0000 7faf63fff640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c044ddcd0 0x564c044e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf63fff640 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c044ddcd0 0x564c044e0140 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5985bd9150aba2fc server_cookie=576b3ed426284732 in_seq=0 out_seq=0 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf43fff640 10 client.18888 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf43fff640 1 -- 172.21.15.96:0/3453705453 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c04435df0 con 0x564c044ddcd0 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf43fff640 10 client.18888 handle_command_reply: tid=0 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 client.18888 shutdown 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 msgr2=0x564c044cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7faf5c011a30 tx=0x7faf5c011ef0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 shutdown_connections 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c044ddcd0 0x564c044e0140 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf43fff640 0 client.18888 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/3453705453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044cdaf0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.830+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 >> 172.21.15.96:0/3453705453 conn(0x564c044bb300 msgr2=0x564c044c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.831+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 shutdown_connections 2024-09-10T23:03:10.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.831+0000 7faf6cf5e740 1 -- 172.21.15.96:0/3453705453 wait complete. 2024-09-10T23:03:10.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.831+0000 7faf6cf5e740 20 client.18888 trim_cache size 0 max 16384 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 Processor -- start 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 -- start start 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 0x564c044e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf641abfd0 con 0x564c044bd4d0 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf641adb10 con 0x564c044ce190 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf641ac6a0 con 0x564c044bddf0 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33384/0 (socket says 172.21.15.96:33384) 2024-09-10T23:03:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 -- 172.21.15.96:0/2200644534 learned_addr learned my addr 172.21.15.96:0/2200644534 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf63fff640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 -- 172.21.15.96:0/2200644534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 msgr2=0x564c044e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 -- 172.21.15.96:0/2200644534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 msgr2=0x564c044e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf69126640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 0x564c044e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 0x564c044e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf68925640 1 -- 172.21.15.96:0/2200644534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c044cbe90 con 0x564c044bd4d0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf63fff640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.844+0000 7faf69126640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 0x564c044e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf68925640 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7faf5c005e10 tx=0x7faf5c008820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98335f5448a43397 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf5a7fc640 1 -- 172.21.15.96:0/2200644534 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c005300 con 0x564c044bd4d0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf5a7fc640 1 -- 172.21.15.96:0/2200644534 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5c0054a0 con 0x564c044bd4d0 2024-09-10T23:03:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf5a7fc640 1 -- 172.21.15.96:0/2200644534 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c003b30 con 0x564c044bd4d0 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2200644534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 msgr2=0x564c044da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7faf5c005e10 tx=0x7faf5c008820 comp rx=0 tx=0).stop 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2200644534 shutdown_connections 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044ce190 0x564c044e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044bddf0 0x564c044e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/2200644534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044da090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2200644534 >> 172.21.15.96:0/2200644534 conn(0x564c044bb300 msgr2=0x564c044ba420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.845+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2200644534 shutdown_connections 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 -- 172.21.15.96:0/2200644534 wait complete. 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 -- start start 2024-09-10T23:03:10.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 Processor -- start 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf641ad440 con 0x564c044bd4d0 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf641acd70 con 0x564c044bddf0 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf641a9fd0 con 0x564c044ce190 2024-09-10T23:03:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf6cf5e740 10 client.0 _ll_register_callbacks cb 0x7faf5c011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf68925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33388/0 (socket says 172.21.15.96:33388) 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.846+0000 7faf68925640 1 -- 172.21.15.96:0/1323229658 learned_addr learned my addr 172.21.15.96:0/1323229658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf63fff640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 -- 172.21.15.96:0/1323229658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 msgr2=0x564c044d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 -- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 msgr2=0x564c044d6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044d6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 -- 172.21.15.96:0/1323229658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c044d39e0 con 0x564c044bd4d0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf68925640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7faf5c023690 tx=0x7faf5c0084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81157edf50251969 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf43fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c005300 con 0x564c044bd4d0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5c0054a0 con 0x564c044bd4d0 2024-09-10T23:03:10.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7faf5c023920 con 0x564c044bd4d0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf5c005650 con 0x564c044bd4d0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf6cf5e740 10 client.18894 fetch_fsmap learned FSMap version 32 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564c0454b770 con 0x564c044bd4d0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.847+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5c0054a0 con 0x564c044bd4d0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7faf5c018040 con 0x564c044bd4d0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 10 client.18894 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 10 client.18894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 20 client.18894 populate_metadata read hostname 'smithi096' 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c0454bdd0 0x564c0454e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 4 client.18894 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 --> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c044318a0 con 0x564c0454bdd0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf63fff640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c0454bdd0 0x564c0454e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf63fff640 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c0454bdd0 0x564c0454e1f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a513d375dcf2f6fb server_cookie=4583c25184872df6 in_seq=0 out_seq=0 2024-09-10T23:03:10.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf43fff640 10 client.18894 ms_handle_connect on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf43fff640 1 -- 172.21.15.96:0/1323229658 <== mds.0 v2:172.21.15.117:6838/4053412961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c044318a0 con 0x564c0454bdd0 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf43fff640 10 client.18894 handle_command_reply: tid=0 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.848+0000 7faf6cf5e740 1 client.18894 shutdown 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 msgr2=0x564c044d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7faf5c023690 tx=0x7faf5c0084f0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 shutdown_connections 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:6838/4053412961,v1:172.21.15.117:6839/4053412961] conn(0x564c0454bdd0 0x564c0454e1f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c044bddf0 0x564c044d6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c044ce190 0x564c044d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf43fff640 0 client.18894 ms_handle_reset on v2:172.21.15.117:6838/4053412961 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 --2- 172.21.15.96:0/1323229658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c044bd4d0 0x564c044d6410 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 >> 172.21.15.96:0/1323229658 conn(0x564c044bb300 msgr2=0x564c044c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 shutdown_connections 2024-09-10T23:03:10.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 1 -- 172.21.15.96:0/1323229658 wait complete. 2024-09-10T23:03:10.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6cf5e740 20 client.18894 trim_cache size 0 max 16384 2024-09-10T23:03:10.856 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 msgr2=0x7faf500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 0x7faf500b77a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c000 tx=0x7faf4c01a000 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 msgr2=0x7faf64199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.849+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf540319f0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 shutdown_connections 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7faf500b52e0 0x7faf500b77a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6410d250 0x7faf6419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf6410a920 0x7faf64199760 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 --2- 172.21.15.96:0/176120179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf64101810 0x7faf64199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 >> 172.21.15.96:0/176120179 conn(0x7faf64076f50 msgr2=0x7faf641005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 shutdown_connections 2024-09-10T23:03:10.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:10.850+0000 7faf6ab89640 1 -- 172.21.15.96:0/176120179 wait complete. 2024-09-10T23:03:11.642 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T23:03:11.643 INFO:tasks.fwd_scrub:done joining 2024-09-10T23:03:11.643 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T23:03:11.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:03:11.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:11 smithi096 ceph-mon[30869]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-10T23:03:11.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/4141278577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b4103ae0 msgr2=0x7ff6b4103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/4141278577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4103f60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009cd0 tx=0x7ff6a802a8e0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/4141278577 shutdown_connections 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/4141278577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b41044a0 0x7ff6b410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/4141278577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4103f60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/4141278577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 0x7ff6b4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/4141278577 >> 172.21.15.96:0/4141278577 conn(0x7ff6b40fe250 msgr2=0x7ff6b4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/4141278577 shutdown_connections 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/4141278577 wait complete. 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.755+0000 7ff6bbc8c640 1 Processor -- start 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 -- start start 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 0x7ff6b4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410e600 con 0x7ff6b41044a0 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410e480 con 0x7ff6b4102990 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6bbc8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b410e780 con 0x7ff6b4103ae0 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33396/0 (socket says 172.21.15.96:33396) 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 -- 172.21.15.96:0/2591277798 learned_addr learned my addr 172.21.15.96:0/2591277798 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6b9a28640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 0x7ff6b4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6b9227640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 -- 172.21.15.96:0/2591277798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b4103ae0 msgr2=0x7ff6b4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 -- 172.21.15.96:0/2591277798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 msgr2=0x7ff6b4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 0x7ff6b4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 -- 172.21.15.96:0/2591277798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b419a860 con 0x7ff6b41044a0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6ba229640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400d750 tx=0x7ff6a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4017070 con 0x7ff6b41044a0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a40054d0 con 0x7ff6b41044a0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.756+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4004c60 con 0x7ff6b41044a0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.757+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b419ab50 con 0x7ff6b41044a0 2024-09-10T23:03:11.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.757+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b41a1220 con 0x7ff6b41044a0 2024-09-10T23:03:11.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.758+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b4109ca0 con 0x7ff6b41044a0 2024-09-10T23:03:11.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.759+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7ff6a4004e00 con 0x7ff6b41044a0 2024-09-10T23:03:11.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.760+0000 7ff6b2ffd640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 0x7ff69c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:11.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.760+0000 7ff6b9a28640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 0x7ff69c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:11.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.760+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7ff6a40e5870 con 0x7ff6b41044a0 2024-09-10T23:03:11.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.760+0000 7ff6b9a28640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 0x7ff69c0b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00bbe0 tx=0x7ff6ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:11.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.763+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff6a40ac180 con 0x7ff6b41044a0 2024-09-10T23:03:11.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.938+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6b4063d50 con 0x7ff6b41044a0 2024-09-10T23:03:11.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.940+0000 7ff6b2ffd640 1 -- 172.21.15.96:0/2591277798 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7ff6a40b0bb0 con 0x7ff6b41044a0 2024-09-10T23:03:11.941 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:11.942 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:03:11.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 msgr2=0x7ff69c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 0x7ff69c0b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00bbe0 tx=0x7ff6ac00b390 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 msgr2=0x7ff6b4199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400d750 tx=0x7ff6a400dc10 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 shutdown_connections 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7ff69c0b52e0 0x7ff69c0b77a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b41044a0 0x7ff6b4199fa0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6b4103ae0 0x7ff6b4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 --2- 172.21.15.96:0/2591277798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6b4102990 0x7ff6b4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 >> 172.21.15.96:0/2591277798 conn(0x7ff6b40fe250 msgr2=0x7ff6b4106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:11.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 shutdown_connections 2024-09-10T23:03:11.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:11.942+0000 7ff6bbc8c640 1 -- 172.21.15.96:0/2591277798 wait complete. 2024-09-10T23:03:11.955 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:40:37.540323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:45:15.242749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:45:17.587866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:03:11.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:03:12.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:11 smithi121 ceph-mon[35119]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-10T23:03:12.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:11 smithi117 ceph-mon[35081]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 -- 172.21.15.96:0/535124769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84101860 msgr2=0x7fed84101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 --2- 172.21.15.96:0/535124769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84101860 0x7fed84101c60 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fed80009930 tx=0x7fed8002a420 comp rx=0 tx=0).stop 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 -- 172.21.15.96:0/535124769 shutdown_connections 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 --2- 172.21.15.96:0/535124769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed841029b0 0x7fed84102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 --2- 172.21.15.96:0/535124769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed84103370 0x7fed84109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 --2- 172.21.15.96:0/535124769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84101860 0x7fed84101c60 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 -- 172.21.15.96:0/535124769 >> 172.21.15.96:0/535124769 conn(0x7fed840fd120 msgr2=0x7fed840ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 -- 172.21.15.96:0/535124769 shutdown_connections 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.057+0000 7fed8bc47640 1 -- 172.21.15.96:0/535124769 wait complete. 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 Processor -- start 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 -- start start 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed84101860 0x7fed84198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed841029b0 0x7fed841993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410e5c0 con 0x7fed84103370 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410e440 con 0x7fed84101860 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8bc47640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed8410e740 con 0x7fed841029b0 2024-09-10T23:03:12.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33420/0 (socket says 172.21.15.96:33420) 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 -- 172.21.15.96:0/1491083438 learned_addr learned my addr 172.21.15.96:0/1491083438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed899e3640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed84101860 0x7fed84198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed891e2640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed841029b0 0x7fed841993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 -- 172.21.15.96:0/1491083438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed841029b0 msgr2=0x7fed841993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed841029b0 0x7fed841993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 -- 172.21.15.96:0/1491083438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed84101860 msgr2=0x7fed84198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed84101860 0x7fed84198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 -- 172.21.15.96:0/1491083438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8419a8e0 con 0x7fed84103370 2024-09-10T23:03:12.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.058+0000 7fed8a1e4640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00eaa0 tx=0x7fed7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.059+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00cdb0 con 0x7fed84103370 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.059+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c005020 con 0x7fed84103370 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.059+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8419abd0 con 0x7fed84103370 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.059+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0116d0 con 0x7fed84103370 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.059+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a12a0 con 0x7fed84103370 2024-09-10T23:03:12.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.060+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed84108ca0 con 0x7fed84103370 2024-09-10T23:03:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.061+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fed7c00f040 con 0x7fed84103370 2024-09-10T23:03:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.062+0000 7fed76ffd640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 0x7fed700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.062+0000 7fed899e3640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 0x7fed700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.062+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fed7c0e5950 con 0x7fed84103370 2024-09-10T23:03:12.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.062+0000 7fed899e3640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 0x7fed700b7820 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed80000c00 tx=0x7fed800314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.066+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fed7c016030 con 0x7fed84103370 2024-09-10T23:03:12.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.240+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed8402e810 con 0x7fed84103370 2024-09-10T23:03:12.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.241+0000 7fed76ffd640 1 -- 172.21.15.96:0/1491083438 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fed7c0ac8b0 con 0x7fed84103370 2024-09-10T23:03:12.242 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-10T23:03:12.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 msgr2=0x7fed700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 0x7fed700b7820 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed80000c00 tx=0x7fed800314a0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 msgr2=0x7fed8419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00eaa0 tx=0x7fed7c00ef60 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 shutdown_connections 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fed700b5360 0x7fed700b7820 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed84103370 0x7fed8419a020 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed841029b0 0x7fed841993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 --2- 172.21.15.96:0/1491083438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed84101860 0x7fed84198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 >> 172.21.15.96:0/1491083438 conn(0x7fed840fd120 msgr2=0x7fed84105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 shutdown_connections 2024-09-10T23:03:12.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.243+0000 7fed8bc47640 1 -- 172.21.15.96:0/1491083438 wait complete. 2024-09-10T23:03:12.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T23:03:12.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/1910423104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 msgr2=0x7f0bb8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/1910423104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8101400 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009930 tx=0x7f0ba802a420 comp rx=0 tx=0).stop 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/1910423104 shutdown_connections 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/1910423104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 0x7f0bb8109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/1910423104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 0x7f0bb81025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/1910423104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8101400 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/1910423104 >> 172.21.15.96:0/1910423104 conn(0x7f0bb80fc8a0 msgr2=0x7f0bb80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/1910423104 shutdown_connections 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.356+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/1910423104 wait complete. 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 Processor -- start 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 -- start start 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 0x7f0bb8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 0x7f0bb8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810d4d0 con 0x7f0bb8101000 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810d350 con 0x7f0bb8102b10 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbe6cd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810d650 con 0x7f0bb8102150 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33424/0 (socket says 172.21.15.96:33424) 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 -- 172.21.15.96:0/534141169 learned_addr learned my addr 172.21.15.96:0/534141169 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bbcc6a640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 0x7f0bb8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0baffff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 0x7f0bb8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 -- 172.21.15.96:0/534141169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 msgr2=0x7f0bb8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 0x7f0bb8198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 -- 172.21.15.96:0/534141169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 msgr2=0x7f0bb8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 0x7f0bb8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 -- 172.21.15.96:0/534141169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb8199760 con 0x7f0bb8101000 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.357+0000 7f0bb7fff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009a50 tx=0x7f0ba8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.358+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba803f030 con 0x7f0bb8101000 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.358+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb81999f0 con 0x7f0bb8101000 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.358+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba80044b0 con 0x7f0bb8101000 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.358+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8035070 con 0x7f0bb8101000 2024-09-10T23:03:12.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.358+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb81a0120 con 0x7f0bb8101000 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.359+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb8101400 con 0x7f0bb8101000 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.361+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0ba803bb90 con 0x7f0bb8101000 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.362+0000 7f0bb5ffb640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 0x7f0ba40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.362+0000 7f0baffff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 0x7f0ba40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.362+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f0ba8106a20 con 0x7f0bb8101000 2024-09-10T23:03:12.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.362+0000 7f0baffff640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 0x7f0ba40b77a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005fd0 tx=0x7f0ba0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.365+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0ba80cd980 con 0x7f0bb8101000 2024-09-10T23:03:12.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.540+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0bb8063d50 con 0x7f0bb8101000 2024-09-10T23:03:12.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.540+0000 7f0bb5ffb640 1 -- 172.21.15.96:0/534141169 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f0ba80cd280 con 0x7f0bb8101000 2024-09-10T23:03:12.542 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-10T23:03:12.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 msgr2=0x7f0ba40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 0x7f0ba40b77a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005fd0 tx=0x7f0ba0005850 comp rx=0 tx=0).stop 2024-09-10T23:03:12.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 msgr2=0x7f0bb8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009a50 tx=0x7f0ba8031a50 comp rx=0 tx=0).stop 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 shutdown_connections 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bb8102b10 0x7f0bb8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0ba40b52e0 0x7f0ba40b77a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bb8102150 0x7f0bb8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 --2- 172.21.15.96:0/534141169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb8101000 0x7f0bb8197c20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 >> 172.21.15.96:0/534141169 conn(0x7f0bb80fc8a0 msgr2=0x7f0bb8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 shutdown_connections 2024-09-10T23:03:12.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.543+0000 7f0bbe6cd640 1 -- 172.21.15.96:0/534141169 wait complete. 2024-09-10T23:03:12.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 -- 172.21.15.96:0/2782005638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc104460 msgr2=0x7f09dc10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 --2- 172.21.15.96:0/2782005638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc104460 0x7f09dc10ade0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009cd0 tx=0x7f09d402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 -- 172.21.15.96:0/2782005638 shutdown_connections 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 --2- 172.21.15.96:0/2782005638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc104460 0x7f09dc10ade0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 --2- 172.21.15.96:0/2782005638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc103aa0 0x7f09dc103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 --2- 172.21.15.96:0/2782005638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 0x7f09dc102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 -- 172.21.15.96:0/2782005638 >> 172.21.15.96:0/2782005638 conn(0x7f09dc0fe250 msgr2=0x7f09dc100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 -- 172.21.15.96:0/2782005638 shutdown_connections 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.657+0000 7f09e1433640 1 -- 172.21.15.96:0/2782005638 wait complete. 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 Processor -- start 2024-09-10T23:03:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 -- start start 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 0x7f09dc076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc104460 0x7f09dc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10e5c0 con 0x7f09dc103aa0 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10e440 con 0x7f09dc102950 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09e1433640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10e740 con 0x7f09dc104460 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33446/0 (socket says 172.21.15.96:33446) 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09da7fc640 1 -- 172.21.15.96:0/4162206282 learned_addr learned my addr 172.21.15.96:0/4162206282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09daffd640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 0x7f09dc076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.658+0000 7f09db7fe640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc104460 0x7f09dc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 -- 172.21.15.96:0/4162206282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc104460 msgr2=0x7f09dc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc104460 0x7f09dc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 -- 172.21.15.96:0/4162206282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 msgr2=0x7f09dc076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 0x7f09dc076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 -- 172.21.15.96:0/4162206282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc0762f0 con 0x7f09dc103aa0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09da7fc640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9f0 tx=0x7f09d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d000cd80 con 0x7f09dc103aa0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d00054d0 con 0x7f09dc103aa0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc1a2e20 con 0x7f09dc103aa0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0010690 con 0x7f09dc103aa0 2024-09-10T23:03:12.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.659+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc1a3290 con 0x7f09dc103aa0 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.660+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc109c60 con 0x7f09dc103aa0 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.661+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f09d0004e30 con 0x7f09dc103aa0 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.662+0000 7f09c3fff640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 0x7f09c40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.662+0000 7f09daffd640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 0x7f09c40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.662+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f09d00e61a0 con 0x7f09dc103aa0 2024-09-10T23:03:12.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.662+0000 7f09daffd640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 0x7f09c40b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00bbe0 tx=0x7f09cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.665+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09d0056220 con 0x7f09dc103aa0 2024-09-10T23:03:12.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.840+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09dc02e5c0 con 0x7f09dc103aa0 2024-09-10T23:03:12.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.841+0000 7f09c3fff640 1 -- 172.21.15.96:0/4162206282 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f09d00ad100 con 0x7f09dc103aa0 2024-09-10T23:03:12.842 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-10T23:03:12.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 msgr2=0x7f09c40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 0x7f09c40b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00bbe0 tx=0x7f09cc00b390 comp rx=0 tx=0).stop 2024-09-10T23:03:12.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 msgr2=0x7f09dc075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9f0 tx=0x7f09d000edf0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 shutdown_connections 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f09c40b52e0 0x7f09c40b77a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc104460 0x7f09dc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09dc103aa0 0x7f09dc075540 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 --2- 172.21.15.96:0/4162206282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09dc102950 0x7f09dc076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 >> 172.21.15.96:0/4162206282 conn(0x7f09dc0fe250 msgr2=0x7f09dc106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 shutdown_connections 2024-09-10T23:03:12.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.843+0000 7f09e1433640 1 -- 172.21.15.96:0/4162206282 wait complete. 2024-09-10T23:03:12.854 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2591277798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:12.854 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1491083438' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:03:12.854 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:12 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/534141169' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:03:12.855 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T23:03:12.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 -- 172.21.15.96:0/2971035791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c06af60 msgr2=0x7fdb9c1055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 --2- 172.21.15.96:0/2971035791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c06af60 0x7fdb9c1055d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009900 tx=0x7fdb9802a410 comp rx=0 tx=0).stop 2024-09-10T23:03:12.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 -- 172.21.15.96:0/2971035791 shutdown_connections 2024-09-10T23:03:12.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 --2- 172.21.15.96:0/2971035791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb9c1086c0 0x7fdb9c10ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 --2- 172.21.15.96:0/2971035791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c105b10 0x7fdb9c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 --2- 172.21.15.96:0/2971035791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c06af60 0x7fdb9c1055d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.952+0000 7fdba46e6640 1 -- 172.21.15.96:0/2971035791 >> 172.21.15.96:0/2971035791 conn(0x7fdb9c0fd180 msgr2=0x7fdb9c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- 172.21.15.96:0/2971035791 shutdown_connections 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- 172.21.15.96:0/2971035791 wait complete. 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 Processor -- start 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- start start 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c06af60 0x7fdb9c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb9c1086c0 0x7fdb9c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10e6c0 con 0x7fdb9c105b10 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10e540 con 0x7fdb9c1086c0 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba46e6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c10e840 con 0x7fdb9c06af60 2024-09-10T23:03:12.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba1c81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba1c81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33458/0 (socket says 172.21.15.96:33458) 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba1c81640 1 -- 172.21.15.96:0/3865195385 learned_addr learned my addr 172.21.15.96:0/3865195385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.953+0000 7fdba2482640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c06af60 0x7fdb9c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 -- 172.21.15.96:0/3865195385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c06af60 msgr2=0x7fdb9c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c06af60 0x7fdb9c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 -- 172.21.15.96:0/3865195385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb9c1086c0 msgr2=0x7fdb9c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb9c1086c0 0x7fdb9c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 -- 172.21.15.96:0/3865195385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c19a930 con 0x7fdb9c105b10 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba1c81640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d740 tx=0x7fdb8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c017070 con 0x7fdb9c105b10 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8c00de50 con 0x7fdb9c105b10 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c19ac20 con 0x7fdb9c105b10 2024-09-10T23:03:12.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004ce0 con 0x7fdb9c105b10 2024-09-10T23:03:12.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.954+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1a12f0 con 0x7fdb9c105b10 2024-09-10T23:03:12.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.955+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c0fec70 con 0x7fdb9c105b10 2024-09-10T23:03:12.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.956+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fdb8c004e80 con 0x7fdb9c105b10 2024-09-10T23:03:12.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.957+0000 7fdb937fe640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 0x7fdb880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:12.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.957+0000 7fdba2482640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 0x7fdb880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:12.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.957+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fdb8c0e5500 con 0x7fdb9c105b10 2024-09-10T23:03:12.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.957+0000 7fdba2482640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 0x7fdb880b77a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb980314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:12.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:12.961+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb8c0ac460 con 0x7fdb9c105b10 2024-09-10T23:03:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2591277798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1491083438' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:03:13.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:12 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/534141169' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:03:13.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2591277798' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:13.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1491083438' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T23:03:13.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:12 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/534141169' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T23:03:13.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.179+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fdb9c02e7c0 con 0x7fdb9c105b10 2024-09-10T23:03:13.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.185+0000 7fdb937fe640 1 -- 172.21.15.96:0/3865195385 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+827941 (secure 0 0 0) 0x7fdb8c0abd60 con 0x7fdb9c105b10 2024-09-10T23:03:13.197 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 20, 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "active_gid": 14150, 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "active_name": "x", 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addrs": { 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3353555679 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3353555679 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addr": "172.21.15.96:6801/3353555679", 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "active_change": "2024-09-10T22:37:57.209985+0000", 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "available": true, 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "standbys": [ 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24112, 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "z", 2024-09-10T23:03:13.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.205 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-09-10T23:03:13.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.208 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.209 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:03:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-09-10T23:03:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-09-10T23:03:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.218 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.223 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.224 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.226 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.227 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.228 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-09-10T23:03:13.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-09-10T23:03:13.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-09-10T23:03:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-09-10T23:03:13.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:03:13.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-09-10T23:03:13.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:03:13.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:03:13.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:03:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:03:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nginx": { 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nginx", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nginx container image", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-09-10T23:03:13.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-09-10T23:03:13.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba_metrics": { 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:03:13.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:03:13.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:03:13.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:03:13.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.279 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:03:13.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.285 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.286 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.287 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-09-10T23:03:13.292 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-09-10T23:03:13.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:03:13.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.299 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-09-10T23:03:13.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-09-10T23:03:13.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.314 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.316 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.320 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-09-10T23:03:13.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-09-10T23:03:13.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:03:13.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.323 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.324 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.333 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.334 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:03:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:03:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:03:13.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-09-10T23:03:13.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:03:13.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:03:13.368 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.369 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.370 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:03:13.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:03:13.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.373 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-09-10T23:03:13.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-09-10T23:03:13.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-09-10T23:03:13.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.380 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.404 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-09-10T23:03:13.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-09-10T23:03:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-09-10T23:03:13.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-09-10T23:03:13.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.430 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.437 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.449 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.453 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-09-10T23:03:13.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-09-10T23:03:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:03:13.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:03:13.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.486 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.498 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.500 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.503 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.515 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.518 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.525 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.526 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.529 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.534 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-09-10T23:03:13.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-09-10T23:03:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.545 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.553 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-09-10T23:03:13.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:13.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:13.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-09-10T23:03:13.560 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.561 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.563 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.565 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.567 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.568 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.569 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.574 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.577 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max_requests": { 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_requests", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.579 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.583 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.584 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.590 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.594 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.595 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-09-10T23:03:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-09-10T23:03:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-09-10T23:03:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-09-10T23:03:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-09-10T23:03:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smb", 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "internal_store_backend": { 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "internal_store_backend", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "update_orchestration": { 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "update_orchestration", 2024-09-10T23:03:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-09-10T23:03:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.693 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.694 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.695 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-09-10T23:03:13.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.711 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-09-10T23:03:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-09-10T23:03:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:03:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:03:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-09-10T23:03:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-09-10T23:03:13.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24125, 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "y", 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.749 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.750 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-09-10T23:03:13.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-09-10T23:03:13.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.754 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-09-10T23:03:13.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-09-10T23:03:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-09-10T23:03:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-09-10T23:03:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-09-10T23:03:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-09-10T23:03:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-09-10T23:03:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-09-10T23:03:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-09-10T23:03:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nginx": { 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nginx", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nginx container image", 2024-09-10T23:03:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-09-10T23:03:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:03:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba_metrics": { 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:03:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-09-10T23:03:13.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-09-10T23:03:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:03:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-09-10T23:03:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-09-10T23:03:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:03:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:03:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:03:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:03:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-09-10T23:03:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-09-10T23:03:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-09-10T23:03:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:13.977 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-09-10T23:03:13.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.981 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.984 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:13.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:13.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:13.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-09-10T23:03:13.997 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:13.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.002 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.003 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.004 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.006 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:03:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.015 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.016 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.018 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.032 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.033 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.071 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.074 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.075 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.076 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.077 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.086 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.101 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max_requests": { 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_requests", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:03:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-09-10T23:03:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.134 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.135 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-09-10T23:03:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smb", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "internal_store_backend": { 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "internal_store_backend", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "update_orchestration": { 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "update_orchestration", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:03:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:03:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-09-10T23:03:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-09-10T23:03:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-09-10T23:03:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-09-10T23:03:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "modules": [ 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard", 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "iostat", 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "nfs", 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "restful" 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-09-10T23:03:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-09-10T23:03:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-09-10T23:03:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:03:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-09-10T23:03:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-09-10T23:03:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.267 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:03:14.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-09-10T23:03:14.270 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:14.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.276 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:03:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:03:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:03:14.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.290 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-09-10T23:03:14.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.292 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.293 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-09-10T23:03:14.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-09-10T23:03:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nginx": { 2024-09-10T23:03:14.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nginx", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nginx container image", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-09-10T23:03:14.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-09-10T23:03:14.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:03:14.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-09-10T23:03:14.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-09-10T23:03:14.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-09-10T23:03:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba_metrics": { 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:03:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:03:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:03:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-09-10T23:03:14.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-09-10T23:03:14.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-09-10T23:03:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-09-10T23:03:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-09-10T23:03:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-09-10T23:03:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:03:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:03:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:03:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:03:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:03:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:03:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:03:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:03:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.379 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:03:14.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:03:14.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:03:14.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.405 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-09-10T23:03:14.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.415 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-09-10T23:03:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:03:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:03:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-09-10T23:03:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-09-10T23:03:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-09-10T23:03:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-09-10T23:03:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-09-10T23:03:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:03:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:03:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-09-10T23:03:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:03:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:03:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-09-10T23:03:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-09-10T23:03:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max_requests": { 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_requests", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:03:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:03:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-09-10T23:03:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T23:03:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-09-10T23:03:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-09-10T23:03:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smb", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "internal_store_backend": { 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "internal_store_backend", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "update_orchestration": { 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "update_orchestration", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.643 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.649 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.650 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:03:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-09-10T23:03:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "services": { 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard": "https://172.21.15.96:8443/" 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "always_on_modules": { 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "octopus": [ 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-09-10T23:03:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "pacific": [ 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-09-10T23:03:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "quincy": [ 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-09-10T23:03:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "reef": [ 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-10T23:03:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "squid": [ 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "force_disabled_modules": {}, 2024-09-10T23:03:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "last_failure_osd_epoch": 3, 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "active_clients": [ 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3362455818 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "libcephsqlite", 2024-09-10T23:03:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2741133105 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-09-10T23:03:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 7310156 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2068503473 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-10T23:03:14.732 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 msgr2=0x7fdb880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 0x7fdb880b77a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb980314a0 comp rx=0 tx=0).stop 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 msgr2=0x7fdb9c199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d740 tx=0x7fdb8c00dc00 comp rx=0 tx=0).stop 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 shutdown_connections 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb9c1086c0 0x7fdb9c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fdb880b52e0 0x7fdb880b77a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb9c105b10 0x7fdb9c199440 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 --2- 172.21.15.96:0/3865195385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb9c06af60 0x7fdb9c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 >> 172.21.15.96:0/3865195385 conn(0x7fdb9c0fd180 msgr2=0x7fdb9c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.191+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 shutdown_connections 2024-09-10T23:03:14.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:13.192+0000 7fdba46e6640 1 -- 172.21.15.96:0/3865195385 wait complete. 2024-09-10T23:03:14.740 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T23:03:14.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T23:03:14.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:13 smithi096 ceph-mon[30869]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-09-10T23:03:14.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/4162206282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:03:14.741 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:13 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3865195385' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:13 smithi117 ceph-mon[35081]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/4162206282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:13 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3865195385' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:13 smithi121 ceph-mon[35119]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/4162206282' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T23:03:14.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:13 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3865195385' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T23:03:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:15.507 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1072, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1074, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 298478, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 340960, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 7, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.096542458, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.039623626, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.020735879, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.028721105, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1072, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.041058737, 2024-09-10T23:03:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000038301 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1265, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1255, 2024-09-10T23:03:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1138734, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 484957, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.145533902, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.064712434, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.048443236, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.030225809, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1254, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.070716714, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000056392 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 903536, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 180944 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1063, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1281, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 358688, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 816748, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.149644996, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.080018361, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.028538885, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.031250624, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1280, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.102906220, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000080395 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 480304 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5138, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.139908143, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000027230 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-10T23:03:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-10T23:03:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 9, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 9, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 9, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 3, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-10T23:03:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-10T23:03:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_write_failed": 0 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-10T23:03:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 13, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4211455, 2024-09-10T23:03:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4210881, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 13 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 12, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 12, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 10, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 10, 2024-09-10T23:03:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 9, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 9, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 49380, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 248246272 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-10T23:03:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 2, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 1, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-10T23:03:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 3040, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 190330, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 57, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 150, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 47466, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 242, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3086, 2024-09-10T23:03:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3086, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3086, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 7.209840116, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002336305 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3086, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3087 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3086, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3086, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 2012, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1072, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-10T23:03:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-10T23:03:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-10T23:03:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 77, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 6, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 1, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3396, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1471325, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3396, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3396, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1471325, 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3086, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 17156, 2024-09-10T23:03:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3086, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 17156, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3086, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3086, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3086, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3086, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x5645f6d4d040": { 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "trackedop": { 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:15.554 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:15.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:15 smithi096 ceph-mon[30869]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T23:03:16.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:15 smithi121 ceph-mon[35119]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T23:03:16.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:15 smithi117 ceph-mon[35081]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T23:03:16.542 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T23:03:16.542 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T23:03:16.542 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-10T23:03:16.542 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-10T23:03:16.542 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T23:03:16.543 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T23:03:16.797 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:17.342 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1373, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1363, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1260720, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 520906, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.187329650, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.093950581, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.058556254, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.032274290, 2024-09-10T23:03:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1362, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.096938557, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000071173 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 904896, 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184144 2024-09-10T23:03:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2025, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2247, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 517938, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1213603, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.179988244, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.079066500, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.036307067, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.053801191, 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2246, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.135627376, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000060386 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 571504 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 6, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1629, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1284, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 8, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.002243012, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000112876, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001818171, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000082399, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-09-10T23:03:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002534430, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000633607 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-10T23:03:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5097, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.131354057, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000025770 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-10T23:03:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:17.351 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-10T23:03:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-10T23:03:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-10T23:03:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_write_failed": 0 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-10T23:03:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 3, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-10T23:03:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4195200, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4194892, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 3 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-10T23:03:17.359 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 48592, 2024-09-10T23:03:17.360 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 248242176 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-10T23:03:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 1, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-10T23:03:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1999, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-10T23:03:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 198304, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 58, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 150, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8672, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3082, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3082, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3082, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 8.501811516, 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002758537 2024-09-10T23:03:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3082, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3083 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3082, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3082, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 2016, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1064, 2024-09-10T23:03:17.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:17.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-10T23:03:17.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 77, 2024-09-10T23:03:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 6, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 9, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 3, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-10T23:03:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3398, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1455229, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3398, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3398, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1455229, 2024-09-10T23:03:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3082, 2024-09-10T23:03:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 901, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3082, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 901, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:17.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3082, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3082, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3082, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3082, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.388 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x56404aa29040": { 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:17.390 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "trackedop": { 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:17.391 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:17.997 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:17 smithi096 ceph-mon[30869]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T23:03:18.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:17 smithi121 ceph-mon[35119]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T23:03:18.026 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-10T23:03:18.026 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-10T23:03:18.026 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-09-10T23:03:18.026 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-09-10T23:03:18.026 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T23:03:18.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T23:03:18.027 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:17 smithi117 ceph-mon[35081]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-10T23:03:18.273 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2133, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2135, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 638539, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 678295, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.211732512, 2024-09-10T23:03:18.963 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.091276194, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.047671614, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.058394343, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2133, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.125441572, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000058809 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 950, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1171, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 234266, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 781898, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.114891870, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.051974582, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018905168, 2024-09-10T23:03:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.032290218, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1169, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.070008299, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000059887 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 480048 2024-09-10T23:03:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 315, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 303, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 907116, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 182350, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.049345574, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.024049481, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.030748654, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000018929, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 303, 2024-09-10T23:03:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.025479212, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000084089 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 906848, 2024-09-10T23:03:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 182032 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5083, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.142477487, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028030 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-09-10T23:03:18.971 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-10T23:03:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-10T23:03:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:18.976 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_started": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uninline_write_failed": 0 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 3, 2024-09-10T23:03:18.977 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4195200, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4194892, 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-10T23:03:18.978 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 3 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-09-10T23:03:18.979 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 49108, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 248238080 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-10T23:03:18.980 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:18.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:18.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.984 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.985 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-10T23:03:18.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-10T23:03:18.992 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 1, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 2000, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:18.993 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:18.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:18.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 190207, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 53, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:18.996 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 150, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8672, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:18.997 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3083, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3083, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3083, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.487922449, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001131340 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3083, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3084 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3083, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3083, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 2011, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1070, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-10T23:03:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-10T23:03:19.001 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 77, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 2, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 7, 2024-09-10T23:03:19.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 2, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:19.003 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3394, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1455474, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3394, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3394, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1455474, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3083, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 901, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3083, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 901, 2024-09-10T23:03:19.006 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:19.007 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3083, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3083, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3083, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3083, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55e204337040": { 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: "trackedop": { 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:19.011 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:19.802 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T23:03:19.802 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T23:03:19.802 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-10T23:03:19.802 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-10T23:03:19.802 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T23:03:19.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T23:03:20.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:19 smithi096 ceph-mon[30869]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-10T23:03:20.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:19 smithi121 ceph-mon[35119]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-10T23:03:20.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:19 smithi117 ceph-mon[35081]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-10T23:03:20.050 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001096971, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:03:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000251898, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 308, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 301, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 858416, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184688, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.053065783, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.024519521, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.035293853, 2024-09-10T23:03:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 301, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.022944445, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000076227 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 858416, 2024-09-10T23:03:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184688 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 228, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 201088, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.021828918, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.018026538, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001495373, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 226, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009964947, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000044092 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 201088 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-10T23:03:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148682, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 29, 2024-09-10T23:03:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-10T23:03:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 968, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 306, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 812399, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 306, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 306, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 812399, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-10T23:03:20.640 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-10T23:03:21.343 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T23:03:21.343 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T23:03:21.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-10T23:03:21.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-10T23:03:21.344 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T23:03:21.344 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T23:03:21.605 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:03:21.726 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:21 smithi117 ceph-mon[35081]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T23:03:22.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:21 smithi096 ceph-mon[30869]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T23:03:22.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:21 smithi121 ceph-mon[35119]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 31461, 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 37555, 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 11769558, 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 22008128, 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 34, 2024-09-10T23:03:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 2.094463297, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 1.074782695, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.993144664, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.259168475, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37554, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 6.369922921, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000169620 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 43779, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.007293324, 2024-09-10T23:03:22.123 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000166 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1002176, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 242400 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 13625, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 15307, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 6663275, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 12166133, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 13, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.038735376, 2024-09-10T23:03:22.124 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.466046051, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.242275286, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.281451724, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 15306, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.322081431, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000151710 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4367, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000934878, 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000214 2024-09-10T23:03:22.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 581200 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 12659, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 15516, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 5103542, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 22170182, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 279, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.207120317, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.564807006, 2024-09-10T23:03:22.126 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.360968323, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.252417958, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 15514, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.191163339, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000141237 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 9015, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001714458, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000190 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 34672, 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 1456 2024-09-10T23:03:22.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 12549, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 17.354077706, 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001382905 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 998, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.048351158, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000048448 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 11481, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 11481, 2024-09-10T23:03:22.129 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11481, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 348.361307587, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.030342418 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 1337, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 3, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 1171, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 7, 2024-09-10T23:03:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 1164, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 1161, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 1, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 1052, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 1052, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 3, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 37662, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 36063, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 476, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 1, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 1, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 19110, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 1027, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 1741929657, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-10T23:03:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 19110, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 656, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 19, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 637, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 18454, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 1, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 5966, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 4862, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 1, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 8648, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 2120, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 217, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 1344 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 172, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 266, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-10T23:03:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 538, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 271, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 267, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 19, 2024-09-10T23:03:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 1, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 3, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 8673, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 8315, 2024-09-10T23:03:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 8315, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 358, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 84, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 78, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 76, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 76, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 1, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 1, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 24745207, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 25674978, 2024-09-10T23:03:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4844, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 39.939250616, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008245097 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 1074, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 1365, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 291, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 68, 2024-09-10T23:03:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 73, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 5, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 1168, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 2189, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 1021, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 1052, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 1633, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 581, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 72152, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 300797952 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 12821, 2024-09-10T23:03:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 1, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 11481, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 561, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 2, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1300, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 3.569442480, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002745724 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7113, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 340.282472780, 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.047839515 2024-09-10T23:03:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 267, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.493333400, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001847690 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1340, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.043201935, 2024-09-10T23:03:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000778508 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.301382788, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008145480 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 527, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.186833902, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000354523 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 68, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.140181274, 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002061489 2024-09-10T23:03:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 496, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.849589133, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003729010 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.013768755, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006884377 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.006483823, 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003241911 2024-09-10T23:03:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 8, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.019952903, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002494112 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 321, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.454664414, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001416400 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 1925, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 1925, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 966, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 382327, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 1194, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 704, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 8, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-10T23:03:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 150, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 3853058, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 53441, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-10T23:03:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 28804, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 25993122, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 37, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 28804, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 28804, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 169.548566560, 2024-09-10T23:03:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005886285 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 28804, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 34992 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 28804, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 19271, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 9533, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1265, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 1552, 2024-09-10T23:03:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 142, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 4967, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 382, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 3, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 806, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 19129, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 3104, 2024-09-10T23:03:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 1, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-10T23:03:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-10T23:03:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 77, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 3371, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-09-10T23:03:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 270 2024-09-10T23:03:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-10T23:03:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-10T23:03:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 13, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 6, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 285, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 271, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 57741, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 18306577, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 57741, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 57741, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 18306577, 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 28804, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 26020493, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 28804, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 26020493, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 28804, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 28804, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 28804, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 28804, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 271, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 27371, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 271, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 123, 2024-09-10T23:03:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 27371, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x562e138f2dc0": { 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 8673, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 21480674, 2024-09-10T23:03:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 8673, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 4844, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 21480674, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-10T23:03:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-10T23:03:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:22.177 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T23:03:23.133 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-09-10T23:03:23.133 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-10T23:03:23.133 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=172 2024-09-10T23:03:23.133 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-10T23:03:23.133 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T23:03:23.133 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T23:03:23.388 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:03:23.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:23 smithi117 ceph-mon[35081]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 74, 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 76, 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 14816, 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 527386, 2024-09-10T23:03:23.907 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 52, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.028866871, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.004082903, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.020038833, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000433680, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 73, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.015565502, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000213226 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 71, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000009816, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000138 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3424, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1948, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1728991, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 705861, 2024-09-10T23:03:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 9, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.226962323, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.114013679, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.146260954, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.004722721, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1949, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.111769588, 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000057347 2024-09-10T23:03:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4310, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000722785, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000167 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 917888, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 190224 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 23, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 249, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 19107, 2024-09-10T23:03:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 595919, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 9, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.017909850, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.012183437, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.003538727, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000496311, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 247, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.026906011, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000108931 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000001517, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000137 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 574512 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001365831, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000036914 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000011717, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000005858 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-10T23:03:23.914 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 2, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 1, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000193924, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000193924 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 1, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:23.915 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 34, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 10, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 24, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 18, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 4, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 1, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 1, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 3, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 75, 2024-09-10T23:03:23.916 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 45, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 1, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 29, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:23.917 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 1027, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 1741929657, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:23.918 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 119, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 74, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 69, 2024-09-10T23:03:23.919 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:23.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-10T23:03:23.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 3, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 3, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 2, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-10T23:03:23.922 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4195200, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.002884016, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001442008 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.923 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 38, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 40, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 2, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 57, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 64, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 7, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 34, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 38, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 4, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 1, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 1, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-10T23:03:23.924 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 51184, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 258727936 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 2, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 2, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 13, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-10T23:03:23.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000193924, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000193924 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-10T23:03:23.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.932 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-10T23:03:23.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 1, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 1, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 1, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 967, 2024-09-10T23:03:23.937 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:23.938 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:23.939 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 247773, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 99, 2024-09-10T23:03:23.940 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 150, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-10T23:03:23.941 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 189023, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 937, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 37, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 11108, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 33, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 37, 2024-09-10T23:03:23.942 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.209995688, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005675559 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 59 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 37, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 0, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 37, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-10T23:03:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 2, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 6, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 6, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:23.944 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-10T23:03:23.945 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-10T23:03:23.946 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 77, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 9, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 14, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 5, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 14, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 1 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-10T23:03:23.947 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 3517, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1425570, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 3517, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 3517, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1425570, 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:23.949 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:23.950 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 37, 2024-09-10T23:03:23.951 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 11108, 2024-09-10T23:03:23.951 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 37, 2024-09-10T23:03:23.951 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 11108, 2024-09-10T23:03:23.951 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:23.951 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:23.952 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 37, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 37, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 37, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 37, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:23.953 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.954 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x557a0c69adc0": { 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 3, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 896, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 3, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 896, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:23.955 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:23.956 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T23:03:24.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:23 smithi096 ceph-mon[30869]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:24.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:23 smithi121 ceph-mon[35119]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:24.797 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:24 smithi117 ceph-mon[35081]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:24.825 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T23:03:24.826 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T23:03:24.826 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-10T23:03:24.826 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=119 2024-09-10T23:03:24.826 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T23:03:24.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T23:03:25.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:24 smithi096 ceph-mon[30869]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:25.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:24 smithi121 ceph-mon[35119]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:25.076 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3506, 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2025, 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1769609, 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 770600, 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 9, 2024-09-10T23:03:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.241836484, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.121268432, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.150390161, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.005747938, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2026, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.102072674, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000050381 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4369, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000989682, 2024-09-10T23:03:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000226 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 933920, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 203120 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 114, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 338, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 41210, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 620373, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 11, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.029870966, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016619212, 2024-09-10T23:03:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.011231188, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000443992, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 336, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.029215843, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000086951 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 178, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000028906, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000162 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:03:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 576944 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 94, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 95, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 33919, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 549729, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 53, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.023514932, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.003028987, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.018436336, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000837370, 2024-09-10T23:03:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 92, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.016894909, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000183640 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 68, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000016515, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000242 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 5216, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 3776 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 96, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.004201297, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000043763 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-10T23:03:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000016323, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000008161 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 70, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 6, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.003568370, 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000594728 2024-09-10T23:03:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 64, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 35, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 10, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 25, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 20, 2024-09-10T23:03:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 4, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 2, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 4, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 158, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 71, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 63, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 24, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-10T23:03:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 1, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 1, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 18, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 1027, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 1741929657, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 18, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 18, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 2, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 2, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 5, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 1, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 1, 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 1 2024-09-10T23:03:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 6, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 110, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 67, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 65, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 6, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 13, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 13, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 2, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-10T23:03:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4211455, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 12, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.023314923, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001942910 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 39, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 41, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 2, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 58, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 62, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 4, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 35, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 39, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 4, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 2, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 6, 2024-09-10T23:03:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 4, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 51436, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 258732032 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 73, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 70, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 17, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 1, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000554970, 2024-09-10T23:03:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000554970 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001631038, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001631038 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000914592, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000304864 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-10T23:03:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000467770, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000467770 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-10T23:03:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-10T23:03:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 1, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 1, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 968, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 258250, 2024-09-10T23:03:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 118, 2024-09-10T23:03:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 150, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 194192, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 1015, 2024-09-10T23:03:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 75, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 29010, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 28, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 75, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 75, 2024-09-10T23:03:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.239995073, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003199934 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 75, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 103 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 75, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 18, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 57, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-09-10T23:03:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 12, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 10, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 6, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 18, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-09-10T23:03:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-10T23:03:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 77, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-10T23:03:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 9, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 14, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 5, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 24, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 3 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-10T23:03:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 3710, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1488307, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 3710, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 3710, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1488307, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 75, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 29010, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 75, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 29010, 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 75, 2024-09-10T23:03:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 75, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 75, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 75, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-10T23:03:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55ef8a28adc0": { 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 13, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 17151, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 13, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 12, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 17151, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-10T23:03:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-10T23:03:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:25.655 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T23:03:26.318 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-10T23:03:26.318 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-09-10T23:03:26.318 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=6 2024-09-10T23:03:26.318 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=110 2024-09-10T23:03:26.318 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T23:03:26.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T23:03:26.566 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:03:26.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:26 smithi117 ceph-mon[35081]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:27.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:26 smithi096 ceph-mon[30869]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:27.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:26 smithi121 ceph-mon[35119]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-09-10T23:03:27.052 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001169184, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000349169, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:27.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 6, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 232, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3360, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 201888, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.017836257, 2024-09-10T23:03:27.054 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.012553597, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.003269525, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 231, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.019124119, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000082788 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-10T23:03:27.055 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 201888 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 306, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 301, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 855920, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 185520, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.043347593, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.018595838, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.032292382, 2024-09-10T23:03:27.056 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 300, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.013098141, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000043660 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 855920, 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 185520 2024-09-10T23:03:27.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:27.058 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:27.059 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:27.060 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 149735, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:27.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 968, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-10T23:03:27.062 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 308, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 812659, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 308, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 308, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 812659, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-10T23:03:27.063 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:27.064 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T23:03:27.064 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T23:03:27.919 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-10T23:03:27.919 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-10T23:03:27.919 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-10T23:03:27.919 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-10T23:03:27.919 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T23:03:27.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T23:03:28.179 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:03:28.715 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T23:03:28.715 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:28.715 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 33, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 34, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 23459, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 73936, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 13, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.010264028, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.000803363, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.005466433, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000524157, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 33, 2024-09-10T23:03:28.716 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.022682128, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000687337 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 15, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000002065, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000137 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 447, 2024-09-10T23:03:28.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 433, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 948567, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 204101, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.059379763, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.027472343, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.040429728, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000490727, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 432, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.041679123, 2024-09-10T23:03:28.718 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000096479 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 171, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000029331, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000171 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 924032, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 189584 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 3114, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1879, 2024-09-10T23:03:28.719 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 811419, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 1469454, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 46, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 8, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.177863533, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.080758101, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.112014137, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.005693661, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1877, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.065964436, 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000035143 2024-09-10T23:03:28.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4421, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000770884, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000174 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 485680 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.721 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 37, 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.001307739, 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000035344 2024-09-10T23:03:28.722 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000017744, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000008872 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 73, 2024-09-10T23:03:28.723 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 73, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 11, 2024-09-10T23:03:28.724 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 34, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 10, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 24, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 18, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 4, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 3, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 145, 2024-09-10T23:03:28.725 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 45, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 73, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 27, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-10T23:03:28.726 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 1027, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 1741929657, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:28.727 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:28.728 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 117, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 72, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 69, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:28.729 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:28.730 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_started": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_write_failed": 0 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-10T23:03:28.731 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 3, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 3, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 2, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 2, 2024-09-10T23:03:28.732 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 2, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 4195200, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.002590607, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001295303 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.733 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 38, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 40, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 2, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 57, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 64, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 7, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 34, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 38, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 4, 2024-09-10T23:03:28.734 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 51336, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 258732032 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 73, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 73, 2024-09-10T23:03:28.735 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 13, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.736 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-10T23:03:28.737 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.738 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-10T23:03:28.740 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.741 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:28.743 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.744 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.745 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.746 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 1, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 0, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 955, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.750 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:28.751 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:28.752 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:28.753 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 243537, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 122, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-10T23:03:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 188687, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 933, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-10T23:03:28.755 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 37, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 11108, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 33, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 37, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 37, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.192997424, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005216146 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-10T23:03:28.756 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 37, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 59 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 37, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 0, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 37, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 0, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 10, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 0, 2024-09-10T23:03:28.757 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 2, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 6, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 6, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 20, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:28.758 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-10T23:03:28.759 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-10T23:03:28.760 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 77, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 8, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 13, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 5, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 14, 2024-09-10T23:03:28.761 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 0, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 1 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-10T23:03:28.762 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:28.763 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3590, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1439540, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3590, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3590, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1439540, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.764 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:28.765 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 37, 2024-09-10T23:03:28.766 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 11108, 2024-09-10T23:03:28.766 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 37, 2024-09-10T23:03:28.766 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 11108, 2024-09-10T23:03:28.766 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:28.767 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 37, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 37, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 37, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 37, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:28.769 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x555bb7986dc0": { 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:28.770 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 896, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 896, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:28.771 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: "trackedop": { 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:28.772 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T23:03:28.876 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:28 smithi121 ceph-mon[35119]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T23:03:29.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:28 smithi096 ceph-mon[30869]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T23:03:29.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:28 smithi117 ceph-mon[35081]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T23:03:29.642 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T23:03:29.642 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T23:03:29.642 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-10T23:03:29.642 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=117 2024-09-10T23:03:29.642 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T23:03:29.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T23:03:29.897 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:03:30.411 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T23:03:30.411 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:30.411 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1903, 2024-09-10T23:03:30.411 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1905, 2024-09-10T23:03:30.411 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 8960849, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 605164, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.188738641, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.075054778, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.048404509, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.051161337, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1903, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.089471734, 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000047016 2024-09-10T23:03:30.412 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1087, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1311, 2024-09-10T23:03:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 8666984, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 920624, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.106165213, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.046772151, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.026212197, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.026175424, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1310, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.062177784, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000047463 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.414 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 576800 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 977, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 967, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 5781379, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 395465, 2024-09-10T23:03:30.415 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 4, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.106447438, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.048091740, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.043477165, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.017250731, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 966, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.036241962, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000037517 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:30.416 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 905600, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 185584 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:30.417 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 6173, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.172839561, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000027999 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.418 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-10T23:03:30.419 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 1893, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 1188, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 705, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 226, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 3, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2024-09-10T23:03:30.420 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 1027, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 1741941143, 2024-09-10T23:03:30.421 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:30.422 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-10T23:03:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_started": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_write_failed": 0 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 8675, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-10T23:03:30.425 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 2, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 71, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 24745207, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 25681988, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 25676116, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 8675 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 1074, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 1346, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 272, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 68, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 73, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 5, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 1893, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 1898, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 5, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-10T23:03:30.427 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 55272, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 256274432 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2024-09-10T23:03:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:30.429 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:30.430 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-10T23:03:30.432 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-10T23:03:30.433 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-10T23:03:30.434 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.435 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.436 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.438 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.439 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-10T23:03:30.440 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 3, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 2, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 2, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 3542, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:30.441 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:30.442 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-10T23:03:30.443 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 237902, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 1108, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:30.444 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 3889827, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 45769, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.445 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 3644, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 26, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 4, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 3644, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3644, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 6.141921512, 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001685488 2024-09-10T23:03:30.446 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3644, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3645 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 3644, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 3644, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 2139, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 1502, 2024-09-10T23:03:30.447 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 2, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:30.448 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-10T23:03:30.449 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-10T23:03:30.450 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 77, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 6, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 6, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2024-09-10T23:03:30.451 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3963, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 23033154, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3963, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3963, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 23033154, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3644, 2024-09-10T23:03:30.454 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 21488320, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3644, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 21488320, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:30.455 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3644, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 3644, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3644, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 3644, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:30.456 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.457 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x55b8dcac9040": { 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:30.458 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "trackedop": { 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:30.459 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T23:03:31.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:30 smithi096 ceph-mon[30869]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T23:03:31.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:30 smithi121 ceph-mon[35119]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T23:03:31.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:30 smithi117 ceph-mon[35081]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-09-10T23:03:31.228 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T23:03:31.228 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T23:03:31.228 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-10T23:03:31.229 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-10T23:03:31.229 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T23:03:31.229 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T23:03:31.476 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:03:32.062 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T23:03:32.062 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 72, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 77, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 27163, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 519514, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 51, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.020775637, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.002152027, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.014232005, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000497832, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 73, 2024-09-10T23:03:32.063 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.020967051, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000287219 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 58, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000007481, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000128 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-10T23:03:32.064 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 736 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 3444, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1959, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 1745104, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 738509, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.202967296, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.093991793, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.135922277, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.006045063, 2024-09-10T23:03:32.065 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1960, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.099005978, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000050513 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4366, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000888515, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000203 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 932560, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 188944 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.066 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 93, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 316, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 25064, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 506023, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 9, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.031196910, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.016410325, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.013094672, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000396703, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 315, 2024-09-10T23:03:32.067 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.029936456, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000095036 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 157, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000026179, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000166 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 487792 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-10T23:03:32.068 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 51, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.002303110, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000045159 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2, 2024-09-10T23:03:32.069 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000012448, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000006224 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 66, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 1, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000155818, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000155818 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-10T23:03:32.070 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 65, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 11, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 34, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 10, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 24, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 18, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 4, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 1, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 1, 2024-09-10T23:03:32.071 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 3, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 139, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 45, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 65, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 29, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-10T23:03:32.072 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 1027, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 1741941143, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.074 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 119, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 74, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 69, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-10T23:03:32.075 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:32.076 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_started": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_write_failed": 0 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 5, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 5, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-10T23:03:32.077 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 2, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 2, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 2, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 4195732, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4, 2024-09-10T23:03:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.007723101, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001930775 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 38, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 40, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 2, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 57, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 64, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 7, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 34, 2024-09-10T23:03:32.079 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 38, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 4, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 1, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 1, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 50780, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 258732032 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 66, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 66, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 17, 2024-09-10T23:03:32.080 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.081 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:32.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000155818, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000155818 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.083 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.084 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-10T23:03:32.085 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:32.089 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.091 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2024-09-10T23:03:32.092 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 1, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 968, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:32.093 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:32.094 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:32.095 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 259169, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 121, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-10T23:03:32.096 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 188999, 2024-09-10T23:03:32.097 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 934, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 41, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 11941, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 29, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 41, 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-10T23:03:32.098 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 41, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.227996971, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005560901 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 41, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 64 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 41, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 0, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 41, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-10T23:03:32.099 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 10, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 4, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 7, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 6, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:32.100 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 20, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-10T23:03:32.101 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-10T23:03:32.102 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 77, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 9, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 12, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 3, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 16, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 0, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 1 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.103 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:32.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3605, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1451902, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3605, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3605, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1451902, 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:32.105 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:32.106 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 41, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 11941, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 41, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 11941, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-10T23:03:32.107 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 41, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 41, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 41, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 41, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.108 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:32.109 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x55fc6d68edc0": { 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 5, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1428, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 5, 2024-09-10T23:03:32.110 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 4, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1428, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "trackedop": { 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:32.111 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T23:03:32.887 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-10T23:03:32.887 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-10T23:03:32.887 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-10T23:03:32.888 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=119 2024-09-10T23:03:32.888 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T23:03:32.888 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T23:03:32.910 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:32 smithi121 ceph-mon[35119]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T23:03:33.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:32 smithi096 ceph-mon[30869]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T23:03:33.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:32 smithi117 ceph-mon[35081]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T23:03:33.136 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:03:33.673 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T23:03:33.673 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T23:03:33.673 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 3149, 2024-09-10T23:03:33.673 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 3149, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 893347, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 1000565, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.258491673, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.112700230, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.055745086, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.074823686, 2024-09-10T23:03:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3148, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.129664630, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000041189 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.675 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 322, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 312, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 907084, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 188682, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 3, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.044838781, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.022013169, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.031413559, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000061215, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 311, 2024-09-10T23:03:33.676 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.019757954, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000063530 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 906528, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 187984 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 3, 2024-09-10T23:03:33.677 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 233, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 588, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 579006, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.016122922, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.011707299, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.002439549, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000031765, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 231, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.020194442, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000087421 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 578688 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-10T23:03:33.679 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 5216, 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.117056630, 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000022441 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-10T23:03:33.680 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.681 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-10T23:03:33.682 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 1, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-10T23:03:33.683 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-10T23:03:33.684 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T23:03:33.685 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-10T23:03:33.686 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-10T23:03:33.687 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_started": 0, 2024-09-10T23:03:33.688 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_succeeded": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "uninline_write_failed": 0 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 5, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-10T23:03:33.689 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 2, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 2, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 4195732, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 4195591, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.690 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 5 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 3, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 3, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 1, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 1, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-10T23:03:33.691 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 49504, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 249286656 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.692 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-10T23:03:33.693 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.694 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.695 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.696 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.697 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.698 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.699 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-10T23:03:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.702 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.704 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.705 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 2, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 3547, 2024-09-10T23:03:33.706 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T23:03:33.707 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T23:03:33.708 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T23:03:33.709 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 182872, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 50, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T23:03:33.710 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 8672, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 13, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-10T23:03:33.711 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 3148, 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 3148, 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3148, 2024-09-10T23:03:33.712 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2.734964758, 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000868794 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3148, 2024-09-10T23:03:33.713 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3149 2024-09-10T23:03:33.714 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.714 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 3148, 2024-09-10T23:03:33.714 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 3148, 2024-09-10T23:03:33.714 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2024-09-10T23:03:33.715 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-10T23:03:33.715 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-10T23:03:33.715 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 2058, 2024-09-10T23:03:33.715 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 1088, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2024-09-10T23:03:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-10T23:03:33.717 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 77, 2024-09-10T23:03:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 4, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 5, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 1, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-10T23:03:33.719 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T23:03:33.720 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3470, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1469169, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3470, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3470, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1469169, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.721 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3148, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 1433, 2024-09-10T23:03:33.722 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3148, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1433, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:33.723 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3148, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 3148, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3148, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 3148, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.724 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.725 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x5591b75a1040": { 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-10T23:03:33.726 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "trackedop": { 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T23:03:33.727 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T23:03:34.556 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T23:03:34.557 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T23:03:34.557 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-10T23:03:34.557 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-10T23:03:34.557 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T23:03:34.567 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T23:03:34.567 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T23:03:34.567 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:03:34.589 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-10T23:03:34.589 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:03:34.649 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T23:03:34.945 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:35.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:34 smithi096 ceph-mon[30869]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T23:03:35.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:34 smithi121 ceph-mon[35119]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T23:03:35.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:34 smithi117 ceph-mon[35081]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-10T23:03:35.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 -- 172.21.15.96:0/41937857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43981087d0 msgr2=0x7f4398108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:35.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 --2- 172.21.15.96:0/41937857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43981087d0 0x7f4398108c10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4380009900 tx=0x7f438002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:35.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 -- 172.21.15.96:0/41937857 shutdown_connections 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 --2- 172.21.15.96:0/41937857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4398109150 0x7f439810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 --2- 172.21.15.96:0/41937857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43981087d0 0x7f4398108c10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 --2- 172.21.15.96:0/41937857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 0x7f43981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 -- 172.21.15.96:0/41937857 >> 172.21.15.96:0/41937857 conn(0x7f43980fe830 msgr2=0x7f4398100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 -- 172.21.15.96:0/41937857 shutdown_connections 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.449+0000 7f439e659640 1 -- 172.21.15.96:0/41937857 wait complete. 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 Processor -- start 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 -- start start 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 0x7f4398199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43981087d0 0x7f43981998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:35.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4398112e10 con 0x7f4398109150 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4398112c90 con 0x7f43981087d0 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439e659640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4398112f90 con 0x7f4398102dc0 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43740/0 (socket says 172.21.15.96:43740) 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 -- 172.21.15.96:0/3091181576 learned_addr learned my addr 172.21.15.96:0/3091181576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f438ffff640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43981087d0 0x7f43981998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f4397fff640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 0x7f4398199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 -- 172.21.15.96:0/3091181576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 msgr2=0x7f4398199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 0x7f4398199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 -- 172.21.15.96:0/3091181576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43981087d0 msgr2=0x7f43981998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43981087d0 0x7f43981998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.450+0000 7f439cbf6640 1 -- 172.21.15.96:0/3091181576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439819f410 con 0x7f4398109150 2024-09-10T23:03:35.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f439cbf6640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f439000cad0 tx=0x7f439000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000ae20 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4390005020 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439819f700 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000f6b0 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.451+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a5dc0 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.452+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43981031a0 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.453+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f43900054f0 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.454+0000 7f4395ffb640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 0x7f43840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.454+0000 7f4397fff640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 0x7f43840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.454+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f43900e5a60 con 0x7f4398109150 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.455+0000 7f4397fff640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 0x7f43840b77a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4388009bd0 tx=0x7f4388005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:35.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.458+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4390014030 con 0x7f4398109150 2024-09-10T23:03:35.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.647+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f439802e5c0 con 0x7f4398109150 2024-09-10T23:03:35.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.648+0000 7f4395ffb640 1 -- 172.21.15.96:0/3091181576 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v77) ==== 81+0+590 (secure 0 0 0) 0x7f43900ac370 con 0x7f4398109150 2024-09-10T23:03:35.649 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/827103951 2024-09-11T22:37:57.209878+0000 2024-09-10T23:03:35.649 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1292035523 2024-09-11T22:37:57.209878+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/1530138287 2024-09-11T22:37:57.209878+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/1530138287 2024-09-11T22:37:57.209878+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/3039659416 2024-09-11T22:37:34.381409+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/3039659416 2024-09-11T22:37:34.381409+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3734789223 2024-09-11T22:37:34.381409+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1542157874 2024-09-11T22:37:34.381409+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/217725041 2024-09-11T22:37:57.209878+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1680117294 2024-09-11T22:37:34.381409+0000 2024-09-10T23:03:35.650 INFO:teuthology.orchestra.run.smithi096.stderr:listed 10 entries 2024-09-10T23:03:35.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 msgr2=0x7f43840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 0x7f43840b77a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4388009bd0 tx=0x7f4388005f90 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 msgr2=0x7f439819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f439000cad0 tx=0x7f439000cf90 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 shutdown_connections 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43981087d0 0x7f43981998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f43840b52e0 0x7f43840b77a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4398109150 0x7f439819ec40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 --2- 172.21.15.96:0/3091181576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4398102dc0 0x7f4398199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 >> 172.21.15.96:0/3091181576 conn(0x7f43980fe830 msgr2=0x7f4398100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 shutdown_connections 2024-09-10T23:03:35.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:35.650+0000 7f439e659640 1 -- 172.21.15.96:0/3091181576 wait complete. 2024-09-10T23:03:35.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:35 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3091181576' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:03:36.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:35 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3091181576' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:03:36.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:35 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3091181576' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T23:03:36.385 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T23:03:36.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:03:36.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /tmp/cephfs 2024-09-10T23:03:36.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T23:03:36.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:03:36.661 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-10T23:03:36.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:03:36.705 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-10T23:03:36.765 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-10T23:03:36.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:03:36.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-10T23:03:36.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:03:36.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-10T23:03:36.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:36 smithi096 ceph-mon[30869]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T23:03:36.928 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T23:03:36.939 INFO:tasks.cephadm:Teardown begin 2024-09-10T23:03:36.939 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:03:36.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:37.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:36 smithi121 ceph-mon[35119]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T23:03:37.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:36 smithi117 ceph-mon[35081]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T23:03:37.194 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 -- 172.21.15.96:0/3878118556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa954108020 msgr2=0x7fa95410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 --2- 172.21.15.96:0/3878118556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa954108020 0x7fa95410e820 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009ca0 tx=0x7fa94c02a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 -- 172.21.15.96:0/3878118556 shutdown_connections 2024-09-10T23:03:37.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 --2- 172.21.15.96:0/3878118556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa954108020 0x7fa95410e820 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 --2- 172.21.15.96:0/3878118556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9541076a0 0x7fa954107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 --2- 172.21.15.96:0/3878118556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa954102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 -- 172.21.15.96:0/3878118556 >> 172.21.15.96:0/3878118556 conn(0x7fa9540fd700 msgr2=0x7fa9540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 -- 172.21.15.96:0/3878118556 shutdown_connections 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.681+0000 7fa959c47640 1 -- 172.21.15.96:0/3878118556 wait complete. 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 Processor -- start 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 -- start start 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa95419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 0x7fa9541a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111ce0 con 0x7fa9541076a0 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111b60 con 0x7fa954101c90 2024-09-10T23:03:37.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa959c47640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111e60 con 0x7fa954108020 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.682+0000 7fa952ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43416/0 (socket says 172.21.15.96:43416) 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 -- 172.21.15.96:0/3758313382 learned_addr learned my addr 172.21.15.96:0/3758313382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa9537fe640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa95419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 -- 172.21.15.96:0/3758313382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 msgr2=0x7fa9541a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa953fff640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 0x7fa9541a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 0x7fa9541a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 -- 172.21.15.96:0/3758313382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 msgr2=0x7fa95419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa95419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 -- 172.21.15.96:0/3758313382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541a55f0 con 0x7fa9541076a0 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa9537fe640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa95419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:37.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa953fff640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 0x7fa9541a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa952ffd640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d770 tx=0x7fa94400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944017070 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a57c0 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9440054d0 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944020060 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.683+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a5c50 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.684+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa954102070 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.686+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7fa944004bf0 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.686+0000 7fa950ff9640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.686+0000 7fa9537fe640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.687+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7fa9440e5c70 con 0x7fa9541076a0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.687+0000 7fa9537fe640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 0x7fa9400b77a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0058c0 tx=0x7fa93c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:37.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.689+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa9440acbd0 con 0x7fa9541076a0 2024-09-10T23:03:37.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.906+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95406af20 con 0x7fa9541076a0 2024-09-10T23:03:37.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.906+0000 7fa950ff9640 1 -- 172.21.15.96:0/3758313382 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7fa9440ac4d0 con 0x7fa9541076a0 2024-09-10T23:03:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:37.908 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:03:37.908 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.908+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 msgr2=0x7fa9400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.908+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 0x7fa9400b77a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0058c0 tx=0x7fa93c005850 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.908+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 msgr2=0x7fa95419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.908+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d770 tx=0x7fa94400dc30 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 shutdown_connections 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa954108020 0x7fa9541a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9541076a0 0x7fa95419a570 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 --2- 172.21.15.96:0/3758313382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa954101c90 0x7fa95419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:37.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 >> 172.21.15.96:0/3758313382 conn(0x7fa9540fd700 msgr2=0x7fa9540ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:37.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 shutdown_connections 2024-09-10T23:03:37.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:37.909+0000 7fa959c47640 1 -- 172.21.15.96:0/3758313382 wait complete. 2024-09-10T23:03:38.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T23:03:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:38.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:38 smithi096 ceph-mon[30869]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T23:03:38.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:38 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3758313382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:39.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:38 smithi121 ceph-mon[35119]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T23:03:39.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:38 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3758313382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:38 smithi117 ceph-mon[35081]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T23:03:39.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:38 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3758313382' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 -- 172.21.15.96:0/3858294190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101cb0 msgr2=0x7f1fa8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 --2- 172.21.15.96:0/3858294190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101cb0 0x7f1fa8102090 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009ca0 tx=0x7f1fa002a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 -- 172.21.15.96:0/3858294190 shutdown_connections 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 --2- 172.21.15.96:0/3858294190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa81076c0 0x7f1fa8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 --2- 172.21.15.96:0/3858294190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8108040 0x7f1fa810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 --2- 172.21.15.96:0/3858294190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa8101cb0 0x7f1fa8102090 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 -- 172.21.15.96:0/3858294190 >> 172.21.15.96:0/3858294190 conn(0x7f1fa80fd700 msgr2=0x7f1fa80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 -- 172.21.15.96:0/3858294190 shutdown_connections 2024-09-10T23:03:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.364+0000 7f1facf87640 1 -- 172.21.15.96:0/3858294190 wait complete. 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 Processor -- start 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 -- start start 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101cb0 0x7f1fa81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 0x7f1fa819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8112e20 con 0x7f1fa81076c0 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8112ca0 con 0x7f1fa8108040 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.365+0000 7f1facf87640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8112fa0 con 0x7f1fa8101cb0 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43436/0 (socket says 172.21.15.96:43436) 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 -- 172.21.15.96:0/2148240601 learned_addr learned my addr 172.21.15.96:0/2148240601 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa659c640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101cb0 0x7f1fa81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa6d9d640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 0x7f1fa819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 -- 172.21.15.96:0/2148240601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101cb0 msgr2=0x7f1fa81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101cb0 0x7f1fa81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 -- 172.21.15.96:0/2148240601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 msgr2=0x7f1fa819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 0x7f1fa819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 -- 172.21.15.96:0/2148240601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa819f370 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa6d9d640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 0x7f1fa819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1fa5d9b640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000d950 tx=0x7f1f9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90017070 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa819f660 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f90009c30 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000f620 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.366+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81a5ef0 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.367+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa8102f60 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.370+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f1f90010460 con 0x7f1fa81076c0 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.370+0000 7f1f9f7fe640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 0x7f1f940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:39.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.371+0000 7f1fa659c640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 0x7f1f940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.371+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f1f900e5770 con 0x7f1fa81076c0 2024-09-10T23:03:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.371+0000 7f1fa659c640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 0x7f1f940b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009430 tx=0x7f1fa0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.373+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1f900ac6d0 con 0x7f1fa81076c0 2024-09-10T23:03:39.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.585+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fa8102090 con 0x7f1fa81076c0 2024-09-10T23:03:39.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.585+0000 7f1f9f7fe640 1 -- 172.21.15.96:0/2148240601 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11979 (secure 0 0 0) 0x7f1fa8102090 con 0x7f1fa81076c0 2024-09-10T23:03:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:39.587 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"btime":"2024-09-10T22:49:35:232947+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14625,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/317768324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":317768324},{"type":"v1","addr":"172.21.15.117:6833","nonce":317768324}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/3071875945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":3071875945},{"type":"v1","addr":"172.21.15.96:6835","nonce":3071875945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:45:19.551704+0000","modified":"2024-09-10T22:49:34.250815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24550,"mds_2":24542,"mds_3":24529,"mds_4":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14649":{"gid":14649,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6837/3503446146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3503446146},{"type":"v1","addr":"172.21.15.96:6837","nonce":3503446146}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/2726589142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2726589142},{"type":"v1","addr":"172.21.15.96:6839","nonce":2726589142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14706":{"gid":14706,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/2618224226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":2618224226},{"type":"v1","addr":"172.21.15.96:6841","nonce":2618224226}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/2699941517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":2699941517},{"type":"v1","addr":"172.21.15.121:6833","nonce":2699941517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/1599221610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1599221610},{"type":"v1","addr":"172.21.15.121:6835","nonce":1599221610}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6835/2943423694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2943423694},{"type":"v1","addr":"172.21.15.117:6835","nonce":2943423694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.121:6837/1420997271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1420997271},{"type":"v1","addr":"172.21.15.121:6837","nonce":1420997271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6837/214150535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":214150535},{"type":"v1","addr":"172.21.15.117:6837","nonce":214150535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.121:6839/693851001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":693851001},{"type":"v1","addr":"172.21.15.121:6839","nonce":693851001}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/4053412961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":4053412961},{"type":"v1","addr":"172.21.15.117:6839","nonce":4053412961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24527,24566,24529,24542,24550]},"id":1}]} 2024-09-10T23:03:39.587 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-09-10T23:03:39.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 msgr2=0x7f1f940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:39.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 0x7f1f940b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009430 tx=0x7f1fa0009360 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 msgr2=0x7f1fa8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000d950 tx=0x7f1f9000de10 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 shutdown_connections 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fa8108040 0x7f1fa819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f1f940b52e0 0x7f1f940b77a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa81076c0 0x7f1fa8199800 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 --2- 172.21.15.96:0/2148240601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fa8101cb0 0x7f1fa81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 >> 172.21.15.96:0/2148240601 conn(0x7f1fa80fd700 msgr2=0x7f1fa80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 shutdown_connections 2024-09-10T23:03:39.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:39.588+0000 7f1facf87640 1 -- 172.21.15.96:0/2148240601 wait complete. 2024-09-10T23:03:39.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:39 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2148240601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:40.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:39 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2148240601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:40.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:39 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2148240601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T23:03:40.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T23:03:40.574 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:41.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:40 smithi096 ceph-mon[30869]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T23:03:41.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:40 smithi117 ceph-mon[35081]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T23:03:41.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 -- 172.21.15.96:0/2385093651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38381055a0 msgr2=0x7f3838105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:41.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 --2- 172.21.15.96:0/2385093651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38381055a0 0x7f3838105a00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3828009900 tx=0x7f382802a410 comp rx=0 tx=0).stop 2024-09-10T23:03:41.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 -- 172.21.15.96:0/2385093651 shutdown_connections 2024-09-10T23:03:41.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 --2- 172.21.15.96:0/2385093651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 0x7f383810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 --2- 172.21.15.96:0/2385093651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38381055a0 0x7f3838105a00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 --2- 172.21.15.96:0/2385093651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3838101250 0x7f3838101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 -- 172.21.15.96:0/2385093651 >> 172.21.15.96:0/2385093651 conn(0x7f3838078140 msgr2=0x7f38380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 -- 172.21.15.96:0/2385093651 shutdown_connections 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 -- 172.21.15.96:0/2385093651 wait complete. 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.105+0000 7f383fe16640 1 Processor -- start 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 -- start start 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 0x7f3838102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 0x7f3838102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3838112db0 con 0x7f3838101250 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3838112c30 con 0x7f38381055a0 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383fe16640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3838112f30 con 0x7f3838105f40 2024-09-10T23:03:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43446/0 (socket says 172.21.15.96:43446) 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 -- 172.21.15.96:0/2829362513 learned_addr learned my addr 172.21.15.96:0/2829362513 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383d3b1640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 0x7f3838102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383e3b3640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 0x7f3838102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 -- 172.21.15.96:0/2829362513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 msgr2=0x7f3838102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 0x7f3838102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 -- 172.21.15.96:0/2829362513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 msgr2=0x7f3838102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 0x7f3838102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 -- 172.21.15.96:0/2829362513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a7680 con 0x7f3838101250 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383d3b1640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 0x7f3838102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f383dbb2640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f383400eab0 tx=0x7f383400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.106+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400ce40 con 0x7f3838101250 2024-09-10T23:03:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.107+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a7970 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.107+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3834011920 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.107+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834010a30 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.107+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a7e80 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.107+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3838101630 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.109+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f383400f040 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.110+0000 7f382effd640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 0x7f38240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.110+0000 7f383d3b1640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 0x7f38240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.110+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f38340e5ef0 con 0x7f3838101250 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.110+0000 7f383d3b1640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 0x7f38240b77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3828031440 tx=0x7f3828005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:41.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.113+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3834016030 con 0x7f3838101250 2024-09-10T23:03:41.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:40 smithi121 ceph-mon[35119]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-10T23:03:41.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.289+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f383802e5c0 con 0x7f3838101250 2024-09-10T23:03:41.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.291+0000 7f382effd640 1 -- 172.21.15.96:0/2829362513 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22574 (secure 0 0 0) 0x7f38340ace50 con 0x7f3838101250 2024-09-10T23:03:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-10T23:03:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":77,"fsid":"14afb286-6fc5-11ef-bcea-c7b262605968","created":"2024-09-10T22:37:12.788608+0000","modified":"2024-09-10T22:45:58.291718+0000","last_up_change":"2024-09-10T22:44:27.708019+0000","last_in_change":"2024-09-10T22:44:04.410370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:40:37.540323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:45:15.242749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:45:17.587866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a28ec19e-b4a4-4a13-8674-1c7d77449432","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6803","nonce":2443850477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6805","nonce":2443850477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6809","nonce":2443850477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2443850477},{"type":"v1","addr":"172.21.15.96:6807","nonce":2443850477}]},"public_addr":"172.21.15.96:6803/2443850477","cluster_addr":"172.21.15.96:6805/2443850477","heartbeat_back_addr":"172.21.15.96:6809/2443850477","heartbeat_front_addr":"172.21.15.96:6807/2443850477","state":["exists","up"]},{"osd":1,"uuid":"afe84389-be35-42dc-bd7b-23672bffb969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6801","nonce":3283899162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6803","nonce":3283899162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6807","nonce":3283899162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3283899162},{"type":"v1","addr":"172.21.15.117:6805","nonce":3283899162}]},"public_addr":"172.21.15.117:6801/3283899162","cluster_addr":"172.21.15.117:6803/3283899162","heartbeat_back_addr":"172.21.15.117:6807/3283899162","heartbeat_front_addr":"172.21.15.117:6805/3283899162","state":["exists","up"]},{"osd":2,"uuid":"7d696b9b-398f-4457-a05b-c14dbb9a42f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6801","nonce":203762338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6803","nonce":203762338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6807","nonce":203762338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":203762338},{"type":"v1","addr":"172.21.15.121:6805","nonce":203762338}]},"public_addr":"172.21.15.121:6801/203762338","cluster_addr":"172.21.15.121:6803/203762338","heartbeat_back_addr":"172.21.15.121:6807/203762338","heartbeat_front_addr":"172.21.15.121:6805/203762338","state":["exists","up"]},{"osd":3,"uuid":"0e749818-335f-4852-b188-2898bf14e7ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6811","nonce":2944383556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6813","nonce":2944383556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6817","nonce":2944383556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2944383556},{"type":"v1","addr":"172.21.15.96:6815","nonce":2944383556}]},"public_addr":"172.21.15.96:6811/2944383556","cluster_addr":"172.21.15.96:6813/2944383556","heartbeat_back_addr":"172.21.15.96:6817/2944383556","heartbeat_front_addr":"172.21.15.96:6815/2944383556","state":["exists","up"]},{"osd":4,"uuid":"43f285c1-14f0-4b31-a928-1705584f5f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6809","nonce":2758163149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6811","nonce":2758163149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6815","nonce":2758163149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2758163149},{"type":"v1","addr":"172.21.15.117:6813","nonce":2758163149}]},"public_addr":"172.21.15.117:6809/2758163149","cluster_addr":"172.21.15.117:6811/2758163149","heartbeat_back_addr":"172.21.15.117:6815/2758163149","heartbeat_front_addr":"172.21.15.117:6813/2758163149","state":["exists","up"]},{"osd":5,"uuid":"5520a789-19f0-4775-a38f-0d7707878cdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6809","nonce":1183691363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6811","nonce":1183691363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6815","nonce":1183691363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1183691363},{"type":"v1","addr":"172.21.15.121:6813","nonce":1183691363}]},"public_addr":"172.21.15.121:6809/1183691363","cluster_addr":"172.21.15.121:6811/1183691363","heartbeat_back_addr":"172.21.15.121:6815/1183691363","heartbeat_front_addr":"172.21.15.121:6813/1183691363","state":["exists","up"]},{"osd":6,"uuid":"372e13d3-06be-488a-aac3-1bae70af8eb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6819","nonce":201788565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6821","nonce":201788565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6825","nonce":201788565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":201788565},{"type":"v1","addr":"172.21.15.96:6823","nonce":201788565}]},"public_addr":"172.21.15.96:6819/201788565","cluster_addr":"172.21.15.96:6821/201788565","heartbeat_back_addr":"172.21.15.96:6825/201788565","heartbeat_front_addr":"172.21.15.96:6823/201788565","state":["exists","up"]},{"osd":7,"uuid":"783ef350-a8c5-4e79-b761-13da9d17c03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6817","nonce":918709723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6819","nonce":918709723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6823","nonce":918709723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":918709723},{"type":"v1","addr":"172.21.15.117:6821","nonce":918709723}]},"public_addr":"172.21.15.117:6817/918709723","cluster_addr":"172.21.15.117:6819/918709723","heartbeat_back_addr":"172.21.15.117:6823/918709723","heartbeat_front_addr":"172.21.15.117:6821/918709723","state":["exists","up"]},{"osd":8,"uuid":"55e87abf-88f6-47fe-9297-9c71f8997270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6817","nonce":4251994917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6819","nonce":4251994917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6823","nonce":4251994917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4251994917},{"type":"v1","addr":"172.21.15.121:6821","nonce":4251994917}]},"public_addr":"172.21.15.121:6817/4251994917","cluster_addr":"172.21.15.121:6819/4251994917","heartbeat_back_addr":"172.21.15.121:6823/4251994917","heartbeat_front_addr":"172.21.15.121:6821/4251994917","state":["exists","up"]},{"osd":9,"uuid":"b757f430-ca41-4632-8891-d74c23873b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6827","nonce":1287082888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6829","nonce":1287082888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6833","nonce":1287082888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1287082888},{"type":"v1","addr":"172.21.15.96:6831","nonce":1287082888}]},"public_addr":"172.21.15.96:6827/1287082888","cluster_addr":"172.21.15.96:6829/1287082888","heartbeat_back_addr":"172.21.15.96:6833/1287082888","heartbeat_front_addr":"172.21.15.96:6831/1287082888","state":["exists","up"]},{"osd":10,"uuid":"e49921d8-dcf9-427a-8a29-57f0450576f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6825","nonce":3915077352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6827","nonce":3915077352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6831","nonce":3915077352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3915077352},{"type":"v1","addr":"172.21.15.117:6829","nonce":3915077352}]},"public_addr":"172.21.15.117:6825/3915077352","cluster_addr":"172.21.15.117:6827/3915077352","heartbeat_back_addr":"172.21.15.117:6831/3915077352","heartbeat_front_addr":"172.21.15.117:6829/3915077352","state":["exists","up"]},{"osd":11,"uuid":"a5084606-c0d3-4731-8e88-17f0ef95d91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6825","nonce":2582866969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6827","nonce":2582866969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6831","nonce":2582866969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2582866969},{"type":"v1","addr":"172.21.15.121:6829","nonce":2582866969}]},"public_addr":"172.21.15.121:6825/2582866969","cluster_addr":"172.21.15.121:6827/2582866969","heartbeat_back_addr":"172.21.15.121:6831/2582866969","heartbeat_front_addr":"172.21.15.121:6829/2582866969","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:39:46.201715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:10.926293+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:40:35.872968+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:00.333707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:26.201174+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:41:51.206924+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:17.805405+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:42:42.910829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:08.724259+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:43:35.186235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:00.407675+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:44:26.555339+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/827103951":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1292035523":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6800/1530138287":"2024-09-11T22:37:57.209878+0000","172.21.15.96:6801/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:6800/3039659416":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/3734789223":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/1542157874":"2024-09-11T22:37:34.381409+0000","172.21.15.96:0/217725041":"2024-09-11T22:37:57.209878+0000","172.21.15.96:0/1680117294":"2024-09-11T22:37:34.381409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:03:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 msgr2=0x7f38240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 0x7f38240b77a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3828031440 tx=0x7f3828005a20 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 msgr2=0x7f3838102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f383400eab0 tx=0x7f383400ef70 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 shutdown_connections 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38381055a0 0x7f3838102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f38240b52e0 0x7f38240b77a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3838105f40 0x7f3838102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 --2- 172.21.15.96:0/2829362513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101250 0x7f3838102220 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 >> 172.21.15.96:0/2829362513 conn(0x7f3838078140 msgr2=0x7f3838108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 shutdown_connections 2024-09-10T23:03:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:41.294+0000 7f383fe16640 1 -- 172.21.15.96:0/2829362513 wait complete. 2024-09-10T23:03:41.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T23:03:41.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:41 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/2829362513' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:42.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:41 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/2829362513' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:42.220 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:42.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:41 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/2829362513' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 -- 172.21.15.96:0/3714623698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27581087d0 msgr2=0x7f2758108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 --2- 172.21.15.96:0/3714623698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27581087d0 0x7f2758108c10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2740009900 tx=0x7f274002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 -- 172.21.15.96:0/3714623698 shutdown_connections 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 --2- 172.21.15.96:0/3714623698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2758109150 0x7f275810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 --2- 172.21.15.96:0/3714623698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27581087d0 0x7f2758108c10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 --2- 172.21.15.96:0/3714623698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758102dc0 0x7f27581031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 -- 172.21.15.96:0/3714623698 >> 172.21.15.96:0/3714623698 conn(0x7f27580fe830 msgr2=0x7f2758100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:42.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 -- 172.21.15.96:0/3714623698 shutdown_connections 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.663+0000 7f275cb3c640 1 -- 172.21.15.96:0/3714623698 wait complete. 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 Processor -- start 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 -- start start 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27581087d0 0x7f27581997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758109150 0x7f275819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112e10 con 0x7f2758102dc0 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112c90 con 0x7f2758109150 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275cb3c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2758112f90 con 0x7f27581087d0 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43466/0 (socket says 172.21.15.96:43466) 2024-09-10T23:03:42.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.664+0000 7f275659c640 1 -- 172.21.15.96:0/1823078897 learned_addr learned my addr 172.21.15.96:0/1823078897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f274ffff640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27581087d0 0x7f27581997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 -- 172.21.15.96:0/1823078897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27581087d0 msgr2=0x7f27581997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27581087d0 0x7f27581997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 -- 172.21.15.96:0/1823078897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758109150 msgr2=0x7f275819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758109150 0x7f275819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 -- 172.21.15.96:0/1823078897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275819f340 con 0x7f2758102dc0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275659c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f274800c920 tx=0x7f274800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748007c70 con 0x7f2758102dc0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2748007e10 con 0x7f2758102dc0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748018820 con 0x7f2758102dc0 2024-09-10T23:03:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275819f630 con 0x7f2758102dc0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.665+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27581a5cf0 con 0x7f2758102dc0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.666+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27581031a0 con 0x7f2758102dc0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.668+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f274800f450 con 0x7f2758102dc0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.668+0000 7f274f7fe640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 0x7f27440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.668+0000 7f274ffff640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 0x7f27440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.668+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8153+0+0 (secure 0 0 0) 0x7f27480e5fb0 con 0x7f2758102dc0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.669+0000 7f274ffff640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 0x7f27440b77a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2740031440 tx=0x7f27400095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:42.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.671+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2748014030 con 0x7f2758102dc0 2024-09-10T23:03:42.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.884+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f275819a3c0 con 0x7f2758102dc0 2024-09-10T23:03:42.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.886+0000 7f274f7fe640 1 -- 172.21.15.96:0/1823078897 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f27480acfa0 con 0x7f2758102dc0 2024-09-10T23:03:42.887 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:03:42.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 msgr2=0x7f27440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:42.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 0x7f27440b77a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2740031440 tx=0x7f27400095f0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 msgr2=0x7f2758199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:42.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f274800c920 tx=0x7f274800cde0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 shutdown_connections 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2758109150 0x7f275819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f27440b52e0 0x7f27440b77a0 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2740031440 tx=0x7f27400095f0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27581087d0 0x7f27581997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 --2- 172.21.15.96:0/1823078897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2758102dc0 0x7f2758199290 secure :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f274800c920 tx=0x7f274800cde0 comp rx=0 tx=0).stop 2024-09-10T23:03:42.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.890+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 >> 172.21.15.96:0/1823078897 conn(0x7f27580fe830 msgr2=0x7f2758105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:42.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.891+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 shutdown_connections 2024-09-10T23:03:42.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:42.891+0000 7f275cb3c640 1 -- 172.21.15.96:0/1823078897 wait complete. 2024-09-10T23:03:42.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-mon[30869]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T23:03:42.927 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a[30865]: 2024-09-10T23:03:42.885+0000 7f2a63f54640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:03:43.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-mon[35081]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T23:03:43.026 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.026 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.026 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: starting mds.e at 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-mds[64983]: -- 172.21.15.117:0/1671668235 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564301379860 con 0x564301357400 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: starting mds.b at 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-mds[64683]: -- 172.21.15.117:0/1305145463 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560792c81860 con 0x560792c5e800 2024-09-10T23:03:43.027 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:03:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: starting mds.h at 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: starting mds.g at 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.250 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-mds[73532]: -- 172.21.15.96:0/2391977094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c514565860 con 0x55c514543000 2024-09-10T23:03:43.251 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: starting mds.a at 2024-09-10T23:03:43.251 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.251 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.251 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-mds[72302]: -- 172.21.15.96:0/846831317 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c3f7c5860 con 0x558c3f7a2800 2024-09-10T23:03:43.251 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:03:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: starting mds.d at 2024-09-10T23:03:43.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-mon[35119]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T23:03:43.263 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: starting mds.f at 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-mds[64966]: -- 172.21.15.121:0/1831436118 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586239f9860 con 0x5586239d7400 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: starting mds.l at 2024-09-10T23:03:43.264 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-mds[65870]: -- 172.21.15.121:0/1298058562 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d60fd3860 con 0x556d60fb1000 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: starting mds.i at 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: ignoring --setuser ceph since I am not root 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: ignoring --setgroup ceph since I am not root 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-mds[65569]: -- 172.21.15.121:0/2108158702 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56207a4ff860 con 0x56207a4dc800 2024-09-10T23:03:43.265 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:03:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: starting mds.c at 2024-09-10T23:03:43.618 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:03:43.921 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T23:03:44.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T23:03:44.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:43 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T23:03:44.026 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:43 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:44.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T23:03:44.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T23:03:44.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:03:44.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1823078897' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T23:03:44.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T23:03:44.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T23:03:44.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:43 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:03:44.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- 172.21.15.96:0/2449694962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 msgr2=0x7f7aac10f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:44.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/2449694962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac10f590 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009ca0 tx=0x7f7aa402a8c0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- 172.21.15.96:0/2449694962 shutdown_connections 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/2449694962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/2449694962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac10f590 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/2449694962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 0x7f7aac103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- 172.21.15.96:0/2449694962 >> 172.21.15.96:0/2449694962 conn(0x7f7aac0fd700 msgr2=0x7f7aac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- 172.21.15.96:0/2449694962 shutdown_connections 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- 172.21.15.96:0/2449694962 wait complete. 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 Processor -- start 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.401+0000 7f7ab3b61640 1 -- start start 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 0x7f7aac1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:44.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac112e00 con 0x7f7aac10d120 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac112c80 con 0x7f7aac104570 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab3b61640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac112f80 con 0x7f7aac103b70 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43548/0 (socket says 172.21.15.96:43548) 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 -- 172.21.15.96:0/3190100825 learned_addr learned my addr 172.21.15.96:0/3190100825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab18fd640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 0x7f7aac1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab10fc640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 -- 172.21.15.96:0/3190100825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 msgr2=0x7f7aac1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 0x7f7aac1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 -- 172.21.15.96:0/3190100825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 msgr2=0x7f7aac1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 -- 172.21.15.96:0/3190100825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19f360 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab10fc640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7ab20fe640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005cc0 tx=0x7f7aa402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.402+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4003bf0 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.403+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa4003d90 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.403+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19f5f0 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.403+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa402fbc0 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.403+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1a5d90 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.403+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac103f50 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.405+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f7aa4004610 con 0x7f7aac10d120 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.406+0000 7f7a9effd640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 0x7f7a980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:44.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.406+0000 7f7ab18fd640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 0x7f7a980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:44.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.406+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8583+0+0 (secure 0 0 0) 0x7f7aa4107320 con 0x7f7aac10d120 2024-09-10T23:03:44.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.406+0000 7f7ab18fd640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 0x7f7a980b7820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005fd0 tx=0x7f7aa000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:44.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.409+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7aa40ce080 con 0x7f7aac10d120 2024-09-10T23:03:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:44.620+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7aac1a6150 con 0x7f7aac10d120 2024-09-10T23:03:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-10T23:03:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6836/1391871731,v1:172.21.15.96:6837/1391871731] up:boot 2024-09-10T23:03:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6832/981015354,v1:172.21.15.121:6833/981015354] up:boot 2024-09-10T23:03:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6834/2065007052,v1:172.21.15.121:6835/2065007052] up:boot 2024-09-10T23:03:45.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6838/748618627,v1:172.21.15.96:6839/748618627] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6834/45398375,v1:172.21.15.117:6835/45398375] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6836/32575987,v1:172.21.15.121:6837/32575987] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.121:6838/198038801,v1:172.21.15.121:6839/198038801] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6836/460967421,v1:172.21.15.117:6837/460967421] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.96:6840/3423514040,v1:172.21.15.96:6841/3423514040] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: mds.? [v2:172.21.15.117:6838/1176539473,v1:172.21.15.117:6839/1176539473] up:boot 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:03:45.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:03:45.252 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:44 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:03:45.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6836/1391871731,v1:172.21.15.96:6837/1391871731] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6832/981015354,v1:172.21.15.121:6833/981015354] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6834/2065007052,v1:172.21.15.121:6835/2065007052] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6838/748618627,v1:172.21.15.96:6839/748618627] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6834/45398375,v1:172.21.15.117:6835/45398375] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6836/32575987,v1:172.21.15.121:6837/32575987] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.121:6838/198038801,v1:172.21.15.121:6839/198038801] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6836/460967421,v1:172.21.15.117:6837/460967421] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.96:6840/3423514040,v1:172.21.15.96:6841/3423514040] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: mds.? [v2:172.21.15.117:6838/1176539473,v1:172.21.15.117:6839/1176539473] up:boot 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T23:03:45.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:03:45.265 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:44 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6836/1391871731,v1:172.21.15.96:6837/1391871731] up:boot 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6832/981015354,v1:172.21.15.121:6833/981015354] up:boot 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6834/2065007052,v1:172.21.15.121:6835/2065007052] up:boot 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6838/748618627,v1:172.21.15.96:6839/748618627] up:boot 2024-09-10T23:03:45.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6834/45398375,v1:172.21.15.117:6835/45398375] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6836/32575987,v1:172.21.15.121:6837/32575987] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.121:6838/198038801,v1:172.21.15.121:6839/198038801] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6836/460967421,v1:172.21.15.117:6837/460967421] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.96:6840/3423514040,v1:172.21.15.96:6841/3423514040] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: mds.? [v2:172.21.15.117:6838/1176539473,v1:172.21.15.117:6839/1176539473] up:boot 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T23:03:45.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='mgr.14150 172.21.15.96:0/1128083787' entity='mgr.x' 2024-09-10T23:03:45.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:44 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:03:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.380+0000 7f7a9effd640 1 -- 172.21.15.96:0/3190100825 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f7aa40cd980 con 0x7f7aac10d120 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 msgr2=0x7f7a980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 0x7f7a980b7820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0005fd0 tx=0x7f7aa000b390 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 msgr2=0x7f7aac19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005cc0 tx=0x7f7aa402f380 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 shutdown_connections 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac104570 0x7f7aac1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f7a980b5360 0x7f7a980b7820 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac10d120 0x7f7aac19eb90 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 --2- 172.21.15.96:0/3190100825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aac103b70 0x7f7aac1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 >> 172.21.15.96:0/3190100825 conn(0x7f7aac0fd700 msgr2=0x7f7aac0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:45.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 shutdown_connections 2024-09-10T23:03:45.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:45.383+0000 7f7ab3b61640 1 -- 172.21.15.96:0/3190100825 wait complete. 2024-09-10T23:03:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:45 smithi096 ceph-mon[30869]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:03:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:45 smithi096 ceph-mon[30869]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:03:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:45 smithi096 ceph-mon[30869]: Cluster is now healthy 2024-09-10T23:03:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:45 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:03:46.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:45 smithi096 ceph-mon[30869]: fsmap 12 up:standby 2024-09-10T23:03:46.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:03:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:45 smithi121 ceph-mon[35119]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:03:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:45 smithi121 ceph-mon[35119]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:03:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:45 smithi121 ceph-mon[35119]: Cluster is now healthy 2024-09-10T23:03:46.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:45 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:03:46.264 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:45 smithi121 ceph-mon[35119]: fsmap 12 up:standby 2024-09-10T23:03:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:45 smithi117 ceph-mon[35081]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T23:03:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:45 smithi117 ceph-mon[35081]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T23:03:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:45 smithi117 ceph-mon[35081]: Cluster is now healthy 2024-09-10T23:03:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:45 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/3190100825' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:03:46.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:45 smithi117 ceph-mon[35081]: fsmap 12 up:standby 2024-09-10T23:03:46.462 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 -- 172.21.15.96:0/1049907953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac1076e0 msgr2=0x7f24ac107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 --2- 172.21.15.96:0/1049907953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac1076e0 0x7f24ac107b20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f249c009900 tx=0x7f249c02a410 comp rx=0 tx=0).stop 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 -- 172.21.15.96:0/1049907953 shutdown_connections 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 --2- 172.21.15.96:0/1049907953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 --2- 172.21.15.96:0/1049907953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac1076e0 0x7f24ac107b20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 --2- 172.21.15.96:0/1049907953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac101cd0 0x7f24ac1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 -- 172.21.15.96:0/1049907953 >> 172.21.15.96:0/1049907953 conn(0x7f24ac0fd700 msgr2=0x7f24ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:46.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 -- 172.21.15.96:0/1049907953 shutdown_connections 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.949+0000 7f24b3407640 1 -- 172.21.15.96:0/1049907953 wait complete. 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 Processor -- start 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 -- start start 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac1076e0 0x7f24ac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111cb0 con 0x7f24ac101cd0 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111b30 con 0x7f24ac1076e0 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b3407640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111e30 con 0x7f24ac108060 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b11a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b11a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45374/0 (socket says 172.21.15.96:45374) 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b11a3640 1 -- 172.21.15.96:0/1219901433 learned_addr learned my addr 172.21.15.96:0/1219901433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.950+0000 7f24b09a2640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac1076e0 0x7f24ac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b19a4640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 -- 172.21.15.96:0/1219901433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 msgr2=0x7f24ac19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 -- 172.21.15.96:0/1219901433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac1076e0 msgr2=0x7f24ac1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac1076e0 0x7f24ac1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 -- 172.21.15.96:0/1219901433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19e230 con 0x7f24ac101cd0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b19a4640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b11a3640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d950 tx=0x7f24a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8017070 con 0x7f24ac101cd0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a8009c30 con 0x7f24ac101cd0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a800f620 con 0x7f24ac101cd0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19e520 con 0x7f24ac101cd0 2024-09-10T23:03:46.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.951+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1a4be0 con 0x7f24ac101cd0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.952+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac1020b0 con 0x7f24ac101cd0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.954+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f24a8010460 con 0x7f24ac101cd0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.955+0000 7f24a27fc640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 0x7f24980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.955+0000 7f24b09a2640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 0x7f24980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.955+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8583+0+0 (secure 0 0 0) 0x7f24a80e58f0 con 0x7f24ac101cd0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.956+0000 7f24b09a2640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 0x7f24980b7820 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f249c031440 tx=0x7f249c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:46.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:46.958+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f24a80ac6d0 con 0x7f24ac101cd0 2024-09-10T23:03:47.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.143+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24ac02e5c0 con 0x7f24ac101cd0 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.389+0000 7f24a27fc640 1 -- 172.21.15.96:0/1219901433 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) ==== 181+0+0 (secure 0 0 0) 0x7f24a8009de0 con 0x7f24ac101cd0 2024-09-10T23:03:47.391 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 msgr2=0x7f24980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 0x7f24980b7820 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f249c031440 tx=0x7f249c005b50 comp rx=0 tx=0).stop 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 msgr2=0x7f24ac198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d950 tx=0x7f24a800de10 comp rx=0 tx=0).stop 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 shutdown_connections 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac1076e0 0x7f24ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f24980b5360 0x7f24980b7820 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24ac108060 0x7f24ac19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 --2- 172.21.15.96:0/1219901433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24ac101cd0 0x7f24ac198180 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:47.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.393+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 >> 172.21.15.96:0/1219901433 conn(0x7f24ac0fd700 msgr2=0x7f24ac0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:47.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.394+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 shutdown_connections 2024-09-10T23:03:47.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:47.394+0000 7f24b3407640 1 -- 172.21.15.96:0/1219901433 wait complete. 2024-09-10T23:03:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:47 smithi096 ceph-mon[30869]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:47 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:47 smithi121 ceph-mon[35119]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:47.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:47 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:47 smithi117 ceph-mon[35081]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-10T23:03:47.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:47 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:48.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:03:48.408 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:03:48.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:48 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:48.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:48 smithi096 ceph-mon[30869]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T23:03:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:48 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:48.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:48 smithi121 ceph-mon[35119]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T23:03:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:48 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/1219901433' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:48.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:48 smithi117 ceph-mon[35081]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 -- 172.21.15.96:0/2186112799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 msgr2=0x7f0b30108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 --2- 172.21.15.96:0/2186112799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30108c10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009900 tx=0x7f0b2002a410 comp rx=0 tx=0).stop 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 -- 172.21.15.96:0/2186112799 shutdown_connections 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 --2- 172.21.15.96:0/2186112799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 0x7f0b3010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 --2- 172.21.15.96:0/2186112799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30108c10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 --2- 172.21.15.96:0/2186112799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 0x7f0b301031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 -- 172.21.15.96:0/2186112799 >> 172.21.15.96:0/2186112799 conn(0x7f0b300fe830 msgr2=0x7f0b30100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 -- 172.21.15.96:0/2186112799 shutdown_connections 2024-09-10T23:03:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.926+0000 7f0b34d74640 1 -- 172.21.15.96:0/2186112799 wait complete. 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 Processor -- start 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 -- start start 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 0x7f0b30199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 0x7f0b3019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112da0 con 0x7f0b301087d0 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112c20 con 0x7f0b30102dc0 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.927+0000 7f0b34d74640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112f20 con 0x7f0b30109150 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45384/0 (socket says 172.21.15.96:45384) 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 -- 172.21.15.96:0/712685974 learned_addr learned my addr 172.21.15.96:0/712685974 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2e59c640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 0x7f0b30199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2ed9d640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 0x7f0b3019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 -- 172.21.15.96:0/712685974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 msgr2=0x7f0b3019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 0x7f0b3019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 -- 172.21.15.96:0/712685974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 msgr2=0x7f0b30199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 0x7f0b30199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 -- 172.21.15.96:0/712685974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3019f2e0 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b2dd9b640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005940 tx=0x7f0b200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2003f030 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b20003cb0 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3019f570 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20035070 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.928+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b301a5c90 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.929+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b301031a0 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.931+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 153250+0+0 (secure 0 0 0) 0x7f0b2003bbf0 con 0x7f0b301087d0 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.931+0000 7f0b277fe640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 0x7f0b1c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:03:48.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.932+0000 7f0b2e59c640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 0x7f0b1c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:03:48.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.932+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8058+0+0 (secure 0 0 0) 0x7f0b201068c0 con 0x7f0b301087d0 2024-09-10T23:03:48.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.932+0000 7f0b2e59c640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 0x7f0b1c0b7820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b18005d00 tx=0x7f0b1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:03:48.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:48.935+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0b200cd190 con 0x7f0b301087d0 2024-09-10T23:03:49.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.110+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b3019a360 con 0x7f0b301087d0 2024-09-10T23:03:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.396+0000 7f0b277fe640 1 -- 172.21.15.96:0/712685974 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) ==== 169+0+0 (secure 0 0 0) 0x7f0b200d1bc0 con 0x7f0b301087d0 2024-09-10T23:03:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-10T23:03:49.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.399+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 msgr2=0x7f0b1c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.400+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 0x7f0b1c0b7820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b18005d00 tx=0x7f0b1800b410 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.400+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 msgr2=0x7f0b30199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.400+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005940 tx=0x7f0b200042f0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 shutdown_connections 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:6800/3353555679,v1:172.21.15.96:6801/3353555679] conn(0x7f0b1c0b5360 0x7f0b1c0b7820 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b30109150 0x7f0b3019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301087d0 0x7f0b30199770 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 --2- 172.21.15.96:0/712685974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b30102dc0 0x7f0b30199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.401+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 >> 172.21.15.96:0/712685974 conn(0x7f0b300fe830 msgr2=0x7f0b30100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.402+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 shutdown_connections 2024-09-10T23:03:49.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-10T23:03:49.402+0000 7f0b34d74640 1 -- 172.21.15.96:0/712685974 wait complete. 2024-09-10T23:03:49.559 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:49 smithi096 ceph-mon[30869]: pgmap v844: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:03:49.560 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:49 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:49 smithi121 ceph-mon[35119]: pgmap v844: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:03:49.763 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:49 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:49.775 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:49 smithi117 ceph-mon[35081]: pgmap v844: 65 pgs: 65 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T23:03:49.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:49 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T23:03:50.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:03:50.154 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:03:50.185 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:03:50.213 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T23:03:50.213 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:03:50.228 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:03:50.246 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:03:50.268 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T23:03:50.269 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T23:03:50.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a 2024-09-10T23:03:50.414 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 systemd[1]: Stopping Ceph mon.a for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:50.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:50 smithi121 ceph-mon[35119]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:50.514 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:50 smithi121 ceph-mon[35119]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T23:03:50.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:50 smithi117 ceph-mon[35081]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:50.526 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:50 smithi117 ceph-mon[35081]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T23:03:50.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 ceph-mon[30869]: from='client.? 172.21.15.96:0/712685974' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T23:03:50.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 ceph-mon[30869]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T23:03:50.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a[30865]: 2024-09-10T23:03:50.591+0000 7f2a6975f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:03:50.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a[30865]: 2024-09-10T23:03:50.591+0000 7f2a6975f640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T23:03:51.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:50 smithi096 podman[116491]: 2024-09-10 23:03:50.826038489 +0000 UTC m=+0.348437846 container died d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T23:03:51.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:51 smithi096 podman[116491]: 2024-09-10 23:03:51.67592915 +0000 UTC m=+1.198328506 container cleanup d87d899ba6133e8afa32f725424a67c98916e1552ff77d45950c58cea49a2c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:03:51.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Sep 10 23:03:51 smithi096 bash[116491]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-a 2024-09-10T23:03:52.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T23:03:52.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:52.194 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T23:03:52.194 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T23:03:52.194 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.b 2024-09-10T23:03:52.518 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:52 smithi117 systemd[1]: Stopping Ceph mon.b for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b[35077]: 2024-09-10T23:03:52.516+0000 7f6eaabb4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:03:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b[35077]: 2024-09-10T23:03:52.516+0000 7f6eaabb4640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T23:03:52.776 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:52 smithi117 podman[69935]: 2024-09-10 23:03:52.700986147 +0000 UTC m=+0.310951957 container died fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T23:03:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:53 smithi117 podman[69935]: 2024-09-10 23:03:53.109172311 +0000 UTC m=+0.719138118 container cleanup fa3be3535633f4b7b1387ebb7fbc9c0c0efa319957022fdad58206174d643b44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T23:03:53.276 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 23:03:53 smithi117 bash[69935]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-b 2024-09-10T23:03:53.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T23:03:53.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:53.586 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T23:03:53.587 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T23:03:53.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.c 2024-09-10T23:03:53.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:53 smithi121 systemd[1]: Stopping Ceph mon.c for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:54.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c[35115]: 2024-09-10T23:03:53.998+0000 7f46e138d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:03:54.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c[35115]: 2024-09-10T23:03:53.998+0000 7f46e138d640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T23:03:54.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:54 smithi121 podman[71267]: 2024-09-10 23:03:54.050073395 +0000 UTC m=+0.262538784 container died 36238d29c8edf24edc54388b935e695fb0f6ce6b45fd88e0d6fc9a02ccd66de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T23:03:55.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:54 smithi121 podman[71267]: 2024-09-10 23:03:54.782567363 +0000 UTC m=+0.995032749 container cleanup 36238d29c8edf24edc54388b935e695fb0f6ce6b45fd88e0d6fc9a02ccd66de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T23:03:55.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:54 smithi121 bash[71267]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c 2024-09-10T23:03:55.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Sep 10 23:03:55 smithi121 podman[71282]: 2024-09-10 23:03:55.268341882 +0000 UTC m=+1.216037937 container remove 36238d29c8edf24edc54388b935e695fb0f6ce6b45fd88e0d6fc9a02ccd66de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T23:03:55.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T23:03:55.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:55.609 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T23:03:55.609 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T23:03:55.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x 2024-09-10T23:03:55.915 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 23:03:55 smithi096 systemd[1]: Stopping Ceph mgr.x for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:56.232 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 23:03:55 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x[31096]: 2024-09-10T23:03:55.913+0000 7f5d7fb3c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:03:56.232 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 23:03:55 smithi096 podman[116616]: 2024-09-10 23:03:55.944956396 +0000 UTC m=+0.140246009 container died 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T23:03:56.500 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 23:03:56 smithi096 podman[116616]: 2024-09-10 23:03:56.439853099 +0000 UTC m=+0.635142706 container cleanup 2513db03b94e44c3fe5698e5be01508b3faeb1076d813f4f7cc707d0f8a513c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:03:56.500 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Sep 10 23:03:56 smithi096 bash[116616]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-x 2024-09-10T23:03:56.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T23:03:56.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:56.898 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T23:03:56.898 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T23:03:56.899 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.y 2024-09-10T23:03:57.244 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 23:03:56 smithi117 systemd[1]: Stopping Ceph mgr.y for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:57.526 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 23:03:57 smithi117 podman[70059]: 2024-09-10 23:03:57.243854051 +0000 UTC m=+0.152494585 container died f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T23:03:57.892 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 23:03:57 smithi117 podman[70059]: 2024-09-10 23:03:57.639740041 +0000 UTC m=+0.548380566 container cleanup f99a087ec9e10d525fafb7d1974c6cbd70705c954f731b2262392a69374a02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T23:03:57.892 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 10 23:03:57 smithi117 bash[70059]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-y 2024-09-10T23:03:58.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T23:03:58.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:58.150 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T23:03:58.150 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T23:03:58.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.z 2024-09-10T23:03:58.476 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 23:03:58 smithi121 systemd[1]: Stopping Ceph mgr.z for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:03:58.763 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 23:03:58 smithi121 podman[71399]: 2024-09-10 23:03:58.475952756 +0000 UTC m=+0.134387512 container died 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T23:03:59.133 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 23:03:58 smithi121 podman[71399]: 2024-09-10 23:03:58.880733633 +0000 UTC m=+0.539168389 container cleanup 626facc9c8072cd9ace6cb473012eeb8f663142d669fb01fc3b2c265ea3ec0a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T23:03:59.133 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Sep 10 23:03:58 smithi121 bash[71399]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mgr-z 2024-09-10T23:03:59.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T23:03:59.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:03:59.286 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T23:03:59.286 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T23:03:59.286 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.0 2024-09-10T23:03:59.586 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:03:59 smithi096 systemd[1]: Stopping Ceph osd.0 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:00.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:03:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T23:03:59.584+0000 7f3c1d9f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:00.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:03:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T23:03:59.584+0000 7f3c1d9f2640 -1 osd.0 80 *** Got signal Terminated *** 2024-09-10T23:04:00.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:03:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0[47159]: 2024-09-10T23:03:59.584+0000 7f3c1d9f2640 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:05.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:05 smithi096 podman[116737]: 2024-09-10 23:04:05.210331518 +0000 UTC m=+5.732337294 container stop 17414d68ceef1feb2290cc98a0e4bf30422063aff24b70705a0228c8dc65825c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T23:04:05.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:05 smithi096 podman[116737]: 2024-09-10 23:04:05.236028029 +0000 UTC m=+5.758033796 container died 17414d68ceef1feb2290cc98a0e4bf30422063aff24b70705a0228c8dc65825c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:04:05.913 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:05 smithi096 podman[116737]: 2024-09-10 23:04:05.719127917 +0000 UTC m=+6.241133728 container cleanup 17414d68ceef1feb2290cc98a0e4bf30422063aff24b70705a0228c8dc65825c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T23:04:05.913 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:05 smithi096 bash[116737]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0 2024-09-10T23:04:06.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:05 smithi096 podman[116754]: 2024-09-10 23:04:05.912588609 +0000 UTC m=+0.695459351 container remove 17414d68ceef1feb2290cc98a0e4bf30422063aff24b70705a0228c8dc65825c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T23:04:06.659 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.236840488 +0000 UTC m=+0.015828702 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:06.660 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.372115425 +0000 UTC m=+0.151103643 container create d82f00dde1d4ee3210e86db612772bdf10b8d2d07e7021519f8ea482462669c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T23:04:07.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.658647763 +0000 UTC m=+0.437635988 container init d82f00dde1d4ee3210e86db612772bdf10b8d2d07e7021519f8ea482462669c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T23:04:07.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.662510919 +0000 UTC m=+0.441499143 container start d82f00dde1d4ee3210e86db612772bdf10b8d2d07e7021519f8ea482462669c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:07.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.758454654 +0000 UTC m=+0.537442886 container attach d82f00dde1d4ee3210e86db612772bdf10b8d2d07e7021519f8ea482462669c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:04:07.000 INFO:journalctl@ceph.osd.0.smithi096.stdout:Sep 10 23:04:06 smithi096 podman[116837]: 2024-09-10 23:04:06.856505213 +0000 UTC m=+0.635493430 container died d82f00dde1d4ee3210e86db612772bdf10b8d2d07e7021519f8ea482462669c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T23:04:07.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T23:04:07.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:07.640 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T23:04:07.640 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T23:04:07.640 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.1 2024-09-10T23:04:07.961 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:07 smithi117 systemd[1]: Stopping Ceph osd.1 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:08.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T23:04:07.959+0000 7f633ffe1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:08.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T23:04:07.959+0000 7f633ffe1640 -1 osd.1 80 *** Got signal Terminated *** 2024-09-10T23:04:08.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1[39552]: 2024-09-10T23:04:07.959+0000 7f633ffe1640 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:13.776 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:13 smithi117 podman[70185]: 2024-09-10 23:04:13.516164883 +0000 UTC m=+5.679965402 container died eb416a8bea2aae9754461dad28850ef77d3357fe19bba20000d1b46006636227 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:04:14.178 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:13 smithi117 podman[70185]: 2024-09-10 23:04:13.918074976 +0000 UTC m=+6.081875485 container cleanup eb416a8bea2aae9754461dad28850ef77d3357fe19bba20000d1b46006636227 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T23:04:14.178 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:13 smithi117 bash[70185]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1 2024-09-10T23:04:14.526 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70203]: 2024-09-10 23:04:14.177852339 +0000 UTC m=+0.661571742 container remove eb416a8bea2aae9754461dad28850ef77d3357fe19bba20000d1b46006636227 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:04:14.840 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70289]: 2024-09-10 23:04:14.435361699 +0000 UTC m=+0.015805293 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:14.840 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70289]: 2024-09-10 23:04:14.562204391 +0000 UTC m=+0.142647992 container create 2bf7695c498b69d7a9ade92776d4049c546eb4a15c6de260c048be879d651f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:15.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70289]: 2024-09-10 23:04:14.839947859 +0000 UTC m=+0.420391465 container init 2bf7695c498b69d7a9ade92776d4049c546eb4a15c6de260c048be879d651f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T23:04:15.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70289]: 2024-09-10 23:04:14.843767253 +0000 UTC m=+0.424210844 container start 2bf7695c498b69d7a9ade92776d4049c546eb4a15c6de260c048be879d651f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T23:04:15.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:14 smithi117 podman[70289]: 2024-09-10 23:04:14.973531709 +0000 UTC m=+0.553975380 container attach 2bf7695c498b69d7a9ade92776d4049c546eb4a15c6de260c048be879d651f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:04:15.276 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 10 23:04:15 smithi117 podman[70289]: 2024-09-10 23:04:15.050072955 +0000 UTC m=+0.630516544 container died 2bf7695c498b69d7a9ade92776d4049c546eb4a15c6de260c048be879d651f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:04:15.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T23:04:15.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:15.792 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T23:04:15.793 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T23:04:15.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.2 2024-09-10T23:04:16.110 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:15 smithi121 systemd[1]: Stopping Ceph osd.2 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:16.513 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T23:04:16.107+0000 7f00ff28a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:16.513 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T23:04:16.107+0000 7f00ff28a640 -1 osd.2 80 *** Got signal Terminated *** 2024-09-10T23:04:16.514 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2[40101]: 2024-09-10T23:04:16.107+0000 7f00ff28a640 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:21.804 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:21 smithi121 podman[71526]: 2024-09-10 23:04:21.526929766 +0000 UTC m=+5.540502906 container died 3b6da4fd8831f4cbe319f194ace4dee81d7062daf93dcc98269b1ee4152bec48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:04:22.188 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:21 smithi121 podman[71526]: 2024-09-10 23:04:21.910971506 +0000 UTC m=+5.924544645 container cleanup 3b6da4fd8831f4cbe319f194ace4dee81d7062daf93dcc98269b1ee4152bec48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T23:04:22.188 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:21 smithi121 bash[71526]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2 2024-09-10T23:04:22.513 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71542]: 2024-09-10 23:04:22.187400365 +0000 UTC m=+0.656657708 container remove 3b6da4fd8831f4cbe319f194ace4dee81d7062daf93dcc98269b1ee4152bec48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:22.851 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71632]: 2024-09-10 23:04:22.425957478 +0000 UTC m=+0.016300042 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:22.851 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71632]: 2024-09-10 23:04:22.571900999 +0000 UTC m=+0.162243552 container create c7302a5429afd4de3c0c21031109c2ee69b66bb348ddf79e167fa50840a900c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:22.851 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71632]: 2024-09-10 23:04:22.8504486 +0000 UTC m=+0.440791158 container init c7302a5429afd4de3c0c21031109c2ee69b66bb348ddf79e167fa50840a900c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T23:04:23.263 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71632]: 2024-09-10 23:04:22.853866618 +0000 UTC m=+0.444209175 container start c7302a5429afd4de3c0c21031109c2ee69b66bb348ddf79e167fa50840a900c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T23:04:23.264 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:22 smithi121 podman[71632]: 2024-09-10 23:04:22.941946108 +0000 UTC m=+0.532288674 container attach c7302a5429afd4de3c0c21031109c2ee69b66bb348ddf79e167fa50840a900c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:23.264 INFO:journalctl@ceph.osd.2.smithi121.stdout:Sep 10 23:04:23 smithi121 podman[71632]: 2024-09-10 23:04:23.045315344 +0000 UTC m=+0.635657898 container died c7302a5429afd4de3c0c21031109c2ee69b66bb348ddf79e167fa50840a900c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T23:04:23.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T23:04:23.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:23.773 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T23:04:23.774 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T23:04:23.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.3 2024-09-10T23:04:24.099 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:23 smithi096 systemd[1]: Stopping Ceph osd.3 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:24.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:24 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T23:04:24.097+0000 7f99da7cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:24.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:24 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T23:04:24.097+0000 7f99da7cd640 -1 osd.3 80 *** Got signal Terminated *** 2024-09-10T23:04:24.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:24 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T23:04:24.097+0000 7f99da7cd640 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:27.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:27.164+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:28.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:27 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:27.692+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:28.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:27 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:27.537+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:28.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:27.584+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:28.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:27 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:27.648+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:28.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:28.207+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:29.000 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:28 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3[53612]: 2024-09-10T23:04:28.664+0000 7f99d2dda640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.408823+0000 front 2024-09-10T23:04:04.409244+0000 (oldest deadline 2024-09-10T23:04:27.908965+0000) 2024-09-10T23:04:29.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:28 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:28.537+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:29.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:28 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:28.738+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:29.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:28.569+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:29.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:28 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:28.673+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:29.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:29.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:29.921 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:29 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:29.697+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:29.921 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:29 smithi096 podman[116965]: 2024-09-10 23:04:29.623610651 +0000 UTC m=+5.645576106 container died a92205c58d6e13b2291812afb5ffc17a259992dbe553bcff249b536774b2d209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T23:04:29.922 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:29 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:29.510+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:30.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:29.601+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:30.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:29 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:29.658+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:30.250 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:30 smithi096 podman[116965]: 2024-09-10 23:04:30.122774675 +0000 UTC m=+6.144740129 container cleanup a92205c58d6e13b2291812afb5ffc17a259992dbe553bcff249b536774b2d209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T23:04:30.250 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:30 smithi096 bash[116965]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3 2024-09-10T23:04:30.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:30.246+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:30.705 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:30 smithi096 podman[116980]: 2024-09-10 23:04:30.391378048 +0000 UTC m=+0.771573016 container remove a92205c58d6e13b2291812afb5ffc17a259992dbe553bcff249b536774b2d209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:04:30.705 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:30 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:30.553+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:31.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:30 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:30.737+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:31.000 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:30 smithi096 podman[117068]: 2024-09-10 23:04:30.606972224 +0000 UTC m=+0.015857763 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:31.000 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:30 smithi096 podman[117068]: 2024-09-10 23:04:30.73384476 +0000 UTC m=+0.142730398 container create fc188085776acb66c2599747ef063ca061bde33ad2999662eabaa98627b4eeeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:31.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:30.646+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:31.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:30 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:30.648+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:31.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:31 smithi096 podman[117068]: 2024-09-10 23:04:31.015317941 +0000 UTC m=+0.424203468 container init fc188085776acb66c2599747ef063ca061bde33ad2999662eabaa98627b4eeeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:04:31.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:31 smithi096 podman[117068]: 2024-09-10 23:04:31.018997654 +0000 UTC m=+0.427883182 container start fc188085776acb66c2599747ef063ca061bde33ad2999662eabaa98627b4eeeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:04:31.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:31 smithi096 podman[117068]: 2024-09-10 23:04:31.136408321 +0000 UTC m=+0.545293856 container attach fc188085776acb66c2599747ef063ca061bde33ad2999662eabaa98627b4eeeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T23:04:31.500 INFO:journalctl@ceph.osd.3.smithi096.stdout:Sep 10 23:04:31 smithi096 podman[117068]: 2024-09-10 23:04:31.208475242 +0000 UTC m=+0.617360780 container died fc188085776acb66c2599747ef063ca061bde33ad2999662eabaa98627b4eeeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:04:31.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:31.209+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:31.823 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:31 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:31.552+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:31.824 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:31 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:31.712+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:31.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T23:04:31.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:31.883 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T23:04:31.883 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T23:04:31.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.4 2024-09-10T23:04:31.911 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:31 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:31.627+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:32.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:31.651+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:32.205 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:31 smithi117 systemd[1]: Stopping Ceph osd.4 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:32.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:32.209+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:32.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:32 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:32.203+0000 7ff38345c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:32.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:32 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:32.203+0000 7ff38345c640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-10T23:04:32.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:32 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:32.203+0000 7ff38345c640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:33.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:32.601+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:33.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:32 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:32.685+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:33.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:32.696+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:33.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:32 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:32.634+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:33.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:33.162+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:34.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:33 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:33.600+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:34.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:33 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:33.661+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:34.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:33.675+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:34.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:33 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:33.637+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:34.456 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:34 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:34.165+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:34.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:34 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:34.270+0000 7ff37fa71640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:07.790192+0000 front 2024-09-10T23:04:07.790300+0000 (oldest deadline 2024-09-10T23:04:33.690424+0000) 2024-09-10T23:04:34.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:34 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:34.693+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:34.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:34 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:34.454+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:35.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:34 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:34.605+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:35.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:34 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:34.688+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:35.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:34 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:34.664+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:35.513 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:35.410+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:35.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:35.171+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:35.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:35 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:35.239+0000 7ff37fa71640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:07.790192+0000 front 2024-09-10T23:04:07.790300+0000 (oldest deadline 2024-09-10T23:04:33.690424+0000) 2024-09-10T23:04:35.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:35 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:35.161+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:36.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:35 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:35.577+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:36.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:35 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:35.696+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:36.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:35.673+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:36.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:35 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:35.673+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:36.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:35 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:35.634+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:36.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:35 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:35.634+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:36.405 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:36 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:36.126+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:36.405 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:36 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:36.126+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:36.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:36 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:36.217+0000 7ff37fa71640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:07.790192+0000 front 2024-09-10T23:04:07.790300+0000 (oldest deadline 2024-09-10T23:04:33.690424+0000) 2024-09-10T23:04:36.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:36 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:36.124+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:36.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:36 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:36.404+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:36.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:36 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:36.632+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:36.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:36 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:36.632+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:37.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:36.600+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:37.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:36.707+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:37.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:36.707+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:37.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:36 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:36.648+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:37.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:36 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:36.648+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:37.513 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:37.361+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:37.568 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:37.171+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:37.568 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:37.171+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:37.568 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:37 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4[46169]: 2024-09-10T23:04:37.180+0000 7ff37fa71640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:07.790192+0000 front 2024-09-10T23:04:07.790300+0000 (oldest deadline 2024-09-10T23:04:33.690424+0000) 2024-09-10T23:04:37.569 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:37 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:37.082+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:38.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:37 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:37.633+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:38.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:37 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:37.633+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:38.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:37 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:37.751+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:38.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:37 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:37.751+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:38.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:37.678+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:38.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:37 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:37.678+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:38.026 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:37 smithi117 podman[70415]: 2024-09-10 23:04:37.810097134 +0000 UTC m=+5.732397141 container died 63c8a705fa28523e2c165cbf017addb1eb1fd2a66cd2ab20de36fa03431cfc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T23:04:38.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:37 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:37.624+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:38.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:37 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:37.624+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:38.439 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:38 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:38.067+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:38.439 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:38 smithi117 podman[70415]: 2024-09-10 23:04:38.212088835 +0000 UTC m=+6.134388845 container cleanup 63c8a705fa28523e2c165cbf017addb1eb1fd2a66cd2ab20de36fa03431cfc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:04:38.439 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:38 smithi117 bash[70415]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4 2024-09-10T23:04:38.513 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:38 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:38.340+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:38.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:38 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:38.196+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:38.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:38 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:38.196+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:38.776 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:38 smithi117 podman[70430]: 2024-09-10 23:04:38.438477753 +0000 UTC m=+0.629740395 container remove 63c8a705fa28523e2c165cbf017addb1eb1fd2a66cd2ab20de36fa03431cfc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:04:38.776 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:38 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:38.631+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:38.776 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:38 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:38.631+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:39.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:38 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:38.624+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:39.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:38 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:38.624+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:39.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:38 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:38.768+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:39.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:38 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:38.768+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:39.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:38 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:38.718+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:39.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:38 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:38.718+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:39.192 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:39 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:39.020+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:39.193 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:38 smithi117 podman[70522]: 2024-09-10 23:04:38.781334541 +0000 UTC m=+0.015948118 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:39.193 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:38 smithi117 podman[70522]: 2024-09-10 23:04:38.906060757 +0000 UTC m=+0.140674325 container create c98e15655e33bb956965dfae5e5ff0375fb2c50b78ccb900f4242ab2ee50d8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T23:04:39.193 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:39 smithi117 podman[70522]: 2024-09-10 23:04:39.192120921 +0000 UTC m=+0.426734490 container init c98e15655e33bb956965dfae5e5ff0375fb2c50b78ccb900f4242ab2ee50d8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:04:39.513 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:39 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:39.390+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:39.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:39 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:39.176+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:39.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:39 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:39.176+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:39.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:39 smithi117 podman[70522]: 2024-09-10 23:04:39.19571494 +0000 UTC m=+0.430328512 container start c98e15655e33bb956965dfae5e5ff0375fb2c50b78ccb900f4242ab2ee50d8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T23:04:39.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:39 smithi117 podman[70522]: 2024-09-10 23:04:39.291989761 +0000 UTC m=+0.526603343 container attach c98e15655e33bb956965dfae5e5ff0375fb2c50b78ccb900f4242ab2ee50d8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T23:04:39.526 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 23:04:39 smithi117 podman[70522]: 2024-09-10 23:04:39.383351867 +0000 UTC m=+0.617965442 container died c98e15655e33bb956965dfae5e5ff0375fb2c50b78ccb900f4242ab2ee50d8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:39.984 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:39 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:39.643+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:39.984 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:39 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:39.643+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:39.999 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:39.635+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:40.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:39.635+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:40.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:39.806+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:40.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:39 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:39.806+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:40.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:39 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:39.703+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:40.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:39 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:39.703+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:40.071 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T23:04:40.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:40.153 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T23:04:40.153 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T23:04:40.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.5 2024-09-10T23:04:40.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:39 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:39.982+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:40.430 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 systemd[1]: Stopping Ceph osd.5 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:40.430 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:40.150+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:40.430 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:40.150+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:40.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:40.428+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:40.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:40.473+0000 7f89bd766640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:40.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:40.473+0000 7f89bd766640 -1 osd.5 80 *** Got signal Terminated *** 2024-09-10T23:04:40.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:40.473+0000 7f89bd766640 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:40.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:40.654+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:40.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:40 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:40.654+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:40.978 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:40 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:40.666+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:40.979 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:40 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:40.666+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:41.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:40.653+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:41.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:40.653+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:41.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:40.799+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:41.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:40.799+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:41.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:40 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:40.977+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:41.450 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:41 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:41.120+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:41.450 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:41 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:41.120+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:41.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:41 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:41.449+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:41.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:41 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:41.686+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:41.763 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:41 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:41.686+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:41.944 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:41 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:41.658+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:41.945 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:41 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:41.658+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:41.999 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:41 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:41.685+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:42.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:41 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:41.685+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:42.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:41 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:41.819+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:42.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:41 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:41.819+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:42.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:41 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:41.943+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:42.402 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:42.076+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:42.402 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:42.076+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:42.402 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:42.076+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:42.730 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:42.400+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:43.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:42.650+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:43.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:42.650+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:43.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:42.851+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:43.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:42 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:42.851+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:43.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:42.729+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:43.013 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:42 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:42.729+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:43.026 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:42.920+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:43.026 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:42.920+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:43.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:42.700+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:43.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:42 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:42.700+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:43.422 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:43.088+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:43.422 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:43.088+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:43.422 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:43.088+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:43.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:43.421+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:43.682+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:43.682+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:43.682+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:43.823+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:43.823+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:44.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:43 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:43.823+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:44.026 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:43.929+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:44.026 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:43.929+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:44.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:43.724+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:44.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:43 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:43.724+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:44.108 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:43.772+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:44.108 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:43.772+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:44.108 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:43 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:43.772+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.002319+0000 front 2024-09-10T23:04:20.002209+0000 (oldest deadline 2024-09-10T23:04:43.502302+0000) 2024-09-10T23:04:44.468 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:44.107+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:44.468 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:44.107+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:44.468 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:44.107+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:44.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:44.466+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:44.978 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:44 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:44.717+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:44.978 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:44 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:44.717+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:44.711+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:44.711+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:44.711+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:44.832+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:44.832+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:45.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:44.832+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:45.062 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:44.794+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:03.600368+0000 front 2024-09-10T23:04:03.600453+0000 (oldest deadline 2024-09-10T23:04:27.100380+0000) 2024-09-10T23:04:45.062 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:44.794+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.401575+0000 front 2024-09-10T23:04:12.401593+0000 (oldest deadline 2024-09-10T23:04:34.701520+0000) 2024-09-10T23:04:45.062 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:44 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5[47014]: 2024-09-10T23:04:44.794+0000 7f89b5d73640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.002319+0000 front 2024-09-10T23:04:20.002209+0000 (oldest deadline 2024-09-10T23:04:43.502302+0000) 2024-09-10T23:04:45.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:44 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:44.976+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:45.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:44 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:44.976+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:45.488 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:45 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:45.061+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:45.488 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:45 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:45.061+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:45.488 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:45 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:45.061+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:45.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:45 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:45.486+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:45.735+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:45.735+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:45.735+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:45.793+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:45.793+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:46.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:45 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:45.793+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:46.021 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:45 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:45.708+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:46.021 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:45 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:45.708+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:46.263 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:45 smithi121 podman[71746]: 2024-09-10 23:04:45.949579956 +0000 UTC m=+5.600362934 container died 1a116bef3803b98c4a09eea2494efa9f7e80fc7b7db4bc01265f4ab03f55ca59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T23:04:46.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.031+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:46.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.031+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:46.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.031+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:46.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:46.019+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:46.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:46.019+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:46.618 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:46.505+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:46.618 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:46.505+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:46.618 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:46 smithi121 podman[71746]: 2024-09-10 23:04:46.35701777 +0000 UTC m=+6.007800755 container cleanup 1a116bef3803b98c4a09eea2494efa9f7e80fc7b7db4bc01265f4ab03f55ca59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T23:04:46.618 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:46 smithi121 bash[71746]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5 2024-09-10T23:04:46.987 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:46.718+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:46.988 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:46.718+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:46.988 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:46.718+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:46.994 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:46 smithi121 podman[71762]: 2024-09-10 23:04:46.617031952 +0000 UTC m=+0.668367694 container remove 1a116bef3803b98c4a09eea2494efa9f7e80fc7b7db4bc01265f4ab03f55ca59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:46.687+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:46.687+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:46.687+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:46.765+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:46.765+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:47.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:46 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:46.765+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:47.263 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:46.927616637 +0000 UTC m=+0.015934960 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:47.264 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:47.059673574 +0000 UTC m=+0.147991888 container create f1b75770213169209bc2fb68677fc49fb561ae060b1edb8731481df0c381f584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T23:04:47.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.993+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:47.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.993+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:47.264 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:46 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:46.993+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:47.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:46.986+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:47.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:46 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:46.986+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:47.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:47 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:47.536+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:47.764 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:47 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:47.536+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:47.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:47.31258634 +0000 UTC m=+0.400904653 container init f1b75770213169209bc2fb68677fc49fb561ae060b1edb8731481df0c381f584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:47.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:47.315911612 +0000 UTC m=+0.404229926 container start f1b75770213169209bc2fb68677fc49fb561ae060b1edb8731481df0c381f584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:04:47.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:47.404443101 +0000 UTC m=+0.492761419 container attach f1b75770213169209bc2fb68677fc49fb561ae060b1edb8731481df0c381f584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T23:04:47.764 INFO:journalctl@ceph.osd.5.smithi121.stdout:Sep 10 23:04:47 smithi121 podman[71851]: 2024-09-10 23:04:47.506642923 +0000 UTC m=+0.594961237 container died f1b75770213169209bc2fb68677fc49fb561ae060b1edb8731481df0c381f584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:47.734+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:47.734+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:47.734+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:47.771+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:47.771+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:48.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:47 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:47.771+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:48.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:47 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:47.714+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:48.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:47 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:47.714+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:48.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:47 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:47.714+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:48.200 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:48 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:48.011+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:48.201 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:48 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:48.011+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:48.201 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:48 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:48.011+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:48.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T23:04:48.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:48.261 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T23:04:48.262 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T23:04:48.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.6 2024-09-10T23:04:48.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:48 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:48.012+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:48.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:48 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:48.012+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:48.642 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 systemd[1]: Stopping Ceph osd.6 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:48.643 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.640+0000 7f426cef4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:48.643 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.640+0000 7f426cef4640 -1 osd.6 80 *** Got signal Terminated *** 2024-09-10T23:04:48.643 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.640+0000 7f426cef4640 -1 osd.6 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:48.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:48 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:48.506+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:48.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:48 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:48.506+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:48.768+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:48.768+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:48.768+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.705+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.705+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:49.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:48.705+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:49.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:48 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:48.703+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:49.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:48 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:48.703+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:49.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:48 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:48.703+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:49.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:49 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:49.042+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:49.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:49 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:49.042+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:49.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:49 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:49.042+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:49.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:49 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:49.058+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:49.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:49 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:49.058+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:49.764+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:49.764+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:49.764+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:49.679+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:49.679+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:50.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:49 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:49.679+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:50.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:49 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:49.539+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:50.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:49 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:49.539+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:50.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:49 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:49.697+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:50.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:49 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:49.697+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:50.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:49 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:49.697+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:50.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:50.083+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:50.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:50.083+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:50.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:50.083+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:50.525 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:50.085+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:50.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:50.085+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:50.728+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:50.728+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:50.728+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:50.644+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:50.644+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:51.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:50 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:50.644+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:51.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:50.581+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:51.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:50 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:50.581+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:51.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:50.739+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:51.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:50.739+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:51.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:50.739+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:51.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:50 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:50.739+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:51.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:51 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:51.128+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:51.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:51 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:51.128+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:51.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:51 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:51.128+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:51.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:51.107+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:51.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:51.107+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:51.773+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:51.773+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:51.773+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:51.650+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:51.650+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:52.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:51 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:51.650+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:52.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:51 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:51.566+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:52.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:51 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:51.566+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:52.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:51.759+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:52.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:51.759+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:52.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:51.759+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:52.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:51 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:51.759+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:52.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:52.153+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:52.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:52.153+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:52.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:52.153+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:52.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:52.078+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:52.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:52.078+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:52.647+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:02.099848+0000 front 2024-09-10T23:04:02.099885+0000 (oldest deadline 2024-09-10T23:04:26.799923+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:52.647+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.700663+0000 front 2024-09-10T23:04:12.700746+0000 (oldest deadline 2024-09-10T23:04:37.400760+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6[60257]: 2024-09-10T23:04:52.647+0000 7f4269509640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.401531+0000 front 2024-09-10T23:04:17.401360+0000 (oldest deadline 2024-09-10T23:04:43.301409+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:52.739+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:52.739+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:52.739+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:53.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:52 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:52.739+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:53.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:52.576+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:53.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:52 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:52.576+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:53.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:52.715+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:53.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:52.715+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:53.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:52.715+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:53.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:52 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:52.715+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:53.281 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:53.165+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:53.281 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:53.165+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:53.282 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:53.165+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:53.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:53.115+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:53.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:53.115+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:54.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:53 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:53.724+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:54.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:53 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:53.724+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:54.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:53 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:53.724+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:54.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:53 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:53.724+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:54.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:53.595+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:54.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:53.595+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:54.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:53 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:53.595+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:54.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:53.757+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:54.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:53.757+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:54.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:53.757+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:54.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:53.757+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:54.300 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:54 smithi096 podman[117171]: 2024-09-10 23:04:54.047711053 +0000 UTC m=+5.591002452 container died 1bfa2703310c012b510f699954fc316c8f20765002cb05961bf2de73d91abc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-10T23:04:54.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:54.213+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:54.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:54.213+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:54.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:54.213+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:54.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:54.130+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:54.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:54.130+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:54.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:54.130+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:54.677 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:54 smithi096 podman[117171]: 2024-09-10 23:04:54.474266995 +0000 UTC m=+6.017558437 container cleanup 1bfa2703310c012b510f699954fc316c8f20765002cb05961bf2de73d91abc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T23:04:54.677 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:54 smithi096 bash[117171]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6 2024-09-10T23:04:55.000 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:54 smithi096 podman[117201]: 2024-09-10 23:04:54.675623032 +0000 UTC m=+0.627802432 container remove 1bfa2703310c012b510f699954fc316c8f20765002cb05961bf2de73d91abc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-10T23:04:55.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:54 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:54.678+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:55.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:54 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:54.678+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:55.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:54 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:54.678+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:55.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:54 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:54.678+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:55.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:54.616+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:55.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:54.616+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:55.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:54 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:54.616+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:55.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:54.773+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:55.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:54.773+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:55.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:54.773+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:55.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:54 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:54.773+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:55.363 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:54.98831321 +0000 UTC m=+0.015826378 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:04:55.363 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:55.109787389 +0000 UTC m=+0.137300553 container create 427a7956d5932ba411bab4f532e4c930e78332daeb6e95eabe79a31a38c9eb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:55.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:55.256+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:55.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:55.256+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:55.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:55.256+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:55.525 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:55.140+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:55.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:55.140+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:55.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:55.140+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:55.691 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:55.362772131 +0000 UTC m=+0.390285301 container init 427a7956d5932ba411bab4f532e4c930e78332daeb6e95eabe79a31a38c9eb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T23:04:55.692 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:55.366812952 +0000 UTC m=+0.394326115 container start 427a7956d5932ba411bab4f532e4c930e78332daeb6e95eabe79a31a38c9eb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T23:04:55.692 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:55.454176823 +0000 UTC m=+0.481689990 container attach 427a7956d5932ba411bab4f532e4c930e78332daeb6e95eabe79a31a38c9eb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:04:55.692 INFO:journalctl@ceph.osd.6.smithi096.stdout:Sep 10 23:04:55 smithi096 podman[117288]: 2024-09-10 23:04:55.552480267 +0000 UTC m=+0.579993430 container died 427a7956d5932ba411bab4f532e4c930e78332daeb6e95eabe79a31a38c9eb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T23:04:56.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:55 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:55.690+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:56.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:55 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:55.690+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:56.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:55 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:55.690+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:56.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:55 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:55.690+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:56.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:55.623+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:56.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:55.623+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:56.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:55 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:55.623+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:56.191 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:55.778+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:56.191 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:55.778+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:56.191 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:55.778+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:56.191 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:55 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:55.778+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:56.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T23:04:56.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:04:56.268 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T23:04:56.268 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T23:04:56.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.7 2024-09-10T23:04:56.487 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.190+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:56.487 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.190+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:56.488 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.190+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:56.488 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 systemd[1]: Stopping Ceph osd.7 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:04:56.609 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:56.282+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:56.609 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:56.282+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:56.609 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:56.282+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:56.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.597+0000 7f5926c18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:04:56.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.597+0000 7f5926c18640 -1 osd.7 80 *** Got signal Terminated *** 2024-09-10T23:04:56.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:56.597+0000 7f5926c18640 -1 osd.7 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:04:56.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:56 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:56.671+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:57.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:56 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:56.671+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:57.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:56 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:56.671+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:57.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:56 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:56.671+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:57.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:56.608+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:57.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:56.608+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:57.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:56 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:56.608+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:57.178 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:56.817+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:57.178 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:56.817+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:57.178 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:56.817+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:57.178 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:56.817+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:57.441 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:57.176+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:57.441 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:57.176+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:57.441 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:57.176+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:57.565 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:57.294+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:57.565 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:57.294+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:57.565 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:57.294+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:58.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:57 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:57.667+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:58.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:57 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:57.667+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:58.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:57 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:57.667+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:58.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:57 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:57.667+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:58.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:57.564+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:58.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:57.564+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:58.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:57 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:57.564+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:58.163 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:57.866+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:58.164 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:57.866+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:58.164 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:57.866+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:58.164 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:57.866+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:58.164 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:57 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:57.866+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:04:58.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:58.162+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:58.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:58.162+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:58.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:58.162+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:58.588 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:58.311+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:58.588 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:58.311+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:58.588 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:58.311+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:04:59.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:58.677+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:04:59.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:58.677+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:04:59.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:58.677+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:04:59.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:58 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:58.677+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:04:59.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:58.587+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:04:59.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:58.587+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:04:59.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:58 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:58.587+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:04:59.174 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:58.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:04:59.174 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:58.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:04:59.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:58.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:04:59.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:58.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:04:59.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:58 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:58.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:04:59.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:59.173+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:04:59.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:59.173+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:04:59.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:04:59.173+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:04:59.622 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:59.338+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:04:59.622 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:59.338+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:04:59.622 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:04:59.338+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:00.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:59.653+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:00.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:59.653+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:00.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:59.653+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:00.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:59.653+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:00.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:04:59 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:04:59.653+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:00.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:59.621+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:00.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:59.621+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:00.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:04:59 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:04:59.621+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:00.203 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:59.925+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:00.203 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:59.925+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:00.203 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:59.925+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:00.203 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:59.925+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:00.203 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:04:59 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:04:59.925+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:00.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:00.201+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:05:00.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:00.201+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:05:00.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:00.201+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:05:00.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:00.201+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:34.894421+0000 front 2024-09-10T23:04:34.894547+0000 (oldest deadline 2024-09-10T23:04:59.495003+0000) 2024-09-10T23:05:00.639 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:00.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:00.639 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:00.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:00.639 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:00.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:01.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:00 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:00.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:01.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:00 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:00.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:01.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:00 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:00.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:01.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:00 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:00.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:01.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:00 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:00.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:01.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:00.637+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:01.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:00.637+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:01.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:00.637+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:01.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:00 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:00.637+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:01.213 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:00.905+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:01.214 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:00.905+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:01.214 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:00.905+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:01.214 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:00.905+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:01.214 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:00.905+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:01.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:01.212+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.091829+0000 front 2024-09-10T23:04:09.091803+0000 (oldest deadline 2024-09-10T23:04:34.391923+0000) 2024-09-10T23:05:01.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:01.212+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.092576+0000 front 2024-09-10T23:04:19.092648+0000 (oldest deadline 2024-09-10T23:04:42.592652+0000) 2024-09-10T23:05:01.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:01.212+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:27.893585+0000 front 2024-09-10T23:04:27.893669+0000 (oldest deadline 2024-09-10T23:04:53.793465+0000) 2024-09-10T23:05:01.526 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7[52654]: 2024-09-10T23:05:01.212+0000 7f592322d640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:34.894421+0000 front 2024-09-10T23:04:34.894547+0000 (oldest deadline 2024-09-10T23:04:59.495003+0000) 2024-09-10T23:05:01.613 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:01.355+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:01.613 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:01.355+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:01.613 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:01.355+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:02.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:01.681+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:02.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:01.681+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:02.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:01.681+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:02.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:01.681+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:02.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:01 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:01.681+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:02.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:01.612+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:02.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:01.612+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:02.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:01.612+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:02.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:01 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:01.612+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:02.276 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:02 smithi117 podman[70626]: 2024-09-10 23:05:02.128986966 +0000 UTC m=+5.663197462 container died 5e52a1bcae4ccc3e613e136168c6e521bc7df5f1f089703821cfe43fec3a8e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:05:02.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:01.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:02.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:01.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:02.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:01.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:02.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:01.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:02.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:01 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:01.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:02.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:02.369+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:02.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:02.369+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:02.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:02.369+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:02.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:02.369+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:02.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:02 smithi117 podman[70626]: 2024-09-10 23:05:02.563220097 +0000 UTC m=+6.097430593 container cleanup 5e52a1bcae4ccc3e613e136168c6e521bc7df5f1f089703821cfe43fec3a8e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T23:05:02.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:02 smithi117 bash[70626]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7 2024-09-10T23:05:03.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:02 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:02.651+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:03.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:02 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:02.651+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:03.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:02 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:02.651+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:03.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:02 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:02.651+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:03.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:02 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:02.651+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:03.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:02.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:03.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:02.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:03.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:02.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:03.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:02 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:02.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:03.225 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:02 smithi117 podman[70653]: 2024-09-10 23:05:02.806194751 +0000 UTC m=+0.674435214 container remove 5e52a1bcae4ccc3e613e136168c6e521bc7df5f1f089703821cfe43fec3a8e8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:05:03.225 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:02 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:02.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:03.225 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:02 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:02.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:03.225 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:02 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:02.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:03.225 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:02 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:02.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:03.226 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:02 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:02.923+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:03.476 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.12742022 +0000 UTC m=+0.015847659 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:05:03.477 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.23224637 +0000 UTC m=+0.120673796 container create 22d812f324c616d4454d1a68f0cd7def13248f5ccefbf29c243c7877c62c1e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T23:05:03.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:03.346+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:03.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:03.346+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:03.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:03.346+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:03.645 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:03.346+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:03.776 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.476230199 +0000 UTC m=+0.364657626 container init 22d812f324c616d4454d1a68f0cd7def13248f5ccefbf29c243c7877c62c1e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T23:05:03.777 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.479522032 +0000 UTC m=+0.367949477 container start 22d812f324c616d4454d1a68f0cd7def13248f5ccefbf29c243c7877c62c1e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:05:03.777 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.559578708 +0000 UTC m=+0.448006134 container attach 22d812f324c616d4454d1a68f0cd7def13248f5ccefbf29c243c7877c62c1e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T23:05:03.777 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 23:05:03 smithi117 podman[70747]: 2024-09-10 23:05:03.670323501 +0000 UTC m=+0.558750940 container died 22d812f324c616d4454d1a68f0cd7def13248f5ccefbf29c243c7877c62c1e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T23:05:04.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:03 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:03.611+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:04.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:03 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:03.611+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:04.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:03 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:03.611+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:04.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:03 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:03.611+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:04.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:03 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:03.611+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:04.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:03.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:04.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:03.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:04.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:03.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:04.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:03 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:03.644+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:04.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:03 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:03.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:04.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:03 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:03.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:04.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:03 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:03.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:04.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:03 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:03.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:04.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:03 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:03.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:04.364 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T23:05:04.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:04.405 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T23:05:04.405 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T23:05:04.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.8 2024-09-10T23:05:04.687 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 systemd[1]: Stopping Ceph osd.8 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:04.687 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:04.340+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:04.687 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:04.340+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:04.687 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:04.340+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:04.687 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:04.341+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:05.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:04.614+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:05.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:04.614+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:05.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:04.614+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:05.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:04.614+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:05.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:04 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:04.614+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.686+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.686+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.686+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.686+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.729+0000 7fc629dc2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:05.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.729+0000 7fc629dc2640 -1 osd.8 80 *** Got signal Terminated *** 2024-09-10T23:05:05.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:04 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:04.729+0000 7fc629dc2640 -1 osd.8 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:05:05.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:04.980+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:05.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:04.980+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:05.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:04.980+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:05.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:04.980+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:05.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:04.980+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:05.712 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:05.380+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:05.712 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:05.380+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:05.712 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:05.380+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:05.712 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:05.380+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:06.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:05 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:05.623+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:06.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:05 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:05.623+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:06.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:05 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:05.623+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:06.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:05 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:05.623+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:06.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:05 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:05.623+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:06.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:05.710+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:06.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:05.710+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:06.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:05.710+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:06.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:05 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:05.710+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:06.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:05 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:05.935+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:06.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:05 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:05.935+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:06.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:05 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:05.935+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:06.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:05 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:05.935+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:06.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:05 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:05.935+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:06.675 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:06.363+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:06.675 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:06.363+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:06.676 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:06.363+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:06.676 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:06.363+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:07.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:06.659+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:07.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:06.659+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:07.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:06.659+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:07.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:06.659+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:07.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:06 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:06.659+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:07.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:06.674+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:07.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:06.674+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:07.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:06.674+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:07.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:06 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:06.674+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:07.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:06 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:06.943+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:07.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:06 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:06.943+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:07.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:06 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:06.943+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:07.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:06 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:06.943+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:07.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:06 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:06.943+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:07.667 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:07.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:07.667 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:07.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:07.667 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:07.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:07.667 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:07.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:07.667 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:07.320+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:08.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:07 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:07.642+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:08.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:07 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:07.642+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:08.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:07 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:07.642+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:08.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:07 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:07.642+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:08.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:07 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:07.642+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:08.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:07.666+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:08.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:07.666+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:08.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:07.666+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:08.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:07 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:07.666+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:08.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:07.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:08.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:07.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:08.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:07.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:08.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:07.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:08.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:07 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:07.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:08.705 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:08.313+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:08.705 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:08.313+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:08.705 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:08.313+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:08.705 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:08.313+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:08.705 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:08.313+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:09.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:08 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:08.636+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:09.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:08.704+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.872872+0000 front 2024-09-10T23:04:11.872838+0000 (oldest deadline 2024-09-10T23:04:34.172880+0000) 2024-09-10T23:05:09.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:08.704+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:21.074594+0000 front 2024-09-10T23:04:21.074539+0000 (oldest deadline 2024-09-10T23:04:46.374740+0000) 2024-09-10T23:05:09.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:08.704+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.076047+0000 front 2024-09-10T23:04:28.076031+0000 (oldest deadline 2024-09-10T23:04:53.375919+0000) 2024-09-10T23:05:09.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:08.704+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:35.076988+0000 front 2024-09-10T23:04:35.077019+0000 (oldest deadline 2024-09-10T23:04:59.776893+0000) 2024-09-10T23:05:09.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:08 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8[53645]: 2024-09-10T23:05:08.704+0000 7fc6223cf640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:44.878774+0000 front 2024-09-10T23:04:44.878814+0000 (oldest deadline 2024-09-10T23:05:07.778849+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:09.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:08 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:08.898+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:09.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:09.348+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:09.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:09.348+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:09.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:09.348+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:09.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:09.348+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:09.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:09.348+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:10.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:09 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:09.658+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:10.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:09 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:09.934+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:10 smithi121 podman[71964]: 2024-09-10 23:05:10.170256826 +0000 UTC m=+5.576767473 container died e577a60d4c791188b06e585457d6bfdebe2b00ee95234d0567366e3e1231f715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:10 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:10.327+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:10 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:10.327+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:10 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:10.327+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:10 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:10.327+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:10.479 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:10 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:10.327+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:10.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:10 smithi121 podman[71964]: 2024-09-10 23:05:10.585115992 +0000 UTC m=+5.991626638 container cleanup e577a60d4c791188b06e585457d6bfdebe2b00ee95234d0567366e3e1231f715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:05:10.763 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:10 smithi121 bash[71964]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:11.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:10 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:10.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:11.192 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:10 smithi121 podman[71979]: 2024-09-10 23:05:10.836408467 +0000 UTC m=+0.661544355 container remove e577a60d4c791188b06e585457d6bfdebe2b00ee95234d0567366e3e1231f715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T23:05:11.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:11.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:11.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:11.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:11.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:11.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:10 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:10.956+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:11.513 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.09483092 +0000 UTC m=+0.015910920 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.235334476 +0000 UTC m=+0.156414468 container create 2f4ce6b853d7dddddfad4d55317778c5e3a8875f2561ec4cb628a71217927208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:11 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:11.324+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:11 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:11.324+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:11 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:11.324+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:11 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:11.324+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:11.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:11 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:11.324+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:12.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:11 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:11.645+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:12.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.515052556 +0000 UTC m=+0.436132547 container init 2f4ce6b853d7dddddfad4d55317778c5e3a8875f2561ec4cb628a71217927208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T23:05:12.013 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.518371931 +0000 UTC m=+0.439451923 container start 2f4ce6b853d7dddddfad4d55317778c5e3a8875f2561ec4cb628a71217927208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:05:12.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.590256286 +0000 UTC m=+0.511336289 container attach 2f4ce6b853d7dddddfad4d55317778c5e3a8875f2561ec4cb628a71217927208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T23:05:12.014 INFO:journalctl@ceph.osd.8.smithi121.stdout:Sep 10 23:05:11 smithi121 podman[72069]: 2024-09-10 23:05:11.709294008 +0000 UTC m=+0.630374000 container died 2f4ce6b853d7dddddfad4d55317778c5e3a8875f2561ec4cb628a71217927208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:12.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:11 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:11.953+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:12.411 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:12.343+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:12.411 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:12.343+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:12.411 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:12.343+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:12.411 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:12.343+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:12.411 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:12.343+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:12.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T23:05:12.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:12.474 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T23:05:12.474 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T23:05:12.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.9 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 systemd[1]: Stopping Ceph osd.9 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.621+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.800+0000 7f5d03d8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.800+0000 7f5d03d8e640 -1 osd.9 80 *** Got signal Terminated *** 2024-09-10T23:05:12.802 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:12 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:12.800+0000 7f5d03d8e640 -1 osd.9 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:05:13.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:13.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:13.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:13.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:13.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:13.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:12 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:12.966+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:13.638 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:13 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:13.333+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:13.638 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:13 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:13.333+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:13.638 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:13 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:13.333+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:13.638 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:13 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:13.333+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:13.638 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:13 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:13.333+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:14.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:13 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:13.654+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:14.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:13 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:13.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:14.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:14 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:14.357+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:14.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:14 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:14.357+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:14.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:14 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:14.357+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:14.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:14 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:14.357+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:14.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:14 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:14.357+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:15.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:14 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:14.687+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:15.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:15.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:15.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:15.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:15.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:15.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:14 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:14.950+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:15.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:15.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:15.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:15.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:15.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:15.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:15 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:15.361+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:16.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:15 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:15.650+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:16.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:15 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:15.946+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:16.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:16.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:16.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:16.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:16.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:16.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:16.356+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:17.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:16 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:16.689+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:17.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:16 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:16.914+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:17.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:17.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:17.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:17.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:17.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:17.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:17 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:17.312+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.506037+0000 front 2024-09-10T23:04:04.506061+0000 (oldest deadline 2024-09-10T23:04:27.406187+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:11.407714+0000 front 2024-09-10T23:04:11.407986+0000 (oldest deadline 2024-09-10T23:04:36.707630+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:17.808679+0000 front 2024-09-10T23:04:17.808638+0000 (oldest deadline 2024-09-10T23:04:43.708619+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:28.011430+0000 front 2024-09-10T23:04:28.011455+0000 (oldest deadline 2024-09-10T23:04:52.111543+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.212646+0000 front 2024-09-10T23:04:33.212738+0000 (oldest deadline 2024-09-10T23:04:59.112457+0000) 2024-09-10T23:05:18.001 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:17 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9[66764]: 2024-09-10T23:05:17.656+0000 7f5cfc39b640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:42.613752+0000 front 2024-09-10T23:04:42.613661+0000 (oldest deadline 2024-09-10T23:05:07.913501+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:18.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:17 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:17.897+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:18.692 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:18 smithi096 podman[117392]: 2024-09-10 23:05:18.354883569 +0000 UTC m=+5.673203897 container died e06eba0f77f083400d864c9969028fbc688ed89dcefad70d6223915f45bbd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:05:18.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:18.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:18.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:18.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:18.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:18.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:18 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:18.337+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:19.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:18 smithi096 podman[117392]: 2024-09-10 23:05:18.691986 +0000 UTC m=+6.010306314 container cleanup e06eba0f77f083400d864c9969028fbc688ed89dcefad70d6223915f45bbd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T23:05:19.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:18 smithi096 bash[117392]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9 2024-09-10T23:05:19.000 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:18 smithi096 podman[117407]: 2024-09-10 23:05:18.859880759 +0000 UTC m=+0.502504850 container remove e06eba0f77f083400d864c9969028fbc688ed89dcefad70d6223915f45bbd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:19.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:18 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:18.927+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:19.505 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.25244103 +0000 UTC m=+0.109759877 container create 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T23:05:19.505 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.158580512 +0000 UTC m=+0.015899360 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:05:19.505 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.50452084 +0000 UTC m=+0.361839684 container init 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:05:19.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:19.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:19.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:19.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:19.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:19.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:19 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:19.316+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:19.983 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.508114758 +0000 UTC m=+0.365433612 container start 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:05:19.983 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.587864908 +0000 UTC m=+0.445183760 container attach 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T23:05:19.983 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:19 smithi096 podman[117496]: 2024-09-10 23:05:19.696323732 +0000 UTC m=+0.553642590 container died 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:05:20.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T23:05:20.235 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:20 smithi096 podman[117519]: 2024-09-10 23:05:20.208263919 +0000 UTC m=+0.500045287 container remove 18c5e88d607616fcd17aa6e93644441cb4f54eab0a1ec9dbb55f34455f2dcf37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:05:20.235 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:20 smithi096 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.9.service: Deactivated successfully. 2024-09-10T23:05:20.235 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:20 smithi096 systemd[1]: Stopped Ceph osd.9 for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:05:20.235 INFO:journalctl@ceph.osd.9.smithi096.stdout:Sep 10 23:05:20 smithi096 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.9.service: Consumed 6min 46.912s CPU time. 2024-09-10T23:05:20.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:20.274 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T23:05:20.274 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T23:05:20.275 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.10 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:20.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:19 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:19.960+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:20.573 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 systemd[1]: Stopping Ceph osd.10 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:20.574 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.572+0000 7f4b4b3b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:20.574 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.572+0000 7f4b4b3b7640 -1 osd.10 80 *** Got signal Terminated *** 2024-09-10T23:05:20.574 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.572+0000 7f4b4b3b7640 -1 osd.10 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:05:20.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:20.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:20.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:20.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:20.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:20.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:20.283+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:21.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:20 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:20.915+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:21.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:21.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:21.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:21.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:21.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:21.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:21 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:21.298+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:22.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:22.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:22.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:22.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:22.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:22.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:22.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:21 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:21.964+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:22.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:22.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:22.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:22.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:22.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:22.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:22 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:22.296+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:23.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:22 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:22.936+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:23.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:23.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:23.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:23.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:23.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:23.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:23 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:23.307+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:24.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:24.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:24.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:24.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:24.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:24.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:24.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:23 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:23.917+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:24.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:24.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:24.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:24.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:24.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:24.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:24 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:24.286+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:25.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:01.011290+0000 front 2024-09-10T23:04:01.011123+0000 (oldest deadline 2024-09-10T23:04:26.911111+0000) 2024-09-10T23:05:25.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:09.811993+0000 front 2024-09-10T23:04:09.812059+0000 (oldest deadline 2024-09-10T23:04:35.112066+0000) 2024-09-10T23:05:25.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:19.713339+0000 front 2024-09-10T23:04:19.713358+0000 (oldest deadline 2024-09-10T23:04:46.714062+0000) 2024-09-10T23:05:25.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-09-10T23:04:26.714533+0000 front 2024-09-10T23:04:26.714610+0000 (oldest deadline 2024-09-10T23:04:50.214506+0000) 2024-09-10T23:05:25.276 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:33.715489+0000 front 2024-09-10T23:04:33.715457+0000 (oldest deadline 2024-09-10T23:04:57.815495+0000) 2024-09-10T23:05:25.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:45.317131+0000 front 2024-09-10T23:04:45.317105+0000 (oldest deadline 2024-09-10T23:05:08.817094+0000) 2024-09-10T23:05:25.277 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:24 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10[59492]: 2024-09-10T23:05:24.883+0000 7f4b479cc640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:52.818806+0000 front 2024-09-10T23:04:52.818645+0000 (oldest deadline 2024-09-10T23:05:18.718688+0000) 2024-09-10T23:05:25.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:25.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:25.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:25.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:25.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:25.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:25.764 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:25 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:25.287+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:26.381 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:26 smithi117 podman[70849]: 2024-09-10 23:05:26.070073094 +0000 UTC m=+5.601265376 container died dc5af199426d80bd902a6011c17c5ef106a6f1e1750a0c6c041923c8b2d1075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T23:05:26.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:26.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:26 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:26.239+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:26.715 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:26 smithi117 podman[70849]: 2024-09-10 23:05:26.480219674 +0000 UTC m=+6.011411958 container cleanup dc5af199426d80bd902a6011c17c5ef106a6f1e1750a0c6c041923c8b2d1075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T23:05:26.716 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:26 smithi117 bash[70849]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10 2024-09-10T23:05:27.026 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:26 smithi117 podman[70866]: 2024-09-10 23:05:26.714807683 +0000 UTC m=+0.645528428 container remove dc5af199426d80bd902a6011c17c5ef106a6f1e1750a0c6c041923c8b2d1075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T23:05:27.327 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:26.978861918 +0000 UTC m=+0.019085901 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:05:27.327 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:27.082367552 +0000 UTC m=+0.122591531 container create 7cdc55aad210830bb872f3ba86f439dbc2320103a490693672f1c013f91e787a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T23:05:27.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:27.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:27.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:27.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:27.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:27.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:27.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:27 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:27.242+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:27.584 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:27.32643456 +0000 UTC m=+0.366658552 container init 7cdc55aad210830bb872f3ba86f439dbc2320103a490693672f1c013f91e787a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T23:05:27.584 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:27.330123152 +0000 UTC m=+0.370347134 container start 7cdc55aad210830bb872f3ba86f439dbc2320103a490693672f1c013f91e787a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T23:05:27.585 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:27.40946434 +0000 UTC m=+0.449688334 container attach 7cdc55aad210830bb872f3ba86f439dbc2320103a490693672f1c013f91e787a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T23:05:27.585 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 10 23:05:27 smithi117 podman[70960]: 2024-09-10 23:05:27.521454805 +0000 UTC m=+0.561678785 container died 7cdc55aad210830bb872f3ba86f439dbc2320103a490693672f1c013f91e787a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T23:05:28.151 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T23:05:28.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:28.190 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T23:05:28.191 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T23:05:28.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.11 2024-09-10T23:05:28.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.215+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:28.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 systemd[1]: Stopping Ceph osd.11 for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:29.013 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.527+0000 7f3f072d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:29.013 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.527+0000 7f3f072d5640 -1 osd.11 80 *** Got signal Terminated *** 2024-09-10T23:05:29.014 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:28 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:28.527+0000 7f3f072d5640 -1 osd.11 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T23:05:29.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:29.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:29.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:29.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:29.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:29.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:29.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:29 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:29.260+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:30.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:30.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:30.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:30.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:30.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:30.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:30.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:30 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:30.244+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:31.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:31.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:31.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:31.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:31.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:31.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:31.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:31 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:31.232+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:32.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:32.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:32.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:32.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:32.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:32.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:32.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:32 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:32.212+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:33.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-09-10T23:04:04.475768+0000 front 2024-09-10T23:04:04.475815+0000 (oldest deadline 2024-09-10T23:04:26.175841+0000) 2024-09-10T23:05:33.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-10T23:04:12.076826+0000 front 2024-09-10T23:04:12.076979+0000 (oldest deadline 2024-09-10T23:04:35.576886+0000) 2024-09-10T23:05:33.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2024-09-10T23:04:20.877852+0000 front 2024-09-10T23:04:20.877914+0000 (oldest deadline 2024-09-10T23:04:41.977941+0000) 2024-09-10T23:05:33.513 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-10T23:04:36.981350+0000 front 2024-09-10T23:04:36.981498+0000 (oldest deadline 2024-09-10T23:05:01.681365+0000) 2024-09-10T23:05:33.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2024-09-10T23:04:41.681811+0000 front 2024-09-10T23:04:41.681843+0000 (oldest deadline 2024-09-10T23:05:06.381869+0000) 2024-09-10T23:05:33.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-09-10T23:04:50.983384+0000 front 2024-09-10T23:04:50.983449+0000 (oldest deadline 2024-09-10T23:05:15.083400+0000) 2024-09-10T23:05:33.514 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:33 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11[60655]: 2024-09-10T23:05:33.175+0000 7f3f038ea640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-10T23:04:59.784368+0000 front 2024-09-10T23:04:59.784297+0000 (oldest deadline 2024-09-10T23:05:24.984855+0000) 2024-09-10T23:05:34.347 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:34 smithi121 podman[72188]: 2024-09-10 23:05:34.058252943 +0000 UTC m=+5.672083445 container died abd3d30330c45b50d3c50c3d19da5ec2124811430c85837e88ecf9988bb2d6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T23:05:34.688 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:34 smithi121 podman[72188]: 2024-09-10 23:05:34.445480745 +0000 UTC m=+6.059311245 container cleanup abd3d30330c45b50d3c50c3d19da5ec2124811430c85837e88ecf9988bb2d6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T23:05:34.689 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:34 smithi121 bash[72188]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11 2024-09-10T23:05:35.013 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:34 smithi121 podman[72204]: 2024-09-10 23:05:34.688322121 +0000 UTC m=+0.629626105 container remove abd3d30330c45b50d3c50c3d19da5ec2124811430c85837e88ecf9988bb2d6eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T23:05:35.375 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:34.960528276 +0000 UTC m=+0.015875774 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:05:35.375 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:35.072589401 +0000 UTC m=+0.127936892 container create 849e6d6508ae59f30fb33514aabbb977fd0c9200aa2f3d1d9672541aeed61898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T23:05:35.375 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:35.299799321 +0000 UTC m=+0.355146815 container init 849e6d6508ae59f30fb33514aabbb977fd0c9200aa2f3d1d9672541aeed61898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T23:05:35.375 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:35.303067719 +0000 UTC m=+0.358415210 container start 849e6d6508ae59f30fb33514aabbb977fd0c9200aa2f3d1d9672541aeed61898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:05:35.375 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:35.374681779 +0000 UTC m=+0.430029280 container attach 849e6d6508ae59f30fb33514aabbb977fd0c9200aa2f3d1d9672541aeed61898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T23:05:35.763 INFO:journalctl@ceph.osd.11.smithi121.stdout:Sep 10 23:05:35 smithi121 podman[72292]: 2024-09-10 23:05:35.494774124 +0000 UTC m=+0.550121618 container died 849e6d6508ae59f30fb33514aabbb977fd0c9200aa2f3d1d9672541aeed61898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T23:05:36.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T23:05:36.135 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:36.135 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T23:05:36.136 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T23:05:36.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.a 2024-09-10T23:05:36.435 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:36 smithi096 systemd[1]: Stopping Ceph mds.a for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:36.749 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: 2024-09-10T23:05:36.433+0000 7f806424a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:36.750 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:36 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a[73514]: 2024-09-10T23:05:36.433+0000 7f806424a640 -1 mds.a *** got signal Terminated *** 2024-09-10T23:05:39.250 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:38 smithi096 podman[117610]: 2024-09-10 23:05:38.966475926 +0000 UTC m=+2.641476726 container died 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:05:39.668 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:39 smithi096 podman[117610]: 2024-09-10 23:05:39.441653242 +0000 UTC m=+3.116654084 container cleanup 27c924faf23d843ebabebcfe40f2aed37ae1eae8b75c810233b2525babe34763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T23:05:39.668 INFO:journalctl@ceph.mds.a.smithi096.stdout:Sep 10 23:05:39 smithi096 bash[117610]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-a 2024-09-10T23:05:39.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T23:05:39.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:39.855 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T23:05:39.855 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T23:05:39.855 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.d 2024-09-10T23:05:40.250 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:39 smithi096 systemd[1]: Stopping Ceph mds.d for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:40.250 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: 2024-09-10T23:05:40.171+0000 7f062fe9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:40.250 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:40 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d[72298]: 2024-09-10T23:05:40.171+0000 7f062fe9b640 -1 mds.d *** got signal Terminated *** 2024-09-10T23:05:43.213 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:42 smithi096 podman[117728]: 2024-09-10 23:05:42.958356342 +0000 UTC m=+2.881360670 container died 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T23:05:43.499 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 podman[117728]: 2024-09-10 23:05:43.387425411 +0000 UTC m=+3.310429772 container cleanup 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T23:05:43.500 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 bash[117728]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d 2024-09-10T23:05:43.826 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T23:05:43.847 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 podman[117739]: 2024-09-10 23:05:43.588731861 +0000 UTC m=+0.630132350 container remove 0c1d29f176c576efca1af4bbba6af38afb614661773355c8a8745c67415ddd4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T23:05:43.848 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-10T23:05:43.848 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 systemd[1]: Stopped Ceph mds.d for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:05:43.848 INFO:journalctl@ceph.mds.d.smithi096.stdout:Sep 10 23:05:43 smithi096 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.d.service: Consumed 2.320s CPU time. 2024-09-10T23:05:43.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:43.859 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T23:05:43.859 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T23:05:43.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.g 2024-09-10T23:05:44.159 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:43 smithi096 systemd[1]: Stopping Ceph mds.g for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:44.500 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: 2024-09-10T23:05:44.157+0000 7f1020ff4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:44.500 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:44 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g[73177]: 2024-09-10T23:05:44.157+0000 7f1020ff4640 -1 mds.g *** got signal Terminated *** 2024-09-10T23:05:47.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:46 smithi096 podman[117846]: 2024-09-10 23:05:46.955418205 +0000 UTC m=+2.880985010 container died 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T23:05:47.595 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:47 smithi096 podman[117846]: 2024-09-10 23:05:47.340959453 +0000 UTC m=+3.266526257 container cleanup 4644660f4c7a5641f42449d253700f5255ca5125456da10ec5813718ffbbc08b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T23:05:47.595 INFO:journalctl@ceph.mds.g.smithi096.stdout:Sep 10 23:05:47 smithi096 bash[117846]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-g 2024-09-10T23:05:47.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T23:05:47.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:47.777 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T23:05:47.777 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T23:05:47.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.j 2024-09-10T23:05:48.250 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:47 smithi096 systemd[1]: Stopping Ceph mds.j for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:48.250 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j[71897]: 2024-09-10T23:05:48.088+0000 7f40398e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:48.250 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:48 smithi096 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j[71897]: 2024-09-10T23:05:48.088+0000 7f40398e7640 -1 mds.j *** got signal Terminated *** 2024-09-10T23:05:52.106 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:51 smithi096 podman[117966]: 2024-09-10 23:05:51.839073125 +0000 UTC m=+3.845243006 container died 2c5fc78c04c67682355d06c26ade9c027bed10521dfacf27097ae6715217d76a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:05:52.500 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:52 smithi096 podman[117966]: 2024-09-10 23:05:52.297130936 +0000 UTC m=+4.303300816 container cleanup 2c5fc78c04c67682355d06c26ade9c027bed10521dfacf27097ae6715217d76a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T23:05:52.500 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:52 smithi096 bash[117966]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j 2024-09-10T23:05:52.792 INFO:journalctl@ceph.mds.j.smithi096.stdout:Sep 10 23:05:52 smithi096 podman[117979]: 2024-09-10 23:05:52.6211926 +0000 UTC m=+0.775071308 container remove 2c5fc78c04c67682355d06c26ade9c027bed10521dfacf27097ae6715217d76a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-j, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T23:05:52.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T23:05:52.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:52.830 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T23:05:52.830 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T23:05:52.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.b 2024-09-10T23:05:53.276 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:52 smithi117 systemd[1]: Stopping Ceph mds.b for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:53.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T23:05:53.327+0000 7feb6f726640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:53.776 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:53 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b[64979]: 2024-09-10T23:05:53.327+0000 7feb6f726640 -1 mds.b *** got signal Terminated *** 2024-09-10T23:05:55.276 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:54 smithi117 podman[71068]: 2024-09-10 23:05:54.95716446 +0000 UTC m=+1.937308780 container died 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:05:55.596 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 podman[71068]: 2024-09-10 23:05:55.394176944 +0000 UTC m=+2.374321262 container cleanup 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T23:05:55.596 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 bash[71068]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b 2024-09-10T23:05:55.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T23:05:55.849 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 podman[71081]: 2024-09-10 23:05:55.595708844 +0000 UTC m=+0.629689021 container remove 9f34090c0e367a65e8e281f77f95127d0f347f5478af0950fa69b6185d0f9338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:05:55.849 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-10T23:05:55.849 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 systemd[1]: Stopped Ceph mds.b for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:05:55.849 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 10 23:05:55 smithi117 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.b.service: Consumed 45.071s CPU time. 2024-09-10T23:05:55.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:55.862 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T23:05:55.862 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T23:05:55.863 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.e 2024-09-10T23:05:56.207 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:55 smithi117 systemd[1]: Stopping Ceph mds.e for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:05:56.526 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: 2024-09-10T23:05:56.206+0000 7fe4367b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:05:56.526 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:56 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e[64380]: 2024-09-10T23:05:56.206+0000 7fe4367b9640 -1 mds.e *** got signal Terminated *** 2024-09-10T23:05:59.241 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:58 smithi117 podman[71189]: 2024-09-10 23:05:58.961018307 +0000 UTC m=+2.872140521 container died ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-10T23:05:59.526 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:59 smithi117 podman[71189]: 2024-09-10 23:05:59.364628653 +0000 UTC m=+3.275750858 container cleanup ae7683f8f0b49d013735b78a35a151b23602c5eccc1744482b0f6f10a31c5ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T23:05:59.526 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 10 23:05:59 smithi117 bash[71189]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-e 2024-09-10T23:05:59.781 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T23:05:59.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:05:59.816 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T23:05:59.817 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T23:05:59.817 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.h 2024-09-10T23:06:00.128 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:05:59 smithi117 systemd[1]: Stopping Ceph mds.h for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:00.128 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:06:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: 2024-09-10T23:06:00.126+0000 7fab5feab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:00.128 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:06:00 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h[64679]: 2024-09-10T23:06:00.126+0000 7fab5feab640 -1 mds.h *** got signal Terminated *** 2024-09-10T23:06:03.261 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:06:02 smithi117 podman[71321]: 2024-09-10 23:06:02.962805451 +0000 UTC m=+2.933257781 container died 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T23:06:03.526 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:06:03 smithi117 podman[71321]: 2024-09-10 23:06:03.33447852 +0000 UTC m=+3.304930848 container cleanup 03196e6d22be657600a98a2e4b73ece538d2cf8044b759934eff2e0989f07e8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:06:03.526 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 10 23:06:03 smithi117 bash[71321]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-h 2024-09-10T23:06:03.764 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T23:06:03.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:03.798 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T23:06:03.798 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T23:06:03.798 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.k 2024-09-10T23:06:04.025 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:03 smithi117 systemd[1]: Stopping Ceph mds.k for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:04.526 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k[64069]: 2024-09-10T23:06:04.159+0000 7f82015fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:04.526 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:04 smithi117 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k[64069]: 2024-09-10T23:06:04.159+0000 7f82015fe640 -1 mds.k *** got signal Terminated *** 2024-09-10T23:06:08.276 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:08 smithi117 podman[71442]: 2024-09-10 23:06:08.172120094 +0000 UTC m=+4.157676287 container died 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T23:06:08.776 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:08 smithi117 podman[71442]: 2024-09-10 23:06:08.531570564 +0000 UTC m=+4.517126747 container cleanup 134167cf3d4cc06e61cf4ac2a3dfb9997ecf04b45337fb805f8ce32e813f0b93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T23:06:08.776 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 10 23:06:08 smithi117 bash[71442]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-k 2024-09-10T23:06:08.951 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T23:06:08.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:08.984 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T23:06:08.984 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T23:06:08.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.c 2024-09-10T23:06:09.263 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:09 smithi121 systemd[1]: Stopping Ceph mds.c for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:09.520 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: 2024-09-10T23:06:09.392+0000 7f9867a9a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:09.521 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:09 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c[65565]: 2024-09-10T23:06:09.392+0000 7f9867a9a640 -1 mds.c *** got signal Terminated *** 2024-09-10T23:06:11.243 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:10 smithi121 podman[72393]: 2024-09-10 23:06:10.950520974 +0000 UTC m=+1.768526636 container died 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:06:11.513 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:11 smithi121 podman[72393]: 2024-09-10 23:06:11.342223418 +0000 UTC m=+2.160229075 container cleanup 0352284e6c402131e53459d6476b06aeee881758ff2a9f3dceb5c286f192b3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T23:06:11.513 INFO:journalctl@ceph.mds.c.smithi121.stdout:Sep 10 23:06:11 smithi121 bash[72393]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-c 2024-09-10T23:06:11.708 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T23:06:11.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:11.744 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T23:06:11.744 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T23:06:11.744 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.f 2024-09-10T23:06:12.072 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:11 smithi121 systemd[1]: Stopping Ceph mds.f for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:12.513 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: 2024-09-10T23:06:12.070+0000 7f5c7d5b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:12.513 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:12 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f[65264]: 2024-09-10T23:06:12.070+0000 7f5c7d5b2640 -1 mds.f *** got signal Terminated *** 2024-09-10T23:06:15.230 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:14 smithi121 podman[72525]: 2024-09-10 23:06:14.958551494 +0000 UTC m=+2.999002469 container died 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T23:06:15.513 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:15 smithi121 podman[72525]: 2024-09-10 23:06:15.312298613 +0000 UTC m=+3.352749598 container cleanup 3e485c048c2d518cc0d6306570a193e0186f0ececf84af76e16d2fd77552cf8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T23:06:15.513 INFO:journalctl@ceph.mds.f.smithi121.stdout:Sep 10 23:06:15 smithi121 bash[72525]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-f 2024-09-10T23:06:15.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T23:06:15.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:15.775 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T23:06:15.775 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T23:06:15.775 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.i 2024-09-10T23:06:16.263 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:15 smithi121 systemd[1]: Stopping Ceph mds.i for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:16.263 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: 2024-09-10T23:06:16.099+0000 7effd7b51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:16.263 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:16 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i[65866]: 2024-09-10T23:06:16.099+0000 7effd7b51640 -1 mds.i *** got signal Terminated *** 2024-09-10T23:06:19.263 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:18 smithi121 podman[72646]: 2024-09-10 23:06:18.947807887 +0000 UTC m=+2.954019528 container died 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T23:06:19.575 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 podman[72646]: 2024-09-10 23:06:19.390909714 +0000 UTC m=+3.397121360 container cleanup 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:06:19.576 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 bash[72646]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i 2024-09-10T23:06:19.808 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T23:06:19.829 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 podman[72661]: 2024-09-10 23:06:19.575242193 +0000 UTC m=+0.620451767 container remove 68c6abbe83e76f6777e6f6000cb5c977a2914c30016a7a0beb1fe211d87b0c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:06:19.829 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-10T23:06:19.829 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 systemd[1]: Stopped Ceph mds.i for 14afb286-6fc5-11ef-bcea-c7b262605968. 2024-09-10T23:06:19.829 INFO:journalctl@ceph.mds.i.smithi121.stdout:Sep 10 23:06:19 smithi121 systemd[1]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.i.service: Consumed 2.327s CPU time. 2024-09-10T23:06:19.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:19.840 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T23:06:19.840 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T23:06:19.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.l 2024-09-10T23:06:20.146 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:19 smithi121 systemd[1]: Stopping Ceph mds.l for 14afb286-6fc5-11ef-bcea-c7b262605968... 2024-09-10T23:06:20.513 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: 2024-09-10T23:06:20.144+0000 7f76eaf13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:06:20.513 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:20 smithi121 ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l[64962]: 2024-09-10T23:06:20.144+0000 7f76eaf13640 -1 mds.l *** got signal Terminated *** 2024-09-10T23:06:23.232 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:22 smithi121 podman[72766]: 2024-09-10 23:06:22.958987119 +0000 UTC m=+2.896814109 container died 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T23:06:23.513 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:23 smithi121 podman[72766]: 2024-09-10 23:06:23.328107625 +0000 UTC m=+3.265934602 container cleanup 8ca6ea62709aec84c1108c1b5612135dc7b678cd0c310c9b619bc30e733cb298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T23:06:23.513 INFO:journalctl@ceph.mds.l.smithi121.stdout:Sep 10 23:06:23 smithi121 bash[72766]: ceph-14afb286-6fc5-11ef-bcea-c7b262605968-mds-l 2024-09-10T23:06:23.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-14afb286-6fc5-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T23:06:23.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T23:06:23.779 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T23:06:23.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:06:23.983 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:06:26.385 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:06:26.594 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:06:29.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T23:06:29.326 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:06:31.678 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:06:31.711 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:06:31.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:06:31.774 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T23:06:31.776 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi096/crash 2024-09-10T23:06:31.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:06:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:06:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:06:31.818 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi117/crash 2024-09-10T23:06:31.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:06:31.869 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:06:31.869 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:06:31.871 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi121/crash 2024-09-10T23:06:31.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash -- . 2024-09-10T23:06:31.904 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T23:06:31.904 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2024-09-10T23:06:31.905 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T23:06:31.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T23:06:31.949 INFO:tasks.cephadm:Compressing logs... 2024-09-10T23:06:31.950 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:06:31.992 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:06:31.995 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:06:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T23:06:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T23:06:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T23:06:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:06:32.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:06:32.019 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:: No such file or directory 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T23:06:32.023 INFO:teuthology.orchestra.run.smithi121.stderr: 82.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:06:32.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T23:06:32.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:06:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T23:06:32.027 INFO:teuthology.orchestra.run.smithi121.stderr: 90.4% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:06:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T23:06:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log: 87.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:06:32.027 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T23:06:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log: 87.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:06:32.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T23:06:32.032 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T23:06:32.032 INFO:teuthology.orchestra.run.smithi117.stderr:: No such file or directory 2024-09-10T23:06:32.032 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T23:06:32.032 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T23:06:32.032 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T23:06:32.033 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:06:32.034 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:06:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log: 90.8% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:06:32.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T23:06:32.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log: 90.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:06:32.036 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T23:06:32.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log: 92.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T23:06:32.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T23:06:32.047 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T23:06:32.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:06:32.056 INFO:teuthology.orchestra.run.smithi117.stderr: 92.8% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T23:06:32.060 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T23:06:32.064 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T23:06:32.072 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:06:32.087 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T23:06:32.087 INFO:teuthology.orchestra.run.smithi117.stderr: 82.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T23:06:32.092 INFO:teuthology.orchestra.run.smithi117.stderr: 90.4% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T23:06:32.092 INFO:teuthology.orchestra.run.smithi117.stderr: 87.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T23:06:32.133 INFO:teuthology.orchestra.run.smithi096.stderr: 94.3% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T23:06:32.135 INFO:teuthology.orchestra.run.smithi121.stderr: 94.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T23:06:32.138 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T23:06:32.142 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T23:06:32.184 INFO:teuthology.orchestra.run.smithi121.stderr: 92.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T23:06:32.205 INFO:teuthology.orchestra.run.smithi121.stderr: 92.7% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T23:06:32.208 INFO:teuthology.orchestra.run.smithi117.stderr: 95.1% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T23:06:32.251 INFO:teuthology.orchestra.run.smithi117.stderr: 92.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T23:06:32.286 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T23:06:32.294 INFO:teuthology.orchestra.run.smithi117.stderr: 90.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T23:06:32.407 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T23:06:32.444 INFO:teuthology.orchestra.run.smithi121.stderr: 91.6% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T23:06:32.534 INFO:teuthology.orchestra.run.smithi117.stderr: 91.5% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T23:06:32.743 INFO:teuthology.orchestra.run.smithi121.stderr: 94.3% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T23:06:33.527 INFO:teuthology.orchestra.run.smithi096.stderr: 90.1% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T23:06:55.905 INFO:teuthology.orchestra.run.smithi117.stderr: 93.8% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T23:07:07.178 INFO:teuthology.orchestra.run.smithi096.stderr: 91.2% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T23:07:12.027 INFO:teuthology.orchestra.run.smithi121.stderr: 91.1% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T23:07:14.793 INFO:teuthology.orchestra.run.smithi117.stderr: 91.4% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T23:07:16.934 INFO:teuthology.orchestra.run.smithi121.stderr: 91.3% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T23:07:17.446 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T23:07:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 91.2% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T23:07:23.247 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T23:07:27.037 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T23:07:27.039 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-10T23:07:27.039 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m55.030s 2024-09-10T23:07:27.039 INFO:teuthology.orchestra.run.smithi121.stderr:user 3m3.770s 2024-09-10T23:07:27.039 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m5.649s 2024-09-10T23:07:30.941 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T23:07:33.966 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T23:07:33.968 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-10T23:07:33.968 INFO:teuthology.orchestra.run.smithi096.stderr:real 1m1.962s 2024-09-10T23:07:33.968 INFO:teuthology.orchestra.run.smithi096.stderr:user 3m16.994s 2024-09-10T23:07:33.968 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m6.198s 2024-09-10T23:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T23:07:36.337 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/14afb286-6fc5-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T23:07:36.340 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-10T23:07:36.340 INFO:teuthology.orchestra.run.smithi117.stderr:real 1m4.330s 2024-09-10T23:07:36.340 INFO:teuthology.orchestra.run.smithi117.stderr:user 3m51.239s 2024-09-10T23:07:36.340 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m6.847s 2024-09-10T23:07:36.340 INFO:tasks.cephadm:Archiving logs... 2024-09-10T23:07:36.341 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi096/log 2024-09-10T23:07:36.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:08:11.014 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi117/log 2024-09-10T23:08:11.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:08:49.084 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi121/log 2024-09-10T23:08:49.085 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:09:22.269 INFO:tasks.cephadm:Removing cluster... 2024-09-10T23:09:22.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:09:22.473 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:09:23.052 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:09:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:09:23.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 14afb286-6fc5-11ef-bcea-c7b262605968 --force 2024-09-10T23:09:24.085 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: 14afb286-6fc5-11ef-bcea-c7b262605968 2024-09-10T23:09:24.640 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T23:09:24.640 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:09:24.659 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:09:24.675 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T23:09:24.691 INFO:tasks.cephadm:Teardown complete 2024-09-10T23:09:24.691 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T23:09:24.702 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T23:09:24.703 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:09:24.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:09:24.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:09:24.856 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T23:09:24.866 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T23:09:24.866 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:09:24.869 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:09:24.871 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:09:24.883 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:09:24.884 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:09:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 -116us[ -52us] +/- 35ms 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +655us[ +720us] +/- 49ms 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -615us[ -615us] +/- 77ms 2024-09-10T23:09:24.888 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 -136us[ -126us] +/- 55ms 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 26 +675us[ +675us] +/- 49ms 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -623us[ -623us] +/- 77ms 2024-09-10T23:09:24.890 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 -94us[-5700ns] +/- 35ms 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +652us[ +740us] +/- 49ms 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -626us[ -626us] +/- 77ms 2024-09-10T23:09:24.891 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:09:24.892 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T23:09:24.901 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T23:09:24.902 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T23:09:24.940 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:09:25.030 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:09:25.313 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T23:09:25.382 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:09:25.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:25.416 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:09:25.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:25.444 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:09:25.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:25.472 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T23:09:25.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T23:09:25.533 INFO:teuthology.task.internal:Duration was 2349.971693 seconds 2024-09-10T23:09:25.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T23:09:25.574 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T23:09:25.574 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:09:25.577 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:09:25.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:09:25.616 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:09:25.618 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:09:25.621 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:09:26.051 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T23:09:26.051 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-10T23:09:26.052 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:09:26.072 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T23:09:26.073 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:09:26.099 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T23:09:26.099 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:09:26.126 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T23:09:26.126 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:09:26.129 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:09:26.142 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:09:26.194 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T23:09:26.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:09:26.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:09:26.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:09:27.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T23:09:27.621 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T23:09:27.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:09:27.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:09:27.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:09:27.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T23:09:27.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:27.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:27.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:27.742 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-10T23:09:27.749 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-10T23:09:27.775 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-09-10T23:09:28.041 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:28.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:28.056 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:28.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:28.075 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:09:28.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:09:28.101 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T23:09:28.112 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T23:09:28.114 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi096 2024-09-10T23:09:28.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:09:28.179 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi117 2024-09-10T23:09:28.179 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:09:28.220 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899739/remote/smithi121 2024-09-10T23:09:28.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:09:28.261 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T23:09:28.261 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:09:28.263 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:09:28.266 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:09:28.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T23:09:28.322 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T23:09:28.322 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T23:09:28.350 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T23:09:28.351 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:09:28.353 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:09:28.355 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:09:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: 265542 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:09 /home/ubuntu/cephtest 2024-09-10T23:09:28.370 INFO:teuthology.orchestra.run.smithi117.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:09 /home/ubuntu/cephtest 2024-09-10T23:09:28.372 INFO:teuthology.orchestra.run.smithi121.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:09 /home/ubuntu/cephtest 2024-09-10T23:09:28.373 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T23:09:28.400 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T23:09:28.507 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 2349.971692800522 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T23:09:28.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:09:28.600 INFO:teuthology.run:pass